HFT trying to pick pockets from folks that don’t hodl
Sometimes I get asked by people if there is any chatroom that is any good. This is a very personal thing, a good room for a trader might be terrible for another. I can only talk about a room service that I use and that I learned a good amount from. Its the “Investors Underground” chat, I have been a member since 2014. There are a number of good solid winning traders there (E1CN, OzarkTrades and lx21 are excellent traders there), some long biased and some short biased. Sometimes they trade small caps, sometimes ETFs, sometimes bigger caps. Mostly, their niche seems to be small cap stock universe.
But the good thing is that in their service, there are actually 3 rooms. One for intraday momentum trading, one for swing trading and one for OTC stocks. These days I mostly hang around in the Swing room (I’m Bulrathi there, if you see me, feel free to say hi or ask questions). I’m currently working in transitioning from day trading to swing/position trading for lifestyle reasons and I’m learning a great deal by observing other swing traders kicking around ideas.
A room service is not to be used as an alert system where you immediatly follow other people into trades, its hard to make that style work because you always will get a worse price than the person you are following. Its more to observe how good traders think, manage trades and make decisions. Through observation you can develop your decision making by building a database of situations which you can then use in the future.
Its a tool that I find very useful for beginning and intermediate traders.
An updated version of my book (Traders of the New Era) is now avaliable on Kindle, as a Paperback and EPUB
In the expanded edition there are 3 additional interviews (The Home Run Swing Trader, The Futures Scalper and the Futures and Options Veteran), 1 extra chapter (The Trader Tool Box: Tricks, techniques and methods to fight against HFT) as well as additional improvements and corrections.
I have been absent from posting here exactly because I was working so hard in the expanded edition
Thanks all for the support
Ever since the publication of the book Dark Pools in 2012, it became public that ECNs and exchanges were accommodating HFTs firms by offering them special order types that most market participants don’t get. These order types enable HFTs to decrease the amount of “adverse selection” they are exposed to and to be at the top of the queue, key components of profitable market making.
Most HFTs firms have dozens (if not hundreds) of special order types at their disposal by exchanges and ECNs. Retail traders who want to be able to fight back and cut down their trading costs should learn and implement useful order types that are available to them.
There is an interesting tool that is not very well known by traders that can help you decrease the amount of HFT “gaming” your orders suffer. Hidden “native” orders are orders that sit on an ECN or exchange and that, in theory, are not seen by anyone. I say in theory because the exchanges/ECNs have broken the rules before in order to accommodate HFTs. It’s possible but unlikely that some of them have leaked information about hidden orders at some point.
These orders are different from iceberg/reserve orders, iceberg orders show up on the Level 2 window and on the direct feed data that HFTs receive. They usually appear as 100 shares and give HFTs an opportunity to “penny jump” the order by going in front of it by 1 cent. This exposes the order to adverse selection, meaning they will get out of the way when they see a shift in the short-term supply and demand and stay ahead of you by 1 or a few cents when they don’t. You tend to miss out on fills that you want and get fills that you don’t want.
Lots of traders report the experience of having a stock move down or up right to the point that their limit order is not get filled by one cent (or get partially filled). Sometimes this happens due to randomness, but a lot of the time this happens because HFTs use passive non-HFT orders as insurance. If they are bidding for a stock, they know that if they get ahead of you by 1 cent, they can get their fills and try to sell on the ask. If all goes well, they make the spread, if it doesn’t and the stock looks to drop, they dump their shares into your orders. You get your fill but now you are also underwater in the trade (or you left money on the table for a buy to cover order while exposing yourself to a squeeze).
A useful way to diminish this passive order gaming is to use “native” hidden orders. With these orders, the HFTs tend to only find out where your orders are after they have been executed (whether in full or partial). Let me give you an example: Let’s say you are short 1,000 shares of stock ABC, it’s trading at $35, it then proceeds to drop as you expected. You decide to take profits in the $34.60-$34.65 area because you think there will be some support at $34.50. You put a hidden buy order for 500 shares of ABC, native to ARCA at $34.64, 200 shares at $34.63 native to NSDQ and 300 native to NYSE at $34.63. When I say native to the ECN/exchange, I mean that it will only interact with the order flow of that ECN/exchange. For instance, if someone sells 500 shares of ABC at $34.64 at NYSE, you will not get filled because your hidden order was sitting at ARCA. Hidden orders can even be traded through, meaning, you can see prints bellow the price of your buy order. This disadvantage can be countered by using what I call a hidden order “net”. You use several ECNs/exchanges that have the most volume, instead of just one. That way you have multiple order flows to interact to.
By having your orders distributed that way, when the stock drops and stops in the $34.60s area, you will avoid being used as insurance for HFTs looking to make markets. If the stock really drops quickly, say from $35 to $34.50, it won’t make much of a difference whether you are using hidden or lit (displayed) orders. It’s on the marginal situations where your orders are on the bottom or close to the bottom that hidden orders can decrease your trading costs by exposing you to less adverse selection.
With the lit orders, having the HFTs penny jump you and prevent you from getting filled at or close to the bottom can cost you significant profits as the stock reverses higher and you are forced to hit the ask in the $34.70s area. With the hidden orders, the HFTs can still game you in a similar fashion but they can do it only after they find out that there are hidden orders at a certain level. In a fast moving stock with good volume, by the time they find out, it’s too late. Your order is likely to at least have been partially filled.
In addition, HFTs don’t like hidden orders because even though they can find out that they are at a certain level, they don’t know when they get cancelled or entirely filled and the free insurance is gone. They do like to use a trick against these orders, odd lot trades. Sometimes, in order to find out if there are hidden orders in an exchange/ECN they will send out orders for a tiny amount of shares (usually less than 50). When they discover the hidden liquidity, they can penny jump it or just keep sending odd lot orders to annoy and/or increase the trading costs of the trader behind it.
Routing a Hidden Order Net
Which routes to use to best increase the change your hidden order net will capture most of the volume in a particular stock? You have to be aware of where, currently in the US, most of the volume is going through. As of September of 2014, volume is distributed as follows:
Source: BATS Exchange.
By the table, you can see that in a NASDAQ stock (Tape C), NASDAQ and ARCA do most of the volume (over 37%). For a NYSE stock (Tape A), NYSE and ARCA do most of the volume (about 31%). For AMEX stocks (Tape C), ARCA and NASDAQ do most of the volume (about 38%).
When routing your net in a NASDAQ stock, the better routes are NASDAQ (sometimes still called Island by some brokers) and ARCA as a secondary route. When trading a NYSE stock, the better routes are NYSE and ARCA. For AMEX, ARCA is the main one and NASDAQ is a secondary route.
The Weaknesses of Hidden Orders
Hidden orders are not the Holy Grail, they just happen to be useful in certain conditions. In what situations are hidden orders not a great tool?
As a rule of thumb, if you have a large order or if the stock has low activity, hidden orders benefit is limited.
For large orders, having your net sit on the bid and ask for a long-time till it’s completely filled, might allow the HFTs to find out what you are doing with odd lot trades or just be looking at “phantom” trade executions. When they do find out, the amount of adverse selection you are exposed to increases because now they can penny jump you. The other weakness of these orders (not being exposed to all the order flow but only from the exchange/ECN you are native to) is still there. If you combine that with more adverse selection, that might have little or no added value over displayed orders.
In low activity stocks the problem is similar. It’s going to take longer for your order to be executed. There is more time for you to get hit with odd lot trades (which increases your costs if your broker has a minimum commission charge) and for HFTs to discover what you are doing.
When to use Hidden Orders
The ideal scenario for hidden orders are stocks with a lot of activity going off in the bid and ask, and a significant spread. This is usually the case in momentum stocks that are higher priced or some small caps.
The spread has to be large because otherwise it is just not worth it to try to save it. If you are trading something with a spread of 3 cents or less, you are better off just taking liquidity. If the spread is 10-20 cents or more, then it’s worth it. If fact, it can get quite expensive to hit that big of a spread over and over again.
Where hidden orders are really useful is when your stock is moving fast in a direction and you have a tough time with HFTs stopping right in front or order. In this case, having a net of smaller sized hidden orders nicely distributed in different exchanges and ECNs can pretty much guarantee that you will get fills with less adverse selection than otherwise would be the case.
Another situation where they can make you money is when you see a trade setup in a large spread stock that you believe is profitable only if you get in without paying the spread. If you get filled with the hidden order, that’s great, you get to participate in the trade. If you don’t (because the move happened before your fill), it’s not a big deal anyway because hitting the spread would turn the trade into a loser. You have a free shot at being in those trades. Over the course of a year, being in these trades can add up to a lot of money.
Pre-market and after hours
Pre-market and after hours trading is also a period where hidden orders can be a good tool in your tool box. First because usually the spreads are huge. Paying it without a piece of fresh news is extremely costly. What a trader can do is to put hidden orders in front of displayed orders in those sessions and participate in some of the “market making” that happens outside of regular trading hours. There won’t be a lot of activity or volume but the gain is so large that it can be worth it.
The way I play is to try to close my swing/overnight positions by having orders sitting at these extreme levels and checking the next day to see if I got a fill. Most of the time you won’t be but it’s so cheap to try it and occasionally you will get filled. Even odd lots trades are not a big deal there. If a stock closed at $40 and you get hit for 20 shares selling at $40.98, you are still profitable even after commissions.
Keep in mind that sometimes it’s better to have your order displayed instead of hidden. You want to show to someone nervous about their position, where the liquidity is so they can hit it.
Also remember that other people will be using hidden orders too, if your lit order has been sitting in the market for a while. It’s quite possible that there will be hidden orders in front of it at the same exchange/ECN.
Hidden orders are a useful tool in a trader’s toolbox if you know where to use them. They won’t solve all your problems or turn a losing trader into a profitable one. They can, however, decrease the amount of adverse selection your orders are exposed to. Especially in stocks with a good amount of activity but with larger spreads. If your order is not large and you choose good routes to send your hidden order net to, you are likely to decrease your trading costs as compared to using displayed orders. Ask your broker if they offer such option and how to use this order type.
Dark Pools recently have been subject of many headlines, mostly due to alleged illegal activities from the banks that operate them. For instance, Barclays is said have given High Frequency Traders (HFTs) unfair advantages over traders and investors in order to boost its trading activity and revenues.
In this article I’m going to explain what dark pools are, how they can be used by traders to reduce trading costs and when they should be avoided in order to achieve the same purpose.
First, what is a dark pool? The technical term for them are ATS (alternative trading system). They are essentially mini exchanges that match buy and sell orders without displaying them to the market. If you route a buy order (say, buy 50,000 shares of MSFT) to a dark pool, in theory, nobody knows your order is there. You have the possibility of getting filled for your entire order without “spooking” the market by displaying a large bid in the Level 2 Window. A lot of smart routers tend to go through Dark Pools first before they hit the displayed exchanges/ECNs, as a result your order would be exposed to that order flow before other participants. Another benefit of dark pools is that because the order is not displayed, sometimes you get the benefit of not having to hit the bid or ask in order to get your order executed. If you send a displayed order and have it appear in the Level 2, frequently, high frequency traders will jump ahead of you by one cent. This sort of “penny jumping” is done automatically by computers, while it has to be adjusted manually by us humans, this is annoying, time consuming and inefficient. By having that order in a dark pool, you can be in front of the HFT bids and asks without, in theory, they knowing about it and you can get the order flow of that dark pool to save you the spread.
Another benefit of dark pools is that when you send liquidity taking orders that go through them first, you tend to get access to additional liquidity not shown in the Level 2 that is inside the NBBO (National Best Bid and Offer). This tends to reduce your trading costs because it reduces the spread that you have to pay. Example: You try to buy 1,000 shares of PG at 80.02 in a market quoted at 80.01 x 80.02, your smart router hits a dark pool before the displayed market and gives back a fill for your entire order at 80.015. You just saved half a cent which comes out at $5. Over the course of a year, these savings add up.
Does that mean that dark pools are the Holy Grail for traders and investors to reduce their trading costs?
No. In addition to the unfair advantages that high frequency traders might be getting in dark pools, there are other issues even through legal means. For instance, Eric Hunsander from Nanex recently did an expose on how modern markets are rigged http://www.nanex.net/aqck2/4661.html
In it he demonstrated that a large order (in this case a buy order for 20,000 shares of Ford) alerted HFTs about his buying intentions because it hit dark pools first before the displayed markets. The HFTs then proceeded to cancel their orders from the displayed exchanges. The trader behind the order ended up only getting 12,133 shares (and 600 were from the dark pool). In this case, having the order go through the dark pool first was an extremely bad deal.
The question now is, if you are looking to decrease trading costs, when should a trader use dark pools and when he should avoid them?
It all depends on your order size. As a rule of thumb, if you are executing a small liquidity taking order (you are hitting the bid or ask), you want to go through as many dark pools as possible. The additional liquidity will give you fills inside the NBBO and reduce the spread that you pay. Some brokers allow you to expand the number of dark pools that your order will go through, being aware of this option can help you expose your order to dark pools when you think that is valuable. Ask your broker if they offer such option.
If you are sending a small liquidity providing order, you want to use a dark pool that has a lot of activity in that stock. This will enable you to not have to sit with a bid or offer for very long, get taken and save the spread. Not a lot of retail brokerages allow routing to dark pools. Ask your broker if they have such option.
On the order hand, if you are executing a liquidity taking large order, you want to go through as few dark pools as possible. As explained by Nanex and evidenced by a lot of frustrated traders, when HFTs notice dark pool activity a lot of the time, they tend to cancel their orders in the displayed market. HFTs know that dark pools are used by institutional traders looking for liquidity. Since institutions usually have a lot of stock to buy and sell, HFTs can make more by widening their spreads. In the case of large liquidity adding orders, it’s also problematic to use a dark pool. Barclays and Credit Suisse are both being probed for potential wrongdoing when dealing with institutional orders. And these are just two banks, there are dozens of dark pools out there. Every time you send your larger orders to dark pools, you run the risk of being taken advantage of by executives looking to increase their market share. The benefit of you having your block trade executed is significant, but so is the cost of having your intentions exposed to HFTs. Choosing the right dark pool becomes crucial.
To summarize, I created the following table of how traders and investors can use dark pools to decrease trading costs. The definitions of small, medium and large orders have to do with how much of a market impact you expect to have. For instance, if you want to buy 400 shares of ABC and there are 5,000 shares bid and 6,000 offered, you have a small order as this is not expected to have much of a market impact. If you are trying to buy 5,000 shares, it’s a large order. A medium sized order would be somewhere in between. There is a forth category for huge orders (say 50,000, 100,000 or even 1 million shares), these are institutional orders that require more advanced techniques to conceal your intentions and require multiple executions over many hours/days. Those techniques are beyond the scope of this article.
Keep in mind that some larger spread stocks without a lot of liquidity (typical of small caps), sometimes even something as little as 200-300 shares is expected to have market impact. Also, when I say Dark Pool, I also mean Dark liquidity like orders that are sent to internalizers.
Two other variables you have to take into account are, how quickly you want to get in or out of a stock, and how fast is the stock moving.
Brokers: Retail brokers like CenterPoint Securities give access to a lot of dark liquidity through their special routes like CPCIT(Citadel), CPKCG (Knight), CSFBDESK (Credit Suisse), and others. That extra liquidity can help when traders are looking for price improvement or for fills when dealing with the Short Sell Restriction.
Interactive Brokers has an option called “Seek Price Improvement” that is turned off by default. If your order is small, turning the option on can decrease your trading costs and expose you to additional intra NBBO liquidity.
Conclusion: Dark pool orders definitely have a place in a trader’s toolbox. Don’t be scared by the headlines you read on the financial websites. If you understand them and know how to use them, you can use your judgment of when they are likely to decrease or increase your trading costs. Being aware of how much market impact you are likely to have and how your broker routes your orders can further help you in decreasing those costs.
I was quoted in the September issue of the CFA magazine. Its an article where Dennis Dick goes through the main fundamentals of how to adapt to a HFT world
Here is why HFTs hate IEX: How to use IEX to improve your order execution, prevent HFTs from gaming you and save money in the process
Ever since Michael Lewis published his big best seller Flash Boys, an interesting exchange has gotten a lot of the public’s attention. IEX, or The Investors Exchange, was created by Brad Katsuyama and his colleagues from the Royal Bank of Canada(RBC) after they realized that the US stock market was severely lacking an exchange that would actually respect and be fair to the investors and traders who were bringing their business there. The main idea is that they will not allow special advantages to HFTs that are routing their orders to their exchange; the way they achieve that is through several methods:
- Limited amount of order types. Unlike the main exchanges that allow almost any kind of order type (even if unfair) to be created.
- No maker/taker rebates – a pricing system designed to give “kickbacks” to certain firms at the expense of other firms.
- No collocated HFTs. They deny HFTs the privilege of being collocated inside their exchange.
- A 350 microsecond latency for any order action taken by a participant (enter, cancel, revise an order), which ensures the IEX Matching Engine has sufficient time to receive and process changes to the NBBO (national best bid and offer) before a fast participant can receive and act upon the change on IEX.
This last point is a very important one. If you check out Eric Hunsader’s twitter feed and website (@nanexllc and nanex.net) he talks about all the time how HFTs can cancel orders faster than order routers can spread out their orders to different exchanges. This lets the HFTs widen out their spreads and make a little more on the transaction than otherwise would be the case.
How can a trader use IEX to its advantage and improve its order execution?
You see, the way IEX works, you can route an order, say buy 10,000 shares of PG, to IEX (which currently is an ATS, a dark pool, but soon it might become a displayed exchange) and if there are orders to sell PG there, you will get executed for whatever amount is resting there for up to 10,000 shares. If there isn’t the full amount that you want, they will re-route the balance of your order to the 11 registered displayed stock exchanges (3 from NYSE, 3 from NASDAQ, 4 from BATS and the Chicago Stock Exchange) plus the LavaFlow ECN. The thing is, Katsuyama and his colleagues are the folks who created Thor, the system that is used to combat HFTs games by getting orders to hit all the exchanges/ECNs at the same exact time to prevent them from cancelling or changing their orders after they discover your interest.
Because IEX is run by folks who know a lot about these HFT games and who were creative about how to work around it, you can expect that they will be intelligent about how IEX routes out your orders. This creates an interesting tool to traders: using IEX as a smart router. They have a proprietary method in which they re-route orders to other pools of liquidity (they have filed patent applications covering it). What they do disclose is the following:
- After leaving IEX, the order does not go through dark pools or internalizers first (which is the case with quite a number of retail brokerages’ smart routers). Frequently orders that hit dark pools tip off HFTs and allow them to cancel their orders in other places. Because routing to a dark pool is a shot in the dark, you don’t know how many or if there are any shares of your stock bid or offered there at all, your upside can be limited and the downside of alerting HFTs is significant.
- They have direct data feeds from the exchanges that they are trying to hit. This means that their order router is looking at the present rather than looking at the past which is the case with the SIP.
- They will hit those 11 exchanges plus LavaFlow over a custom-built dark fibre network (which is proprietary) that is designed as to not allow the HFTs there to cancel or modify orders in other places when they see the buy or sell interest.
- Their own 350 microsecond delay prevents HFTs from outracing IEX’s smart order router to the other displayed exchanges based on “signaling” from buy or sell interest on IEX.
IEX’s new order routing approach doesn’t use (or need) Thor for several reasons:
First because their technology is much better than it was at RBC. RBC, being a broker dealer, didn’t have much incentive to get the fastest connections to the exchanges or the best hardware. They weren’t in the “arms race” to zero latency, whereas IEX’s hardware and connections are as good or almost as good as the top HFT firms these days.
Secondly, because their router is centrally located among data centers in New Jersey, they don’t need to spread out orders in a Thor-like fashion because they are quite close to the other data centers (and the exchanges housed within).
At RBC, they were routing off lower Manhattan. Signals had to travel up the Hudson River, and across into New Jersey (NJ), before reaching each of the data centers sequentially: first Weekawken (BATS), then Secaucus (Direct Edge), then Cartaret (NASDAQ) and finally Mahwah (NYSE). The time it took to reach each data center was relatively consistent, and predictable. This enabled the HFTs to play their games.
By having their router close to the other data centers in NJ, there is no need for Thor anymore. Any difference in reaching different destinations will be so tiny that it won’t matter.
With this unique architecture, you can expect significant improvements over other routers. The results from their own statistics show that it is indeed the case. They have a 98% fill rate (data from June 2014). Where they route is tightly correlated to each exchange’s market share, and not to the exchange’s pricing schedules (like most routers). They also report a ~15% fill rate on their venue, where 70-80% of their trading is done at the midpoint.
What this means is that if you send an order for 10,000 PG, it has a real opportunity to first execute at the midpoint on IEX (and getting executed at midpoint is a “price improvement” compared to the NBBO, it puts a bit of money in your pocket through better priced fills), and in the case where IEX does not have all the offsetting liquidity, the balance is sent out to the displayed exchanges. If there are 10,000 shares (or whatever balance that wasn’t filled at IEX) in those displayed exchanges (at the NBBO), you are likely to get 98% of it filled. This resolves the problems that a lot of traders have experienced: the market disappears on you when you are trying to work out a big order.
Eric Hunsader from Nanex recently did an exposé on this issue. It should be a required reading to anyone trading US equities in any significant size. In it he showed a chart of the ideal results while routing a big order, the results from a regular router and the results from the IEX router. The different in performance in remarkable:
The cost of using the IEX smart router is also not significant, just $0.0001 per share (about $1 for a 10,000 share order), plus the exchange fee/rebate incurred when orders are routed out to other exchanges (that you will pay or receive anyway). When the orders are matched at IEX, the cost is $0.0009. Its low when you consider that occasionally getting at least part of your order filled at midpoint will save you a significant amount ($0.005 for a 1 cent spread stock, the savings are more than five times the cost).
Currently (as of August of 2014) there are three retail brokerage houses that allow order routing to IEX. They are Interactive Brokers, Tradestation Securities and Lightspeed. Having an account for bigger orders at those brokerages can decrease your costs dramatically. It’s also important to ask your brokerage (in case it’s not one of the three) to enable IEX routing: the more clients demand it, the more they will feel forced to do it. As a result, traders and investors won’t have to rely on the inefficient and frequently biased smart routers the some brokers offer.
As it will be the case with a lot of advice in this site, try it out and see if your results improve.
In this site I’m going to share tricks and tactics in order to avoid being gamed by HFTs. How to adapt to these modern markets, improve your trading results, improve your order execution, become more disciplined, etc.
Ever since the rise of HFTs in 2007/2008, traders have struggled more than ever as a result of HFTs monopolizing the spread, getting access to special order types, colocation, parsing and trading news faster than any human, etc. But, make no mistake, it is STILL possible to become a profitable daytrader, the secret is to know how to ADAPT and make adjustments (both technical and psychological) in a HFT driven world.
In this website and through my twitter feed (https://twitter.com/beathft) I will be sharing these secrets that I learned from others and from my own experience.