As indicated in a previous post, high-freq market data is typically in the form of order updates, for example:
This can be used to reconstruct the orderbook at any given instant, but can also be used to analyze movements within the orderbook. The most primitive approaches look at the incidence of size being removed or hitting a given price-level. More sophisticated approaches try to determine order streams to see where an order is being moved to in the book.
Given that we can produce a variety of measures that can add information to our trading decisions. Here is an example of a USD/JPY on a downward move of 10 pips with strong momentum. The mid price is in the upper pane and the cumulative bid and ask movements (derived from order data) are in the lower pane:
One will note that the ask flows are more aggressive than the bid flows. The above is not yet terribly useful, as the cumulative flows look to follow the price path more or less during a period of momentum.
Momentum
More interesting is to look at what is happening in the book:
The reds represent areas of high cancellation and blues high insertion (I amplify this by scaling by the size of movement from one region to another). You’ll notice the following:
- Seller Book
There is high movement into the inside (or more aggressive) areas of the book, in-line with the fact that the price has downward momentum. There are vertically “trailing” deletions from the outside price levels as traders move their prices en-mass to the inside. - Buyer Book
Buyers / market makers are risk adverse and are moving their orders deeper into the book away from the inside. Again we see a trail of deletions in the inside-most levels and strong movements in a couple of deeper bands.
Sideways Movement
What does “sideways” or non-directional movement look like then?
It looks a lot more random, with some upward and downward directional patterns in sequence.
Cleaning It Up
The activity can be denoised to produce a signal that is much easier to read:
Summary
The approach is particularly useful for understanding movements over a short to medium timeframe. However long-running price movements often have periods of “indecision” where the flow and price flounders for a period. Ultimately this sort of approach needs to be evaluated on multiple windows to accomodate various timeframes.
(sorry this is mostly pretty pictures without much explanation. Should provide some ideas though).
Tr8dr 
Hi,
First thanks for this very interesting topic !
Can you precise me the axis-Y of order book graph ? Axis-X are the Time but Y ?
You say of denoised the book for easier read… nice but how ? 🙂 with PCA ?
Sorry for me stupid question 😉
Regards.
Ludo.
I am guessing that it is the number of ticks (since it is USDJPY) that the trades are moved against the last trade. So a negative reading on the sell side means that they are trying to sell in the mid, hit the bid or sell through the bid.
Ludo,
Actually is quite simple in this case. I amplified the order movements > some threshold, reduced the # of buckets, and used aliasing in allocating to buckets. The Y axis is the price in pips away from the inside. 0 = inside, +10 would be +10 away from the inside (i.e. a worse price).
Thematically, feels like visualizing principles from the RMT tradition: “sideways” is random baseline (at multiple time scales), while “momentum” is characterized by statistical divergence from baseline of a suitable, denoised matrix.
I was wondering if you can share where one can get such valuable feeds of FX data that contains order id and size information. I guess that without order id and volume at the bid / ask this analysis would not have been possible…
Regular FX feeds contain only bid/ask price with no order id OR volume.
I’m not aware of any 3rd party that provides this information. Basically I am directly connected to the FX venues like Currenex, Hotspot, FXAll, etc. In any case, the order flows are most useful for short timeframes, hard to target with platforms like IB.
This is all very nice indeed, but is the dynamics inside the order book contemporaneous with the price change? Or does it have any predictive power over some future period? Also what’s the time scale?
Orderbook activity will occur before a price change, however there is a lag in detecting aggregate behavior which would bring it to be almost contemporaneous. This is really only useful over very short time scales, presenting too much noise for a typical trend over many minutes. Most trends that last for minutes have aggressive stages and periods of indecision. In the order book would see noise in the period of indecision.
Tr8der,
I remember older posts of yours where you said you kept away from HF due to commissions etc. I’d be very interested to hear what changed your opinion on this.
What I am interested in is if you have found brokers that have a commissions structure that is appropriate for either personal or small shop Level II trading (interested mostly in FX).
Many thanks
Well, you need to have enough capital to get a prime brokerage account. With a PB, you can connect to venues directly. The reason why I do both medium and HF, is that there are times in the market where there is more pickup doing some form of market making and other time where medium frequency makes more sense. Doing both allows me to stay in the market most of the time.
ok, makes sense. Another question I’d be thankful if you could answer is this,
in your posts I see a lot of Java, C# etc, so it makes sense to assume that you are using a managed environment for your execution. How have you found managed environments (both .net and the JVM) for your high-frequency strategies? I don’t work in HFT so I may be talking nonsense here but if I were to pick one language for a high-frequency execution system, that would be C++.
Of course if you haven’t seen an impact from garbage collectors so far, C# is a much cleaner language and I’d use that,
Many thanks!
If one is doing ultra-high frequency (like micro-seconds), you’ll want to write to the metal. This will be a combination of low-level C, device drivers, FPGA, or other approaches. Very expensive and limited approach, but appropriate for a specific style of trading.
I use a hybrid approach, C/C++ for execution engine and F#/C# for model evaluation. I really dislike C++ for all sorts of reasons. Performance is no longer a good reason not to use a managed environment, unless you need an extra 5-10%.
What is more important is avoiding unpredictable pauses that can be caused by GC. In C++ one directly controls GC, in C#, say, one does not. However, GC can be avoided by being clever about not generating it in the first place, particularly for high volume objects.
tr8dr,
What venue feed give you “update” “delete” “trade” “new” etc etc diffuse this information ?
FiX APi ? HITCH Protocol ? API Java ?
You stopped your post ?
Regards
played with the idea a while ago on Coinbase data, I couldn’t extract a signal myself.
http://prnt.sc/esii5z
amazing blog btw, hope it takes off again
Thanks, I have been doing proprietary work that I can’t write about for a while. From my experience there is short-term predictability in order book movements, more suitable to HFT and market making. I have backed away from trading bitcoin due to security concerns, however found that the bitcoin book data is quite rich. A problem for bitcoin is that some exchanges don’t readily provide the book data unless you have a special arrangement with them. Also hard to keep up with the pace of change and the relatively large # of exchanges out there.
thanks for the comment. I’ve been able to figure out a decent directional strategy by weighting different LOB indicators. will be usable for MM style strategy too, I think. I also learned the value of queue position, have little idea how to work with this so far. Most of the information online seems to be garbage.
I can understand the security concerns when it comes to bitcoin, even Coinbase who looked good to me had an ugly hack last year. I did some hard work on the engine that I wrote from scratch, but one only needs to write this once I guess.