Ponzi-Like Bidding Scheme

Just for fun:  There is a retail bidding site QuiBids which has received a bit of press in the US recently.   The attraction is that they offer products (such as ipads) for, say, an average price of $30-50 (whatever the highest bid is).    One may wonder how this works.

Most auctions are “penny auctions”, meaning that each bid placed on an item increments the price by $0.01.   Prospective buyers pay for each bid they put forward on an item.   In fact, the company charges $0.60 / bid.   So with an end-auction price of, say, $50, the ipad has brought in $3000, because there were 5000 bids to achieve that price.  Wow, these guys are making a lot of money in terms of % markup!

The bidding session gets prolonged by ~10 seconds each time a bid is made in the last seconds.   Basically, it seems to me that the winner is the person in the pool of bidders that has not exhausted his prepaid bids or otherwise given up.

Since one can see the bidder with the top bid in real-time as the end-game bidding occurs, perhaps one could game it, observing the timeseries of bids each participant has placed and come up with a view on likelihood of having reached their bidding limit.   At that point one would start testing by putting in a bid and determine who is left.

If the gaming commission understood what they are doing here, suspect they would have to get a gambling license.    Interesting “not-quite-a-scam” business concept.   I have to think that eventually most participants, not having won anything will discontinue with the site.

Bipartite Matching

For very low-level order analysis we try to determine continuity from one order to the next as a stream of orders across a period.   This is applicable for orders generated by a given trader (algo) that are more-or-less continually in the market.

In most cases an order near the top of book has a short lifetime, but an algo will maintain continuity through resubmission into the order book at a different price level.   This stream of orders is often masked in various ways, but often has discernable patterns, depending on the sophistication or consideration towards masking this in the algo.

Going from order deletions to new order insertions we want to attempt to match up the old and new orders (where applicable), to determine the continuity of an order stream.   We therefore want to map from an old-set to a new-set:

However, in order to determine, we have to explore all possible mappings with weights between these edges:

The optimal mapping will be one that finds the set of one-to-one edges that has maximal weight.   This is a problem in graph theory and optimisation theory.   This can be expressed rather neatly as a linear constraints problem where we have:

Where Xij is a matrix of 0 or 1, 1 indicating an edge and 0 no edge.  Wij is a corresponding matrix of weights for each edge.   Integer (or integral) linear programming solutions are classified as NP-hard and do not have an efficient algorithm.    We will be evaluating these on a relatively small # of orders each time, so the cost of a brute force solution is not bad.

The real trick is in determining the weights, representing the degree of similarity.  Going further, one may generate a tree of these (that spans out exponentially, but is trimmed regularly).   Some decisions on prior matches will be wrong and should be reevaluated holistically.

Is it worth the effort?   Depends on what information you are interested in and what your game is.   We do a mixture range ultra-high frequency to medium frequency.   This is one sub-problem of a bigger picture.

Addendum

I should mention that the worst case brute-force approach involves examining combined weights for each permutation of edges.  This is a O(n!) algorithm that quickly becomes uncomputable.   For the specific bipartite matching problem, rather than the general Integral Linear Programming problem there are much more efficient solutions.   One of the simplest (though not the most efficient) is the “Hungarian algorithm“.   A reader pointed out a paper for an “online” approach to matching as well.

What you don’t know …

Sometimes there are precipitous drops (or ascents) in price over a very short period in the market.   Over a short period,  momentum analysis from prices or order flow, is not going to allow one to avoid becoming a casualty — Unless, you have  up-front information or lower-latency access to the event source than most participants.

However, when it comes to information impacting intra-day FX, it is disproportionately known to a small # of players:

1. large market makers (such as banks in the FX market) have access to significant flow and therefore can have a read on supply and demand for a given asset.
2. banks also know who their trading-counterparts (customers) are and whether the customer is “smart”,  tending to trade ahead of movements.  They also have a good view on how much size is traded over what period.  This is valuable information that they can piggyback on.

The small HF hedge fund participates on the anonymous venues without access to either of these.    Now a fund may have better models, but still lacks a direct view of the market flows as indicated above.    Whether it be large-flow houses or smart hedge funds, some smaller % of orders represent “smart” or in-the-know money and the larger % will tend to be more reactive.

A Drop

Here is an example of a ~10 pip drop that occurred over a period of 200ms:

The bottom pane contains order-streams for a few bid-side orders of interest in this discussion.   dp is expressed in distance from the inside price (for the bid side this means a larger DP means a lower order price).   By order-streams, referring to implicit cancel/replace activity by a trader (algo).

Notice that the orders have been moved deeper into the book (as seen by a spike in dp) as a defensive move against a falling price.  Looking more closely, this set of orders actually made the move before the drop (aha, “smart” algo working here).

Notice that four of the order streams (green – blue) moved orders deeper into the book ~60ms before the drop and the other colors a ms or 2 before drop but in-time to avoid a fill.

Conclusion

If you don’t have early access to information, look for a participant who does and use appropriately.   This is easier said than done, however. Detecting participants (i.e. order-streams) and determining whether they are “smart” or “dumb” is not straightforward.   I will touch on some of the related problems in a later post, but unfortunately cannot discuss everything in detail.

HF Simulation

Simulation for high-frequency strategies is, at best, a decent way to take one’s strategy through its paces, perhaps giving a view on profitability.   In no way can it present an accurate view on profitability, rather at best can look at the strategy’s performance under various synthetic assumptions.

Naturally our simulator must reflect:

1. typical order event latency (perhaps with some jitter)
2. typical market event latency
3. accurate orderbook
4. view on trade fills
5. view on impact (icing on the cake, perhaps not necessary)

The Data

There is just not enough information presented in market data to model with complete accuracy nor can we predict with certainty, the effects of our own trades on subsequent market events.   With access to a HF feed, we receive events such as:

We are able to see most individual order activity.  Primarily, the operations are New, Update, and Delete order.    The exceptions are:

1. hidden orders
2. orders that immediately cross
3. IOC orders (immediate or cancel)

What is shown or not shown may vary from venue to venue.   With the above we can reconstruct the orderbook as a set of price levels on the bid and another set on the ask.  Each price-level represents a queue of orders against which executions will occur on a FIFO basis.

The orderbook (abbreviated) might look something like this:

A simulator using historical data needs to reconstruct the orderbook, maintaining proper FIFO behavior.   The two biggest problems in simulation are:

1. determining when / if a trade happened
2. determining market impact of strategy trades on historical data

With the various FX venues trades are shown selectively, and at that, with significant lag to the actual event.   Order (New or Update) events that would have crossed are also not shown.   This leaves us to guess when a trade is likely to have happened.   We can execute our strategy under various scenarios of trade fill.

Possible Trade Events

Let’s look at some possible signals that a trade may have occurred:

1. all orders on our level are removed
   1. due to trade or cancellation?
2. inside order level(s) removed
   1. due to trade(s) or cancellation?
3. one of the above + our level has moved within epsilon of crossing
4. our level is now crossing

None of these except (4) can indicate a trade with certainty, however without some assumptions about the first three scenarios would only leave us with aggressive crossing or waiting for price levels to collide.

All orders on our level removed

Consider the above orderbook and the transformed below (where the removed orders are in red):

All orders on our orderlevel of 82.709 on the ask have been removed in the historical events (except our synthetic order from the strategy we are testing).   The likelihood that a trade happened is higher if the # of orders on the level and/or the maximum size historically for the life of the level is high.   With one order on the level removed it may well have been a cancellation.

In this particular case we see the order on our level and the first order on the next level removed.   This strengthens a view that a trade may have occurred.

We can assign a probability of trade based on evidence like this (though if using a random / probability approach, ones results will differ with every run).

Deeper Inside Level removed

Consider the orderbook below:

Our order is on the inside level and the level(s) above it is either cancelled or taken out.  This is an even stronger indication of trade if more than one level taken out.   Levels can disappear quickly from a number of possible stimuli:

1. traded: large buying algorithm buys in size (this happens quite often during the day)
2. moved: level primarily lead by market makers and activity elsewhere demands changing the offering
3. risk aversion: pending event or volatility cause pullback

#2 is harder to gauge perhaps, but #1 and #3 are relatively easy to see in retrospect.

Order Moves Close to Crossing

We may have a passive order that drifts towards crossing (or is placed very close to crossing):

We see that our order is 2/10ths pips away from the cross.  On some venues there may be inside-of-inside hidden orders (for venues that don’t require partial exposure of size).  In those cases, a move that tight is likely to get executed right away.   Market makers with inventory may find a cross of 2/10ths appealing as well.

Other Trade Signals?

We did not look at what was happening on the other side of the book, but could possibly correlate activity on the other side and activity in the above scenarios to determine a stronger likelihood of a trade.   I’d appreciate ideas.