When HFT Steals Liquidity - Exploratory Trading In The eMini

Tyler Durden's picture

Via Nanex,

Exploratory Trading in the eMini

On November 12, 2012,  Adam D. Clark-Joseph published Exploratory Trading, which analyzes CFTC audit level trading data in the eMini S&P 500 futures market. This is a special, "regulators-only" data-set that contains all orders and trades, and each order and trade has a trader identifier. What this paper exposes is astounding. The following is our summary of Clark-Joseph's paper.

Exploratory trading

Exploratory trading is a form of manipulation designed to test the market's reaction to a trade. Probing for stop orders would be one form of exploratory trading. This paper specifically investigates exploratory trading that attempts to determine whether the bid/ask spread is about to shift up or down a level. The impact on the market would be an increase in intraday volatility. Exploratory trading distorts the market's view of supply and demand and induces trading activity from other participants. Furthermore, as participants learn of the strategy, they will employ counter-measures - which will further muddy an accurate picture of supply and demand for everyone else. This is why regulations ban manipulation.

The Top 8 HFTs Remove Liquidity 59% of the Time

Passive market making involves buying at the bid, and selling at the ask, which earns the market maker the bid/ask spread. Passive market making provides liquidity, narrows spreads, and lowers trading costs. Aggressive trading removes liquidity: buying at the ask (removes sell orders) and selling at the bid (removes buy orders).

Between September 17, 2010 and November 1, 2010 in the eMini futures contract (December 2010 contract, symbol ESZ0):

  • 41,778 accounts traded this contract
  • 30 of these accounts (less than 1/10th of 1%) met criteria to be classified as HFT.

These 30 HFT accounts:

  • participated in 46.7% of total trading volume.
  • grossed $1.51 million per trading day.

Of these 30 HFT, the top 8:

  • were aggressive 59.2% by volume (the other 22 were aggressive 35.9% by volume).
  • grossed $793,342 per trading day.

Nanex Discussion

The top HFTs probe the market by aggressively pinging order books and then analyzing market reaction: a practice that allows them to get a private glimpse of the "true" supply and demand at the expense of everyone else. Once the market direction is ascertained, these HFT aggressively remove liquidity, causing an immediate market move. Since the eMini is heavily arbitraged by SPY (which in turn is arbitraged by its many components and options), these sudden moves in the eMini will set off waves of overwhelming message traffic as traders and algos react and reprice thousands of instruments in milliseconds.

In light of our discovery that Waddell and Reed's trades in the eMini on May 6, 2010 were entirely passive (0% aggressive), we wonder if this probing by HFTs may have set in motion the downward spiral on that day, resulting in the Flash Crash. These HFTs not only manipulated markets on that day in a disastrous way, they drove liquidity providers away from the market.

A lot of media discussion about HFT focuses on 3 benefits: they provide liquidity, narrow spreads and lower trading costs. This Harvard paper exposes some disturbing truths: the top HFT engage in a predatory market manipulation strategy that removes liquidity 59.2% of the time (by volume), causes undue intraday volatility (which amounts to a tax on investors), warps the true picture of supply and demand, and raises trading costs for everyone processing market data.

Perhaps even more disturbing was the Bloomberg article where we first learned of this paper. It appears that rather than investigate HFT manipulating the markets, the regulator is investigating academic access to their audit level data-set.

Next time the media writes about the benefits of HFT - ask them if they've read Adam Clark-Joseph's paper on Exploratory Trading.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Say What Again's picture

Someone PLEASE download the data before its blocked!!!

Say What Again's picture

From the Bloomberg article


The agency oversees data on derivatives trades by firms including Goldman Sachs Group Inc. (GS) and JPMorgan Chase & Co. (JPM) that take place on exchanges including the one operated by CME Group (CME) Inc.

Now we know who's really trying to block access to the historical data.




And while you're there, check out the video from Roubini about "a bigger bubble than 2004."


kaiserhoff's picture

Machines scalping the public for fun and profit.  Wunderbar.

Now, if we could only capture the energy from our founding fathers SPINNING IN THEIR GRAVES!

prains's picture

rent-seeking is the attempt to obtain economic rent through a deployment in the political, social or environmental landscape in which the economic activities seeks to occur. This is different to the value-adding approach of creating new sources of wealth. One example is spending money on political lobbying in order to be given a share of wealth that has already been created. A famous example of rent-seeking is the limiting of access to lucrative occupations, as by medieval guilds or modern state certifications and licensures. People accused of rent seeking typically argue that they are indeed creating new wealth (or preventing the reduction of old wealth) by improving quality controls, guaranteeing that charlatans do not prey on a gullible public, and preventing bubbles.

kaiserhoff's picture


damn fat finger

bank guy in Brussels's picture

HFT = Hugely F*cked Trading

Shut that criminal shite down

q99x2's picture

Funny I don't feel astounded. And, F'k Bloomberg too.

f16hoser's picture

Hoser: HFT is shit. 

Johnny-5: "Shit?" "Where see shit?"

chump666's picture

What if HFT's become self aware and realize that the system had become too 'crowded' or to excessive and then send a correction sell order.

Imagine the crash?

disabledvet's picture

then "Mr. T-2000" would have to shut himself down. http://www.youtube.com/watch?v=DEMICfWLOig which goes to show you why those who complain about the long side bias are right to do so. having said that if the "freq" show is crushing volume..."disabling" price discovery as it were...then how would that be good for ANY market participant...let alone "the longs." I would argue STRONGLY that the problem we now have is one of "the media" (so defined) and the fear of taking any position save for "odd lot size" since there is no way to take sizeable trading positions without same said positions being known and recognized. (that would be both long AND short.) it obviously stands to reason why "zee authorities" have taken the position of "no large secretive trades" given 9/11 and now QE. i'm sure trying to "get your handle" to do any of this type of trading is a real bear as well. http://www.youtube.com/watch?v=3EWIn-GFOto

Smuckers's picture

Analysis in HFT <ANAGRAM> Flash Insanity

Dewey Cheatum Howe's picture

From the Bloomberg article cited in the article above.

The top U.S. derivatives regulator has suspended a program of visiting academic researchers over concerns about the handling of confidential trading data.

Sums it all up. Derivatives and confidential trading a recipe for financial disaster when done in behind closed doors and no one to keep the regulators honest. Open information keeps those in charge of regulating honest and doing their actual jobs.

Dewey Cheatum Howe's picture

It is so easy even a caveman can understand it!

Pandorable's picture

Wonder who paid whom to have that program suspended.  Not really.   Effers.

Pandorable's picture

How the F is that legal?  Oh, wait -- because it's over the heads of most of the pea-brained, political empty suits. And the ones who might halfway get it end up paid off.  No wonder ROB-BEN HOOD gets over.  

The freaking politicoots in DC can't even agree on how to distribute 85bil for the sequester, much less comprehend the insidious depth of economic corruption and degradation that will result from allowing these HFT programs to run unregulated. We're all hosed.

Orly's picture

The old regulatory witch-hunt trick, eh?  They don't want to investigate the wrong-doing that they must clearly be aware of...they want to know how you got the data.



Pandorable's picture

Effers.  Em effing effers.

chump666's picture

Did you follow the HFT 5day on the EJ, or EUR/JPY trades?  1.24 handle beckons.

Orly's picture

I have been trying.  That's only 80 pips from here but there are...mysterious forces...at work, seems like.  They don't want this thing down.  Like, at all.

I caught the first wave down when SPX went negative but right after that, an invisible hand came in and is pushing it up whenever it reaches 124.76.

Looks like it's back to "swing trading" the five minute chart again for the rest of the week.  This is getting kinda sad, tell the truth.

Topix and Nikkei both off 0.4% but the JPY still getting sold.  I bet the after-lunch session sees some cross selling...


chump666's picture

Unless you are running large margins and follow HFTs every second of the day and night, yeah, it's 'I'm out till the sh*t flies'.

Copper went 1.0% + on a crowded trade, when most of the commodity complex is trending down.  The EUR is a joke, a commie joke, ECB buying via NY FED support (what is it now, 4billion a month? http://online.wsj.com/article/BT-CO-20130214-715308.html borrowed on nearly 0% interest.  Please...).  Did you see Marc Faber on CNBC?  This bull euphoria is all CB's and Wall Street players now upping the margin.  He is like. 'You'll see this blow this year'

It's a three way dance with death, The Fed, Wall Street and HFTs. 

DirkDiggler11's picture

Great way to sum it all up " A three way Dance with Death" .

No HFT at the coin shop, Silver and Gold are both on sale at rock bottom prices, compliments of Bernanke and company ...

Atomizer's picture

World on A String in my back pocket when trading eMini…



Search & seize all gambling muppets

WallowaMountainMan's picture

" we wonder if this probing by HFTs may have set in motion the downward spiral on that day, resulting in the Flash Crash."

nanex is being polite.

Catflappo's picture

Holy Fucking Toledo!

Racer's picture

And all this excessive quoting and cancelling is legal?

Clowns on Acid's picture

I do think it depends on the specific Exchange, does it not ?

Stuck on Zero's picture

It's called "impulse response trading."  It has been practiced forever.  It is similar to probing a filter's characteristics by pumping in an impulse and examining the response.  You may then invert the response to obtain a Wiener filter for pre-modulation correction.  Even dogs and wolves do it.  They bark and snarl at a stranger to gauge the reaction. 


Orly's picture

So, you're saying bark and snarl back?

nanex's picture

Just one slight and important difference here. In the case of dogs and wolves (and traders in a pit), there is no anonymity. That one characteristic is what makes this practice manipulative and therefore illegal. If all parties weren't anonymous (like in a trading pit), then it would be fine. Got it?



Stuck on Zero's picture

Very true.  It's very easy to wipe out algos using impulse generators, however.  Respond in kind and then when they feel they have the market figured out invert the transfer function and wipe them out.  Sometimes companies do it themselves with the timing of dividend announcements or splits.  It's also a good way to shake out the naked shorts.  The only drawback is that you have to have some deep and steady pockets if the other party is a Martingale trader.

Clowns on Acid's picture

nanex - great work....but I do think that the Exchanges or ECNs were the HFT guys use and abuse their microsec advantage and gum up the bandwidth of the Exchange....ceratinly the Exchanges can see "who" it is.

All quotes and cancellations would have their unique identifier of the firm sedning the orders / quotes to the Matching (Trading) engine.

Indeed some  Exchanges have put on a fee for  "excessive" cancellation / order ratios and  bandwidth utilization. In order to do this the Excahnge must ID each individual order / cancellation in pretty much real time.

Am I missing something here?

BuyMeAPony's picture

There are no retail punters in the futures market. So this 1.5m was just taken off investment bank flow traders and fund managers who charge a fortune for underperforming the index -- hard to feel sorry for these guys. Considering the notional amounts involved, 1.5m isn't much of a skim. It is almost like everyone hates to see the nerds they picked on in high school making a living.

nanex's picture

This is not a case of nerds vs the prom king/queen. The prime ingredient that makes this strategy work is having the right connections to pull off massive manipulation right in front of the regulator's nose. Sound familiar? (hint, this is not a "nerd" skill or asset). Coming up with the idea to send in test trades is trading 101 - hardly rocket science.

And no "retail punters"? You didn't read the article did you? "41,778 accounts traded that contract". You think there are that many IB flow traders/fund managers? !

When rules are evenly enforced, nerds flourish. 



fonzannoon's picture

Thanks nanex. Your stuff is awesome.

ebworthen's picture

Bernanke:  "Please God - please get people to buy equities!  Mom and Pop, the Young, the Pension funds, the day traders.  Please!  Please!  I'm begging!!!"

hooligan2009's picture

don't need algos to kill this fish...just a million punters to simultaneously issue 5 lot order sizes and pull them immediately...algos cant price that..anyone can be an algo..

Atomizer's picture

The headline you read today was pure bullshit. Fake addresses, phone numbers, [excreta] < ---Keyword


What will unfold, exposing the Wall Street HFT cheat games and banking information to policy changers who want the internet controlled. Political pet projects who are funded generally go south quickly.  This ideology of creating new jobs has a smell of blowback. Paying tax on a service & creating double taxation without representation is going to be interesting. Sorry for the riddled explanation, but it’s coming.


If they really believe we will cave in, we won’t. The DHS ammunition vaults will be opened by a keystroke. Thanks for helping us stock up Janet! The history books will not treat you so well.

The Invisible Foot's picture

It's Alive!! It's Alive!!!!!

WTF_247's picture

was anyone able to download the paper - it is gone now of course.

If so can someone repost it online to reddit or other?

Crash Overide's picture

Excuse me does anyone know if IBM let Watson out of the lab lately...? 

Anyone done a word-O-gram on John Conner and Watson ummm I mean Skynet.