This page has been archived and commenting is disabled.

When HFT Steals Liquidity - Exploratory Trading In The eMini

Tyler Durden's picture




 

Via Nanex,

Exploratory Trading in the eMini

On November 12, 2012,  Adam D. Clark-Joseph published Exploratory Trading, which analyzes CFTC audit level trading data in the eMini S&P 500 futures market. This is a special, "regulators-only" data-set that contains all orders and trades, and each order and trade has a trader identifier. What this paper exposes is astounding. The following is our summary of Clark-Joseph's paper.

Exploratory trading

Exploratory trading is a form of manipulation designed to test the market's reaction to a trade. Probing for stop orders would be one form of exploratory trading. This paper specifically investigates exploratory trading that attempts to determine whether the bid/ask spread is about to shift up or down a level. The impact on the market would be an increase in intraday volatility. Exploratory trading distorts the market's view of supply and demand and induces trading activity from other participants. Furthermore, as participants learn of the strategy, they will employ counter-measures - which will further muddy an accurate picture of supply and demand for everyone else. This is why regulations ban manipulation.

The Top 8 HFTs Remove Liquidity 59% of the Time

Passive market making involves buying at the bid, and selling at the ask, which earns the market maker the bid/ask spread. Passive market making provides liquidity, narrows spreads, and lowers trading costs. Aggressive trading removes liquidity: buying at the ask (removes sell orders) and selling at the bid (removes buy orders).

Between September 17, 2010 and November 1, 2010 in the eMini futures contract (December 2010 contract, symbol ESZ0):

  • 41,778 accounts traded this contract
  • 30 of these accounts (less than 1/10th of 1%) met criteria to be classified as HFT.

These 30 HFT accounts:

  • participated in 46.7% of total trading volume.
  • grossed $1.51 million per trading day.

Of these 30 HFT, the top 8:

  • were aggressive 59.2% by volume (the other 22 were aggressive 35.9% by volume).
  • grossed $793,342 per trading day.

Nanex Discussion

The top HFTs probe the market by aggressively pinging order books and then analyzing market reaction: a practice that allows them to get a private glimpse of the "true" supply and demand at the expense of everyone else. Once the market direction is ascertained, these HFT aggressively remove liquidity, causing an immediate market move. Since the eMini is heavily arbitraged by SPY (which in turn is arbitraged by its many components and options), these sudden moves in the eMini will set off waves of overwhelming message traffic as traders and algos react and reprice thousands of instruments in milliseconds.

In light of our discovery that Waddell and Reed's trades in the eMini on May 6, 2010 were entirely passive (0% aggressive), we wonder if this probing by HFTs may have set in motion the downward spiral on that day, resulting in the Flash Crash. These HFTs not only manipulated markets on that day in a disastrous way, they drove liquidity providers away from the market.

A lot of media discussion about HFT focuses on 3 benefits: they provide liquidity, narrow spreads and lower trading costs. This Harvard paper exposes some disturbing truths: the top HFT engage in a predatory market manipulation strategy that removes liquidity 59.2% of the time (by volume), causes undue intraday volatility (which amounts to a tax on investors), warps the true picture of supply and demand, and raises trading costs for everyone processing market data.

Perhaps even more disturbing was the Bloomberg article where we first learned of this paper. It appears that rather than investigate HFT manipulating the markets, the regulator is investigating academic access to their audit level data-set.

Next time the media writes about the benefits of HFT - ask them if they've read Adam Clark-Joseph's paper on Exploratory Trading.

 

- advertisements -

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Tue, 03/12/2013 - 19:45 | 3324293 Say What Again
Say What Again's picture

Someone PLEASE download the data before its blocked!!!

Tue, 03/12/2013 - 20:10 | 3324357 Say What Again
Say What Again's picture

From the Bloomberg article

http://www.bloomberg.com/news/2013-03-06/academic-use-of-cftc-s-private-...

The agency oversees data on derivatives trades by firms including Goldman Sachs Group Inc. (GS) and JPMorgan Chase & Co. (JPM) that take place on exchanges including the one operated by CME Group (CME) Inc.

Now we know who's really trying to block access to the historical data.

.

.

.

And while you're there, check out the video from Roubini about "a bigger bubble than 2004."

http://www.bloomberg.com/video/roubini-says-austerity-backlash-big-econo...

Tue, 03/12/2013 - 21:02 | 3324362 kaiserhoff
kaiserhoff's picture

Machines scalping the public for fun and profit.  Wunderbar.

Now, if we could only capture the energy from our founding fathers SPINNING IN THEIR GRAVES!

Tue, 03/12/2013 - 23:47 | 3324934 prains
prains's picture

rent-seeking is the attempt to obtain economic rent through a deployment in the political, social or environmental landscape in which the economic activities seeks to occur. This is different to the value-adding approach of creating new sources of wealth. One example is spending money on political lobbying in order to be given a share of wealth that has already been created. A famous example of rent-seeking is the limiting of access to lucrative occupations, as by medieval guilds or modern state certifications and licensures. People accused of rent seeking typically argue that they are indeed creating new wealth (or preventing the reduction of old wealth) by improving quality controls, guaranteeing that charlatans do not prey on a gullible public, and preventing bubbles.

Tue, 03/12/2013 - 21:00 | 3324366 kaiserhoff
kaiserhoff's picture

CONDEMNO ARBILLA DIGITAS

damn fat finger

Tue, 03/12/2013 - 23:08 | 3324826 CalibratedConfidence
CalibratedConfidence's picture

Good work

Tue, 03/12/2013 - 19:46 | 3324296 bank guy in Brussels
bank guy in Brussels's picture

HFT = Hugely F*cked Trading

Shut that criminal shite down

Wed, 03/13/2013 - 00:22 | 3324995 prains
prains's picture

HFT = Hi Fuck Tard

Tue, 03/12/2013 - 19:47 | 3324303 q99x2
q99x2's picture

Funny I don't feel astounded. And, F'k Bloomberg too.

Tue, 03/12/2013 - 19:48 | 3324308 f16hoser
f16hoser's picture

Hoser: HFT is shit. 

Johnny-5: "Shit?" "Where see shit?"

Tue, 03/12/2013 - 19:52 | 3324314 chump666
chump666's picture

What if HFT's become self aware and realize that the system had become too 'crowded' or to excessive and then send a correction sell order.

Imagine the crash?

Tue, 03/12/2013 - 22:27 | 3324723 disabledvet
disabledvet's picture

then "Mr. T-2000" would have to shut himself down. http://www.youtube.com/watch?v=DEMICfWLOig which goes to show you why those who complain about the long side bias are right to do so. having said that if the "freq" show is crushing volume..."disabling" price discovery as it were...then how would that be good for ANY market participant...let alone "the longs." I would argue STRONGLY that the problem we now have is one of "the media" (so defined) and the fear of taking any position save for "odd lot size" since there is no way to take sizeable trading positions without same said positions being known and recognized. (that would be both long AND short.) it obviously stands to reason why "zee authorities" have taken the position of "no large secretive trades" given 9/11 and now QE. i'm sure trying to "get your handle" to do any of this type of trading is a real bear as well. http://www.youtube.com/watch?v=3EWIn-GFOto

Tue, 03/12/2013 - 19:51 | 3324315 Smuckers
Smuckers's picture

Analysis in HFT <ANAGRAM> Flash Insanity

Tue, 03/12/2013 - 19:56 | 3324316 Dewey Cheatum Howe
Dewey Cheatum Howe's picture

From the Bloomberg article cited in the article above.

The top U.S. derivatives regulator has suspended a program of visiting academic researchers over concerns about the handling of confidential trading data.

Sums it all up. Derivatives and confidential trading a recipe for financial disaster when done in behind closed doors and no one to keep the regulators honest. Open information keeps those in charge of regulating honest and doing their actual jobs.

Tue, 03/12/2013 - 19:58 | 3324329 Orly
Orly's picture

You got it!

Tue, 03/12/2013 - 20:16 | 3324378 Dewey Cheatum Howe
Dewey Cheatum Howe's picture

It is so easy even a caveman can understand it!

Tue, 03/12/2013 - 20:00 | 3324341 Pandorable
Pandorable's picture

Wonder who paid whom to have that program suspended.  Not really.   Effers.

Tue, 03/12/2013 - 19:57 | 3324322 Pandorable
Pandorable's picture

How the F is that legal?  Oh, wait -- because it's over the heads of most of the pea-brained, political empty suits. And the ones who might halfway get it end up paid off.  No wonder ROB-BEN HOOD gets over.  

The freaking politicoots in DC can't even agree on how to distribute 85bil for the sequester, much less comprehend the insidious depth of economic corruption and degradation that will result from allowing these HFT programs to run unregulated. We're all hosed.

Tue, 03/12/2013 - 19:56 | 3324323 Orly
Orly's picture

The old regulatory witch-hunt trick, eh?  They don't want to investigate the wrong-doing that they must clearly be aware of...they want to know how you got the data.

Pathetic.

:/

Tue, 03/12/2013 - 19:58 | 3324333 Pandorable
Pandorable's picture

Effers.  Em effing effers.

Tue, 03/12/2013 - 19:59 | 3324338 chump666
chump666's picture

Did you follow the HFT 5day on the EJ, or EUR/JPY trades?  1.24 handle beckons.

Tue, 03/12/2013 - 20:41 | 3324352 Orly
Orly's picture

I have been trying.  That's only 80 pips from here but there are...mysterious forces...at work, seems like.  They don't want this thing down.  Like, at all.

I caught the first wave down when SPX went negative but right after that, an invisible hand came in and is pushing it up whenever it reaches 124.76.

Looks like it's back to "swing trading" the five minute chart again for the rest of the week.  This is getting kinda sad, tell the truth.

Topix and Nikkei both off 0.4% but the JPY still getting sold.  I bet the after-lunch session sees some cross selling...

:(

Tue, 03/12/2013 - 21:01 | 3324468 chump666
chump666's picture

Unless you are running large margins and follow HFTs every second of the day and night, yeah, it's 'I'm out till the sh*t flies'.

Copper went 1.0% + on a crowded trade, when most of the commodity complex is trending down.  The EUR is a joke, a commie joke, ECB buying via NY FED support (what is it now, 4billion a month? http://online.wsj.com/article/BT-CO-20130214-715308.html borrowed on nearly 0% interest.  Please...).  Did you see Marc Faber on CNBC?  This bull euphoria is all CB's and Wall Street players now upping the margin.  He is like. 'You'll see this blow this year'

It's a three way dance with death, The Fed, Wall Street and HFTs. 

Tue, 03/12/2013 - 22:35 | 3324743 DirkDiggler11
DirkDiggler11's picture

Great way to sum it all up " A three way Dance with Death" .

No HFT at the coin shop, Silver and Gold are both on sale at rock bottom prices, compliments of Bernanke and company ...

Tue, 03/12/2013 - 20:03 | 3324331 Atomizer
Atomizer's picture

World on A String in my back pocket when trading eMini…

 

/lol 

Search & seize all gambling muppets

Tue, 03/12/2013 - 20:00 | 3324336 WallowaMountainMan
WallowaMountainMan's picture

" we wonder if this probing by HFTs may have set in motion the downward spiral on that day, resulting in the Flash Crash."

nanex is being polite.

Tue, 03/12/2013 - 20:18 | 3324382 Atomizer
Tue, 03/12/2013 - 20:01 | 3324344 Catflappo
Catflappo's picture

Holy Fucking Toledo!

Tue, 03/12/2013 - 20:27 | 3324396 Racer
Racer's picture

And all this excessive quoting and cancelling is legal?

Tue, 03/12/2013 - 20:34 | 3324418 nanex
nanex's picture

Hell no!

 

nanex

Tue, 03/12/2013 - 23:17 | 3324858 Clowns on Acid
Clowns on Acid's picture

I do think it depends on the specific Exchange, does it not ?

Tue, 03/12/2013 - 20:36 | 3324424 Stuck on Zero
Stuck on Zero's picture

It's called "impulse response trading."  It has been practiced forever.  It is similar to probing a filter's characteristics by pumping in an impulse and examining the response.  You may then invert the response to obtain a Wiener filter for pre-modulation correction.  Even dogs and wolves do it.  They bark and snarl at a stranger to gauge the reaction. 

 

Tue, 03/12/2013 - 20:43 | 3324440 Orly
Orly's picture

So, you're saying bark and snarl back?

Tue, 03/12/2013 - 20:43 | 3324441 nanex
nanex's picture

Just one slight and important difference here. In the case of dogs and wolves (and traders in a pit), there is no anonymity. That one characteristic is what makes this practice manipulative and therefore illegal. If all parties weren't anonymous (like in a trading pit), then it would be fine. Got it?

 

nanex

Tue, 03/12/2013 - 23:05 | 3324819 Stuck on Zero
Stuck on Zero's picture

Very true.  It's very easy to wipe out algos using impulse generators, however.  Respond in kind and then when they feel they have the market figured out invert the transfer function and wipe them out.  Sometimes companies do it themselves with the timing of dividend announcements or splits.  It's also a good way to shake out the naked shorts.  The only drawback is that you have to have some deep and steady pockets if the other party is a Martingale trader.

Tue, 03/12/2013 - 23:11 | 3324835 Clowns on Acid
Clowns on Acid's picture

nanex - great work....but I do think that the Exchanges or ECNs were the HFT guys use and abuse their microsec advantage and gum up the bandwidth of the Exchange....ceratinly the Exchanges can see "who" it is.

All quotes and cancellations would have their unique identifier of the firm sedning the orders / quotes to the Matching (Trading) engine.

Indeed some  Exchanges have put on a fee for  "excessive" cancellation / order ratios and  bandwidth utilization. In order to do this the Excahnge must ID each individual order / cancellation in pretty much real time.

Am I missing something here?

Tue, 03/12/2013 - 20:43 | 3324437 BuyMeAPony
BuyMeAPony's picture

There are no retail punters in the futures market. So this 1.5m was just taken off investment bank flow traders and fund managers who charge a fortune for underperforming the index -- hard to feel sorry for these guys. Considering the notional amounts involved, 1.5m isn't much of a skim. It is almost like everyone hates to see the nerds they picked on in high school making a living.

Tue, 03/12/2013 - 20:52 | 3324457 nanex
nanex's picture

This is not a case of nerds vs the prom king/queen. The prime ingredient that makes this strategy work is having the right connections to pull off massive manipulation right in front of the regulator's nose. Sound familiar? (hint, this is not a "nerd" skill or asset). Coming up with the idea to send in test trades is trading 101 - hardly rocket science.

And no "retail punters"? You didn't read the article did you? "41,778 accounts traded that contract". You think there are that many IB flow traders/fund managers? !

When rules are evenly enforced, nerds flourish. 

 

nanex

Tue, 03/12/2013 - 21:01 | 3324479 fonzannoon
fonzannoon's picture

Thanks nanex. Your stuff is awesome.

Tue, 03/12/2013 - 21:33 | 3324588 CalibratedConfidence
CalibratedConfidence's picture

I support this statement

Tue, 03/12/2013 - 20:49 | 3324452 ebworthen
ebworthen's picture

Bernanke:  "Please God - please get people to buy equities!  Mom and Pop, the Young, the Pension funds, the day traders.  Please!  Please!  I'm begging!!!"

Tue, 03/12/2013 - 20:54 | 3324463 hooligan2009
hooligan2009's picture

don't need algos to kill this fish...just a million punters to simultaneously issue 5 lot order sizes and pull them immediately...algos cant price that..anyone can be an algo..

Tue, 03/12/2013 - 20:54 | 3324465 Atomizer
Atomizer's picture

The headline you read today was pure bullshit. Fake addresses, phone numbers, [excreta] < ---Keyword

 

What will unfold, exposing the Wall Street HFT cheat games and banking information to policy changers who want the internet controlled. Political pet projects who are funded generally go south quickly.  This ideology of creating new jobs has a smell of blowback. Paying tax on a service & creating double taxation without representation is going to be interesting. Sorry for the riddled explanation, but it’s coming.

 

If they really believe we will cave in, we won’t. The DHS ammunition vaults will be opened by a keystroke. Thanks for helping us stock up Janet! The history books will not treat you so well.

Tue, 03/12/2013 - 20:56 | 3324470 The Invisible Foot
The Invisible Foot's picture

It's Alive!! It's Alive!!!!!

Tue, 03/12/2013 - 21:13 | 3324518 WTF_247
WTF_247's picture

was anyone able to download the paper - it is gone now of course.

If so can someone repost it online to reddit or other?

Tue, 03/12/2013 - 23:14 | 3324848 Atomizer
Atomizer's picture

The document is in safe guard. Mirror upon mirror upon mirror.

Wed, 03/13/2013 - 11:38 | 3326050 Ident 7777 economy
Ident 7777 economy's picture

Thanks much

Tue, 03/12/2013 - 21:18 | 3324530 Crash Overide
Crash Overide's picture

Excuse me does anyone know if IBM let Watson out of the lab lately...? 

Anyone done a word-O-gram on John Conner and Watson ummm I mean Skynet.   

 

Tue, 03/12/2013 - 21:47 | 3324626 Downtoolong
Downtoolong's picture

Exploratory Trading, because, “Hell, we can’t call it market manipulation, they might sue us for slander.”

One day I’ve got to get me a copy of the Wall Street thesaurus. It must have at least 500 pages of synonyms and phrases for lying, cheating, and stealing.

Wed, 03/13/2013 - 01:07 | 3325069 SheepleRUs
SheepleRUs's picture

My question is what can we do to help???  Sending this to our elected officials will be a waste, they don't care to read any of this.  One would think this is the smoking gun we need.  How do we use it?

Wed, 03/13/2013 - 08:27 | 3325400 AvoidingTaxation
AvoidingTaxation's picture
Accès interdit!

Vous n'avez pas le droit d'accéder à l'objet demandé. Soit celui-ci est protégé, soit il ne peut être lu par le serveur.

Si vous pensez qu'il s'agit d'une erreur du serveur, veuillez contacter le gestionnaire du site.

Error 403 www.people.fas.harvard.edu
Wed Mar 13 08:26:58 2013
Apache/2.0
Wed, 03/13/2013 - 08:59 | 3325434 Mediocritas
Mediocritas's picture

This article should have been a sticky post more than the Kyle Bass one. Get a copy of that paper while you still can and spread it around.

Unprecedented market manipulation here. Day in day out, take losses running the eMini around with aggressive liquidity removal (hitting the bids and asks), reap much larger profits in every tracking instrument using the latency advantage you have from effectively knowing the future.

Absolutely nobody is surprised by this at Nanex (or around here), but it's nice you finally see the proof. It's also nice to see clear proof of how corrupted, how captured, how criminal the regulators are. There need to be prison terms over this, starting with the regulators then flowing through to their customers. That cunt Gensler is first in line.

As usual, the crickets are chirping.

FYI, here's another, slightly older paper that's related: http://www.bankofcanada.ca/wp-content/uploads/2012/11/Brogaard-Jonathan.pdf

Wed, 03/13/2013 - 10:32 | 3325791 moneybots
moneybots's picture

" the top HFT engage in a predatory market manipulation strategy"

 

Isn't that commonly known as fraud?

Wed, 03/13/2013 - 10:35 | 3325805 moneybots
moneybots's picture

" the top HFT engage in a... market manipulation strategy"

 

Isn't that what the FED is doing with QE and ZIRP?

 

Isn't that also a fraud?

Wed, 03/13/2013 - 10:40 | 3325826 moneybots
moneybots's picture

Why is fraud acceptable to people- when it makes markets go up?  People don't like the truth when ti comes out and markets crash.  But they are all ready to support the next fraud that pushes a market up.

Wed, 03/13/2013 - 12:18 | 3326135 dvsteenk
dvsteenk's picture

key in this story is that the issuers of the derivatives have an inventory of all long and short positions also, at any time, as they control the data

GS for instance provides (and receives) data to/from European banks relative to turbos (leveraged products, with V-shaped return function) traded on indices, currencies, commodities = they know every change in position in the market and can trade against this, possibly with the help of HFT.

data is control... full control

Do NOT follow this link or you will be banned from the site!