This page has been archived and commenting is disabled.

Market Rigging Explained

Tyler Durden's picture




 

Submitted by Nanex

Market Rigging Explained

We received trade execution reports from an active trader who wanted to know why his larger orders almost never completely filled, even when the amount of stock advertised exceeded the number of shares wanted. For example, if 25,000 shares were at the best offer, and he sent in a limit order at the best offer price for 20,000 shares, the trade would, more likely than not, come back partially filled. In some cases, more than half of the amount of stock advertised (quoted) would disappear immediately before his order arrived at the exchange. This was the case, even in deeply liquid stocks such as Ford Motor Co (symbol F, market cap: $70 Billion). The trader sent us his trade execution reports, and we matched up his trades with our detailed consolidated quote and trade data to discover that the mechanism described in Michael Lewis's "Flash Boys" was alive and well on Wall Street.

Let's take a look at what we found from analyzing 5 large trades executed at different times over a 4 minute period in Ford Motor Co. Before each of these trades, the activity in the stock was whisper quiet. Here's a chart showing millisecond by millisecond trade and quote counts in Ford leading up to one of these 5 trades:

You can clearly tell when the trade hits: activity explodes to over 80 quotes in 1 millisecond (this is equivalent to 80K messages/second as far as network/system latency goes). But the point here is that nothing was going on in this stock in the immediate period before this trade hits the market.

In this particular example, there were a total of 24,800 shares advertised for sale at $17.38 (all trades and offered liquidity will be at this same price) from 8 exchanges. The trader wanted 20,000 of these shares. What he got was only 12,133 shares and 600 of these were on a dark pool (which wasn't part of the 24,800 shares of liquidity on the lit exchanges)! Worse, someone ELSE was filled for 1,570 shares during these same milliseconds! Remember, nothing was happening in Ford until his order came into the market. Based on the other 4 examples, we are sure that no trades would have occurred during these few milliseconds of time if it wasn't for this trader's order.

What happened to the 24,800 shares offered and why couldn't he get at least 20,000 of them? How is it that others were able to get shares during this time? This is especially disturbing when you consider these other traders (HFT) only bought shares in reaction to the original trader's order.

Detailed Analysis of a Trade

To answer these questions, let's take a look at the individual order executions and cancellations at the $17.38 limit price. In the table below, there are 7 columns. The number in the 1st column we'll use to reference a record. The 2nd column is the timestamp of the event - this is from CQS/CTA, the consolidated quote and trade SIP (Securities Information Processor). The 3rd column shows which reporting exchange sent the information in this record. The 4th column shows the total number of shares offered at $17.38 among the exchanges that trade this stock (Ford) and the 5th column (Add/Cancel) shows the number of shares added to, or removed from the total offered. If the 5th column is blank, it's because this record represents a trade execution.

The 6th and 7th columns show information about one order execution resulting in a trade at $17.38 (these columns are blank if the record is a quote update). The 6th column shows the number of shares executed, and the 7th column contains either a number which corresponds to the trade report # received by our trader, or an "X" which indicates someone else got this trade execution.

Note that other than the first entry (which is a reference to how long the 24,800 shares was sitting in the books waiting for execution), there is only 3 milliseconds of time shown here! The entire trade is reported in about 15 milliseconds (but 82% completes in 5 milliseconds).

Looking at the table data, we note the first trade is 100 shares on BOST (NQ-OMX Boston) and belongs to our trader. Note, however, it was the 4th trade reported back to him ("4" in the last column).

The next trade is 67 shares at EDGE (New Bats Edge-A), and that someone else bought those shares ("X"). How does that happen?

Lines 4 and 5 are order cancellations of 100 shares and 600 shares from ARCA and NYSE respectively - all in the same millisecond!

Lines 6, 7 and 8 are trade executions from EDGE and belong to our trader - note the first trade (line 3) was also from this exchange but went to someone else. This also means that the order cancellations from NYSE and ARCA happened before any size appeared.

The next 19 entries (lines 9 to 27) show a flurry of order cancellations coming in from NYSE, ARCA, BATS, NQEX and EDGX. This is before the first trade execution at any of those exchanges! This flurry of cancellations removes 10,300 shares from the number of shares offered (Shares Avail. column drop from 21,400 down to 11,100)!

Within 2 milliseconds, half of the shares have disappeared, someone else stole 67 shares, and our trader has only 13.5% (2,700 shares) of his order filled!

Let's chart the changes to the available liquidity, the shares executed, and orders canceled to get another look at this problem.

First off, let's start with the ideal case, which is shown in the chart on the left. The number of shares available at $17.38 is shown by the green line, which starts at 24,800 and decreases from order cancellations and trade executions (it would increase if additional sell orders were added). Note the green line is a plot of the Shares Avail. column in the table above. The cumulative number of shares executed is shown as a blue line.

In the ideal case (left chart), a trade arriving first, would execute against all the liquidity until the order was complete or liquidity (shares available) became exhausted. The blue line rises in synch with the green line, until 20,000 shares are traded.

In the real world, things are not that simple. There is another variable to account for: namely order cancellations. The cumulative number shares for sale at $17.38 that have been canceled is shown by the red line.

There is also a fourth variable - trade executions that belong to other traders, but we'll leave that out for now for simplicity sake.

The chart on the right clearly shows that order cancellations happen far faster than trade executions (red line goes up faster than blue line). This is why our trader wasn't able to get the advertised liquidity - the orders simply disappeared faster than exchanges processed his buy order.

In fact, more shares are canceled than executed - note the red line ends above the blue line. Also note how much of the activity takes places in about 2 milliseconds (09:47:56.571 to 09:47:56.573).

Perhaps this is just an isolated case and the order cancellations happening a fraction of a second before the trader's order were just a coincidence?    

Not an Isolated Case

If we take the number of shares available when the order first started executing, and plot the percentage of those shares that were canceled during the trade execution, we'll arrive at the Share Cancel % (which is simply the percentage of shares were canceled and not executed). In the first example we detailed above, the percentage was slightly more than 50%. The following chart shows the percentage of orders canceled for each of the 5 trade examples. If the chart below doesn't sufficiently cause alarms to go off, then you might want to restudy the data up to this point. The Share Cancel %  is shockingly high:

The next (and last chart) breaks down into detail where the available shares went when our trader's order started executing. It includes the component "HFT Shares" - which are trades that should have gone to our trader, but were instead stolen by other market participants, who only made those trades after reacting to his order. Note in some examples, the number of shares stolen is disturbingly high.

Conclusion

All this evidence points to an inescapable conclusion:

The order cancellations and trades executing just before, or during the traders order were not a coincidence. This is premeditated, programmed theft, plain and simple.

Michael Lewis probably said it best when he told 60 minutes that the stock market is rigged. To the fantastic claims made by HFT that they provide liquidity, perhaps we should ask, what kind of liquidity? To the now obviously ludicrous claim that "everyone's order uses the same tools that HFT uses", we'll just say, the data shows otherwise. To Mary Jo White and other officials who claim the market isn't rigged and that regulators need to look at the data before making any decisions, well, you made it this far - if things aren't clear, just re-read the above, or just call us and we'll explain it to you. Or dust off Midas and lets us show you how to work with data.

One more note to the SEC in particular - if you believe that the industry can fix these problems on their own, then we believe you are no longer fit to regulate, because that is not, and never was, how Wall Street works. Honestly, a free for all, no–holds–barred environment would be better than the current system of complicated rules which are partially enforced, but only against some participants. And make no mistake, what is shown above is as close to automatic pilfering as one can get. It probably results in a few firms showing spectacularly perfect trading records; it definitely results in people believing the market is unfair and corrupt.

And to CNBC and other financial media companies who say these problems have all been fixed - we think you might have been lied to. Probably by the ones doing the market rigging.

And finally, to our regular readers: we are taking a break. Everyone has a limit to how much corruption they can witness and digest in a given period of time and we've simply reached our limit.

* * *

We wish Nanex an enjoyable break.

We here at Zero Hedge, on the other hand, are not only just getting started, but every new case of corruption (which inevitably 6/12/24 months prior was nothing but another ridiculous "conspiracy theory") merely doubles our resolve to expose and chronicle this farce of a rigged market, rigged economy, and rigged political theater, as an aid to whatever comes next. One can only hope that the mistakes of this period of near-terminal lunacy will be studied and, hopefully, not repeated.

 

- advertisements -

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Tue, 07/15/2014 - 21:13 | 4961196 XenoFrog
XenoFrog's picture

I can't believe anyone would try and trade equities right now.

Tue, 07/15/2014 - 21:21 | 4961220 SafelyGraze
SafelyGraze's picture

maybe this sort of trading violates some obscure SEC regulation.

maybe not.

but one thing is for sure:

the local news is All Over the stories about children in hot cars.

they are everywhere.

and we can all be outraged at their parents.

and feel good about that outrage.

 

Tue, 07/15/2014 - 21:23 | 4961227 BobTheSlob
BobTheSlob's picture

My wife told me about that story in TWC. Her reaction to the woman who is pushing legislation to "mandate" an alarm she doesn't know how to make...."She's proving that ya can't fix stupid".

Typical progressive...blame someone else for my failure and mandate legislation that will push future blame to the auto companies.

Tue, 07/15/2014 - 22:41 | 4961447 rehypothecator
rehypothecator's picture

Markets can remain irrational longer than one can remain solvent, but this article indicates that markets can also remain rigged longer than Nanex can stay pressing the issue. 

Still, kudos to Nanex for doing the jobs that the SEC, FINRA, FBI, and a host of other deadweight government agencies refuse to do.  

Tue, 07/15/2014 - 23:46 | 4961601 Dollarmedes
Dollarmedes's picture

This is all planned.

In exchange for political donations, the Obama administration is looking the other way so that firms can steal.

HFT isn't even the tip of the iceberg of corruption.

Wed, 07/16/2014 - 01:39 | 4961776 CrazyCooter
CrazyCooter's picture

Retirement is a lie. That is what is driving all this.

Once a man bucks up and realizes he is hoe'in until his last day, it gets pretty simple.

Regards,

Cooter

Wed, 07/16/2014 - 05:24 | 4961918 MeMadMax
MeMadMax's picture

So, next time, stagger your trades: 100/500/whatever a second/min/hour/whatever...

 

I'm sure you could wait a couple of minutes to get your trade in...

 

At least you would get your trade in...

Wed, 07/16/2014 - 09:08 | 4962212 SWCroaker
SWCroaker's picture

That's like making 20 trips to the grocery store, but only taking $5 with you each time 'cause the muggers go after big money.   Effective maybe, but not what I'd call a solution to an egregious problem.

Wed, 07/16/2014 - 10:34 | 4962542 maxamus
maxamus's picture

Um, and for those of us with large accounts, what, am I supposed to only buy $10k worth of stock at a time and now use multiple multiple orders?  That is your solution?

Wed, 07/16/2014 - 12:07 | 4962850 MeMadMax
MeMadMax's picture

You got any better ideas?

It's the large orders that are tipping off the bull in the china shop...

 

Now, what would you rather do?

Take a little more time, and money to ensure that the order gets thru and the customer is happy?

Or would you rather get stiffed while some other asshole makes money off your move?

 

>_>

Wed, 07/16/2014 - 10:48 | 4962591 PrintemDano
PrintemDano's picture

Since numerous small trades generally cost more than one large trade your solution will cost the trader more money.  Throwing gasoline on the fire.

Wed, 07/16/2014 - 04:31 | 4961887 asteroids
asteroids's picture

Sorry you missed the point here. The market isn't irrational at all. The point NANEX is making is that it's premeditated. The result is a criminal environment. NANEX isn't calling it theft, but I'll call it fraud. The tape is lying to you.

Wed, 07/16/2014 - 09:29 | 4962268 BobPaulson
BobPaulson's picture

Can't tell if you are being deeply facetious by further pointing attention to the tangential farcical new story, or if you are serious and therefore revealing how you are part of the problem. 

(oops, sorry, just read your post below and now I can see you are a troll)

Tue, 07/15/2014 - 21:41 | 4961281 NoDecaf
NoDecaf's picture

So is it too late to make an investment club on Yahoo groups and start day trading with my neighbors? I saw some housewives on TV doing it back in 2005 and always wanted to get in on the action.

Wed, 07/16/2014 - 03:55 | 4961866 Jannn
Tue, 07/15/2014 - 21:13 | 4961197 BobTheSlob
BobTheSlob's picture

Amusing, but "rigged" isn't the same as manipulated. So they effectively are competing against a computer. If you don't like it, go buy Ford from your next door neighbor after looking up the last price in the news paper. This is the 21st century, and someone has found a fun way to electronically make some change off of each transaction. Not "rigged", manipulated. And rather cleverly!

Tue, 07/15/2014 - 21:25 | 4961231 nanex
nanex's picture

Fuck you Bob. And I mean that.

 

--nanex

Tue, 07/15/2014 - 21:27 | 4961244 BobTheSlob
BobTheSlob's picture

Send me pics. You have mine already.

Tue, 07/15/2014 - 22:20 | 4961398 Cliff Claven Cheers
Cliff Claven Cheers's picture

I like to refer to it as Financial Fuckery TM.

Wed, 07/16/2014 - 09:31 | 4962291 BobPaulson
BobPaulson's picture

It took me reading two of his posts to see he/she is a troll. Sorry, MillionDollarBonus is funnier. Who was the one before that about two years ago?

Tue, 07/15/2014 - 22:21 | 4961402 Excursionist
Excursionist's picture

Regardless of your views on what is clever, cute, etc., rules appear to be flouted as an ordinary course of business.

Example:  scroll to the summary discussion of Reg NMS Rule 611 on page 25:  http://www.sec.gov/rules/final/34-51808.pdf

Of note is the language pertaining to "intermarket sweep orders, quotations displayed by markets that fail to meet the response requirements for automated quotations, and flickering quotations with multiple prices displayed in a single second."

Whatever.  Flash Boys and the ensuing media attention created a two-week blip on the public's Kardashian-filled minds.  A critical mass doesn't give a shit, so the status quo will prevail.  If you're employed by Cliff Asness, Ken Griffin or someone of their ilk, send 'em my congrats.

Wed, 07/16/2014 - 12:34 | 4963005 rum_runner
rum_runner's picture

Half of America owns no stocks, doesn't understand concepts that aren't pictures of tits and ass, and is both hooked on social media and also trying to survive.

Tue, 07/15/2014 - 23:22 | 4961553 TheReplacement
TheReplacement's picture

Thank you for a very informative and not-50,000-feet-over-my-head write up of this subject.  You can ignore bob.  He's probably on the juice to say whatever they want him to say.  I hear propaganda-troll is the leading jobs gainer.

Wed, 07/16/2014 - 06:02 | 4961936 ugmug
ugmug's picture

Wasn't the Titanic the epitome of technology in its day. Enough said.......

Tue, 07/15/2014 - 21:46 | 4961301 Downtoolong
Downtoolong's picture

You’re so proud of your smart technology.

Question: What’s dumber than an algo that thinks it can purchase a $40.00 stock for $0.01?

Answer: An algo that manages to purchase a $40.00 stock for $0.01, then turns around a microsecond later and sells it for $0.02

Tue, 07/15/2014 - 23:34 | 4961581 crazyjsmith
crazyjsmith's picture

I think the conclusion is "rigged" AND "manipulated"

 

It can and is Both at the same time.  Plenty of room to be both.  

Wed, 07/16/2014 - 02:02 | 4961635 bunnyswanson
bunnyswanson's picture

We've seen this before.  "Story of Civilization, Rousseau and Revolution," p. 634.)

 

 

"In Germany, The swej were relatively safe and prosperous, though they suffered various disabilities in economic and political life.  Special taxes were levied upon them in most of the pricipalities.  The law allowed on a limited number of sjews to live in Berlin but the law was loosely enforced.  ... many were employed to manage finances or provision their armies.  Joseph Oppenheimer known as the "Jew Suss" served the Duke and earned him many enemies. 

Accused of malfeasance at the mint, he was exonerated by a board of investigators and raised to member in the Duke's Privy Council where he soon became the dominant power.  He invented new taxes, established royal monopolies, and apparently accepted bribes - which he divided with the Duke.  When the Duke proposed that all church monies should be deposited in a central state bank, the protestant clergy joined with the nobility in opposition of the Duke and his minister.  On March 3, 1737, the Duke suddenly died.  Oppenheimer was tried ad convicted, and on February 3, 1738, he was strangled and his corpse was suspended in a cage in a public square."

 

Will Durant - The Jews

Wed, 07/16/2014 - 09:50 | 4962378 venturen
venturen's picture

Bob can you explain why they had to change Global Warming to Climate Change and how they actually mean the same things, just one doesn't require any increase in tempurature and is permanently true? What an idiot you are! 

Tue, 07/15/2014 - 21:15 | 4961203 buzzsaw99
buzzsaw99's picture

i hope they didn't chase it higher. not even one penny.

Tue, 07/15/2014 - 21:18 | 4961215 BobTheSlob
BobTheSlob's picture

And that's the point exactly. Nobody is forcing anyone to settle for a higher price. Why would you if you knew you were being goaded by a computer. My dog, however, falls for it every time.

Tue, 07/15/2014 - 21:23 | 4961226 buzzsaw99
buzzsaw99's picture

a pension fund should be able to enter a market order without paying a tax to the HFTs

Tue, 07/15/2014 - 21:27 | 4961241 BobTheSlob
BobTheSlob's picture

Well I guess a "tax" is one way to look at it. The other way to look at it is that someone invented Robobidder. NOt an easy task to place a "moral/immoral" tag on that IMO.

Tue, 07/15/2014 - 23:20 | 4961546 TheReplacement
TheReplacement's picture

The day you go to the grocery store and someone takes what you intended to buy and asks you nicely to pay them more for it, that is the day you will get it. 

Wed, 07/16/2014 - 09:52 | 4962394 ILoveDebt
ILoveDebt's picture

More like -- "The day you go to the grocery store and it allows someone to take what you intended to buy and asks you nicely to pay them more for it, that is the day you will get it."

 

The real problem here is that the exchanges and regulators are complicit with the theft.  

Wed, 07/16/2014 - 12:03 | 4962838 g speed
g speed's picture

buzz--I gave you the td for resonding to BTS----

Tue, 07/15/2014 - 23:17 | 4961543 RaceToTheBottom
RaceToTheBottom's picture

"My dog, however, falls for it every time."

Forcing Sexual favors is not acceptable, even against animals.

Tue, 07/15/2014 - 21:15 | 4961206 ekm1
ekm1's picture

Is there anybody on earth who can prove that market is .....not.....rigged?

Wed, 07/16/2014 - 06:54 | 4961970 negative rates
negative rates's picture

These people can't prove anything but their own insanity.

Tue, 07/15/2014 - 21:17 | 4961210 novictim
novictim's picture

Rigged?  Rigged, you say?

I'll tell you about rigged!  Let us start with "compounded interest"! Every modern model of corporate earnings and profit has this little devil hiding at its core.  

How in the hell can anyone think that you can realistically apply pure mathematical fantasy onto natural systems?

The concept of compounded interest is a "rigged" notion if their ever was one.  Every prediction that has this nonsense baked into it is "rigged" to inevitably disappoint!  

Tue, 07/15/2014 - 21:33 | 4961259 Yen Cross
Yen Cross's picture

    The "Bait & Switch" is alive and thriving...

Wed, 07/16/2014 - 13:58 | 4963469 Chief Wonder Bread
Chief Wonder Bread's picture

delete

Wed, 07/16/2014 - 13:59 | 4963475 Chief Wonder Bread
Chief Wonder Bread's picture

+1. That simple phrase sums it up. Bait & Switch.

This is corrosive to trust in the system, what little trust is left. It creates a bogus picture of liquidity that isn't there when execution needed.  This article is the first time I've truly understood this issue. Even though it won't affect me, I'm a minnow, I say Fuck'm.

Tue, 07/15/2014 - 21:33 | 4961260 Muppet
Muppet's picture

Excellent article ZH!    Informative and factual.   Great job.

Tue, 07/15/2014 - 21:39 | 4961272 IllusionOfChoice
IllusionOfChoice's picture

Keep up the good work Tyler(s).

Tue, 07/15/2014 - 22:30 | 4961423 John Law Lives
John Law Lives's picture

+1

Good work, ZH.

Wed, 07/16/2014 - 09:59 | 4962422 Tall Tom
Tall Tom's picture

Thank you Nanex.

 

Do not think that your efforts have not been appreciated. I know that this type of investigation wears upon you and you need to step back from it to keep your sanity.

 

Take a vacation if you must. The fight for integrity and honesty will continue as long as I can utter my words.

Tue, 07/15/2014 - 21:44 | 4961295 orangegeek
orangegeek's picture

Nicely done Tylers!!!

Tue, 07/15/2014 - 21:46 | 4961300 fonzannoon
fonzannoon's picture

don't you mean nanex?

Wed, 07/16/2014 - 02:54 | 4961821 Skateboarder
Skateboarder's picture

I want to personally bitchslap everyone who didn't recognize Nanex for their great article. Thanks to the Tylers for bringing this to ZH. This is some of the most shareable information I've ever come across. It's legit as fuck and you can't question it.

Tue, 07/15/2014 - 21:50 | 4961315 DavidC
DavidC's picture

Tylers,
Don't stop, please. Scott at Nanex, I hope this is just a break.

Articles like this one make me hope that where people KNOW what is going on at some point it WILL stop. When the Berlin Wall went up people thought it was forever.

Scott, and Joe and Sal at Themis, and Tyler(s), make me know that there are people who CARE.

DavidC

Wed, 07/16/2014 - 10:04 | 4962434 Tall Tom
Tall Tom's picture

Nanex deserves a break. This wears upon you day after day while watching the fraud.

 

Even reading ZH...Sometimes I will take a break to gain perspective and realize that there is still a World with things to do and enjoy.

 

They have fought the good fight. Thank them for it.

Tue, 07/15/2014 - 22:00 | 4961324 Muppet
Muppet's picture

The cancelling of trades itself is not necessarily abnormal or criminal.  Consider:  If  traders on a trading floor are all offering a stock at $10 and a large buyer comes in at $10, it is normal for many of the traders to pull down their hands to reevaluate (thus cancelling their ask).   This is normal floor trading... that when ignition comes in... some sellers will pull back their ask to see if higher prices can be obtained.   

Therefore, IMO, this algo cancellation of liquidity does adhere to longstanding accepted floor trading by humans.

The problem is that the public does not have competive access in terms of time.   The problem is that only selected parties have access to this info and long before all others.  In that sense, it is a violation akin to 'trading off the floor'.

 

 

Tue, 07/15/2014 - 22:02 | 4961347 nanex
nanex's picture

Dude! This is like, every price shift! We are talking 100's of thousands of times a day.

 

Did they do it that often on the floor? 

 

Didn't think so.

Tue, 07/15/2014 - 22:10 | 4961367 Muppet
Muppet's picture

If a time delay or a tax on trades were instituted, this would be solved.  

Wed, 07/16/2014 - 00:38 | 4961696 mc888
mc888's picture

No... if a transaction fee was levied for every canceled quote, it would be solved.

 

Wed, 07/16/2014 - 09:37 | 4962321 BobPaulson
BobPaulson's picture

I hate to jump on armchair solutions without knowing all the details but that does seem reasonable. Why did everybody down arrow the slowing down idea? Just curious. 

Wed, 07/16/2014 - 10:14 | 4962467 Tall Tom
Tall Tom's picture

Because he opened his dialouge being an apologist for the fraud through minimizing the affect, pretending that the events were not happening systematically.. That was a dishonest tactic

 

Every other testimony offered by him after that initial post is understood to be IMPEACHABLE. He destroyed his own credibility with his intial post.

 

Yes..THIS IS A TRIAL WITH THE TRADING COMMUNITY PEERS ACTING AS THE JURY. And it is the PREPONDERANCE OF EVIDENCE which leads to CONVICTIONS, whether benevolent or malevolent.

 

It is all a test about YOUR CHARACTER.

 

We are weary of the fucking FRAUD and DECEPTION. Do you get it yet?

Wed, 07/16/2014 - 16:03 | 4964079 Muppet
Muppet's picture

Not a dishonest tatic, rather an educational one. Many may not have considered that cancelled/withdrawal of asks does have some roots.

Tue, 07/15/2014 - 22:09 | 4961359 buzzsaw99
buzzsaw99's picture

i could accept this explanation if some of the large block offered at the same price were in fact the same shares listed by multiple agents. alas, it is nothing but front running by co-located servers sniping shares ahead of a real order.

Tue, 07/15/2014 - 22:09 | 4961365 fonzannoon
fonzannoon's picture

Think how much more interesting the articles would be around here if the authors had the balls to jump into the comments section like nanex is doing.

Wed, 07/16/2014 - 01:14 | 4961745 Ralph Spoilsport
Ralph Spoilsport's picture

Nanex definitely gets Fight Club. I'd like to see Krugman jump in here and try to defend one of his articles. Maybe we'd get an assist from Niall Ferguson and Ralph Benko.

Tue, 07/15/2014 - 22:14 | 4961382 Muppet
Muppet's picture

It was offered by multiple agents as revealed by the parties doing the cancels.   Agreed it is not an equal playing field withr regards to time. 

Tue, 07/15/2014 - 22:11 | 4961371 AdvancingTime
AdvancingTime's picture

To say the market is rigged is an understatement. After over 30 years of trading commodities I will flat out state without any reservations that lies and manipulation run rampant. If you think anyone is looking out for the small independent trader you are wrong. An unholy alliance of the Federal Reserve, the government, and the too big to fail has left the rest of us in a precarious position.

For the big boys, its insider information and computer trading, this includes computing patterns that exploit where stops are placed, this improves their ability to wash the weak out of their positions. The bottom-line is that the higher the market goes the more vulnerable it becomes to a major collapse and sudden downward move. More on this subject in the article below.

http://brucewilds.blogspot.com/2013/07/markets-more-lies-and-munipulatio...

 

 

Tue, 07/15/2014 - 22:15 | 4961385 fonzannoon
fonzannoon's picture

pathetic attempt at adding value to conversation - check

pimping blog - check

immediate upvote - check

Tue, 07/15/2014 - 22:24 | 4961411 Youri Carma
Youri Carma's picture
Wall Street Grabs NATO Towers in Traders’ Speed-of-Light Quest, so that they can frontrun and fleece their clients some more
http://www.bloomberg.com/news/print/2014-07-15/wall-street-grabs-nato-towers-in-traders-speed-of-light-quest.html
Tue, 07/15/2014 - 22:45 | 4961456 Never One Roach
Never One Roach's picture

Very very good article..thanks so much!

Tue, 07/15/2014 - 22:59 | 4961498 BobTheSlob
BobTheSlob's picture

Well ZHers, here's the thing, you might not like the reality of the situation which is "there will always be a flaw in the system". In this HFT case - and in the future expect more of it - technology is the vector, the "market" is the host.

Here's another example, one that you bitcoin "enthusiasts" will enjoy! This is NO different the what the HFT boyz dish out:

"Bitcoin has a flaw. It's always been there.

Satochi Nakamoto (the nom de guerre of the mind behind Bitcoin) designed this flaw into the software because of one false assumption. What was that assumption? He believed that it was possible to build an open and free economic system built purely on simple self-interest (selfishness). Lots of people make this mistake (famously, Greenspan believed that selfish decision making would prevent the banking crisis of 2008).

It's not, obviously. Eventually, selfishness will lead one group to rig, cheat, control, etc. the system so they can get better returns. That's exactly what happened. Here's the flaw they exploited.

When a single entity ("a single miner" or "a mining pool") controls over 50% of the transaction processing it can control the entire system. This means they can "see" every transaction, spend the same coins more than once, and deny transactions they don't approve of.

Full article, highly recommended as it has the final "NSA punchline" in it! ENJOY!

http://globalguerrillas.typepad.com/globalguerrillas/2014/06/the-nsa-now...

Wed, 07/16/2014 - 00:18 | 4961669 Godisanhftbot
Godisanhftbot's picture

 the flaw in bitcoin is that its crap. only an imbecile would pay  $10 or  $600, $1000, for a fantasy.

Tue, 07/15/2014 - 23:04 | 4961506 bilbert
bilbert's picture

A.S. said it best:  "Goddamn cocksucking cocksuckers.............."

 

Thanks Nanex - good work!

 

Tue, 07/15/2014 - 23:02 | 4961511 Socrates
Socrates's picture

Could someone post, if they can find it, the ZH link for the article on Holder hiring top white-collar crime attorneys to the SEC who have represented these banks and will do so again in private practice?

Thanks in advance

Tue, 07/15/2014 - 23:06 | 4961512 economessed
economessed's picture

This is what it's come to:  a system of extracting wealth instead of being compensated for the value you create.  Every parasite needs a good host.  If these are the only good options left in our post-growth economy, then please tell me where we can find the reset button?

Wed, 07/16/2014 - 00:06 | 4961645 Dollarmedes
Dollarmedes's picture

That's the key: not one scintilla of value added. Not to the buyer, not to the seller, not to the market in general. No extra information is revealed in these trades.

If "speculation" is just a gamble of higher future prices, then this isn't speculation. There is no risk in HFT, because there is no uncertainty. Is there even a word for this? (apart from "theft," of course)

Tue, 07/15/2014 - 23:10 | 4961526 ThirdCoastSurfer
ThirdCoastSurfer's picture

This parallels my experience as well, but is important to identify that it is specific to day or STCG trading. The truth is that this form of liquidity syphoning is reserved and discouraged. If you watch a stocks Level II trading, it quickly becomes clear that the vast majority of executed trades are accomplished in single blocks of 100 shares. For someone paying $9.95 to buy and $9.95 to sell, trading a block of 100 shares requires a price movement of near $0.20 (or 1.2% at this $17.38 price point) to break even.  I've never been able to determine what the actual price of a similiar trade by a broker comes to, mostly becuase they get payment and kickbacks for their activity. 

Again, this an issue of a computer generated algorithm, preprogrammed with a plethora of physiological and statistical precepts, against a human trader as well as a myriad of other althos and maybe even involving of the few remaining "live" trading desks. 

Why then does anyone submit a order that exceeds the standard deviation? Should an algo presume this deviation is a positive development based on advance knowledge? This rabbit hole can be chased until time itself does regress, dear Alice, and my time to delelop my thesis further has run out. 

 

 

Wed, 07/16/2014 - 00:17 | 4961666 Godisanhftbot
Godisanhftbot's picture

 lol, get a clue.  those 100 shares are not some ameritrade shmuck.

Tue, 07/15/2014 - 23:25 | 4961557 disabledvet
disabledvet's picture

I think a very large conflagration (WAR for all you homegamers out there) is being rigged here...and the market is just following along with it.

And quite honestly "it's just a shot away" as Mick and the boyz famously said.

Tue, 07/15/2014 - 23:43 | 4961597 Dazman
Dazman's picture

And they wonder why volume has dropped. Because nobody trusts the "market" anymore. I'm a trader. Well, honestly I'm more like a spectator (not speculator!!!) recently because committing money to this "market" is like committing suicide. Soon it will be bots just killing bots with no humans left. But maybe that's exactly what they want... no humans in there because the humans should be slaving to the rich for a few dollars an hour. No way should they even have even a slight chance of success in the "market"

Tue, 07/15/2014 - 23:48 | 4961611 Fuku Ben
Fuku Ben's picture

<- High priority @ ZH find it asap
<- F*ck you we don't care about the interview

I've heard a great interview I saved about a year or so ago . As I recall it went into a decent level of technical detail on at least a portion of this and how and why it happens. It just confirms the rigging we all know exists but the detail was what caught my attention. You don't typically hear the details.

I've got a lot on my plate coming up. I'll add it to the to do list.

Wed, 07/16/2014 - 00:10 | 4961654 insidious
insidious's picture

Can someone talk about how exactly the HFT algorithms are profiting in this example?

Initially there are 25,000 shares offered at the asking price. The trader orders a limit order for 20,000 shares. It is only partially filled but for the shares he purchased he paid no more than the limit price he specified.

Was the initial offered price artificially high due to the fake offers which were immediately cancelled?

Assuming the trader had to place an additional follow on order to purchase the balance of the required 20,000 shares would the asking price for the subsuquent purchase be expected be higher as a result of the previous partially filled offer?

I'm interested in understanding exactly how the HFT algorithm is making money.

For instance, trade number 3 was for 67 shares which went to someone other than the trader placing the large block trade. Seems weird (or perhaps wrong) that it occured but what is the significance, if any, from a potential profit perspective?

Wed, 07/16/2014 - 01:21 | 4961757 NAP
NAP's picture

I would like to hear answers to 'insidious' questions in post 4961654, too.

Who is selling as stock prices are run up constantly?

Wed, 07/16/2014 - 08:33 | 4962130 ThirdCoastSurfer
ThirdCoastSurfer's picture

Shorting a stock while a stock price rises is great way to enhace the move. It is also a way to balance the risk creating a "straddle", independent of the options market, but possibly still tied to positions held or created in that market as well. 

Wed, 07/16/2014 - 08:28 | 4962116 ThirdCoastSurfer
ThirdCoastSurfer's picture

This articles example is short on showing profit and the trading activity cannot conclusively prove that HFT's were responsible, but in the milliseconds of an otherwise untraded stock, one can see that the mere execution of a trade triggered someone or something into a preprogrammed strategy. In this example, bids were listed as decoys that were cancelled as soon as a competing bid was received. The pool of liquidity at $17.38 was a mirage. As soon as someone bit, the "who, what and why" was almost instantly analyzed and a corresponding trade execution pattern was initiated. Here it was to vanish and humans cannot act so quickly. It's not hard to extrapolate how this pattern could be used to profit in a more active stock and the corresponding activity in the options market in this example is unknown. The gist of the "rigged" allegation then is that the trades initiated in this example was not executed based on the stock, but on the trade or trader. Again, the poker analogy to trading holds sway as position in trading as in poker is a key determinant of success. Being able to always position yourself as the dealer or button in poker would be a obvious rigged game to anyone who knows it's importance to success.

Wed, 07/16/2014 - 09:48 | 4962371 BobPaulson
BobPaulson's picture

Here's a guess: Now the HFT algo holds a small amount of stock for which it knows there is only a partially filled demand out there. Not sure, but if the action makes the price bump a tiny bit, can HFT now sell to the bidder and steal the arb knowing bidder wants those shares?

Wed, 07/16/2014 - 00:15 | 4961663 Godisanhftbot
Godisanhftbot's picture

 Criminals. All.

 Clearly.

 Unambiguously.

 We let them get away with it, so we're just sheep.

 If they can cancel orders ex post facto ,AFTER an order to buy is placed, then some person or persons should be in jail.

 When will we see that?

 

 

 

Wed, 07/16/2014 - 01:18 | 4961751 JuliaS
JuliaS's picture

No justice - just hired hits and suicides when flash boys step on eachother's toes. That's the only way it'll be for as long as regular gamblers keep throwing fresh money into the pot.

Wed, 07/16/2014 - 01:29 | 4961764 Kelley
Kelley's picture

You should see some arrests the second Tuesday of this week.

Wed, 07/16/2014 - 00:24 | 4961674 qazwsx
qazwsx's picture

good article, thanks ZH

Wed, 07/16/2014 - 05:27 | 4961689 syntaxterror
syntaxterror's picture

The US government is so far up my ass that I can barely breath, yet these motherfuckers get away with this graft, untouched. FUCK THEM ALL!!!

Wed, 07/16/2014 - 00:46 | 4961691 Ralph Spoilsport
Ralph Spoilsport's picture

Good job Nanex.

When "Flash Boys" first hit the media outlets and the sheeple were outraged, it was nice to just shrug and say "Hell, this is old news. Didn't you know about this"?

"So how did you hear about it?"

"Two words - Zerohedge and Nanex".

"Isn't that three words and a conjunction?"

Wed, 07/16/2014 - 00:52 | 4961717 theonewhowaskazu
theonewhowaskazu's picture

I fail to see the problem here. All I see is: 

Somebody puts a market order in (blank check buy X of this shit) 

Somebody pulls some orders before it can execute (I'm not selling you these at the price you thought I was) 

Person is mad because he bought at a price he otherwise wouldn't have bought at... or maybe he would have otherwise bought at but prefers the old price to the new one.

The very simple solution is to not put in "blank check" market orders. That way, if you're buying into fake asks or selling into fake bids, you don't get burned. So long as the market is just a whole bunch of "buy X shares at X price or better" and the inverse for sells, there's absolutely no way for anybody to rig anything, aside from maybe messing with who gets first dibs on the "or better" part. 

Wed, 07/16/2014 - 01:06 | 4961730 Ralph Spoilsport
Ralph Spoilsport's picture

Did you miss the part where Nanex explained that the trader was placing Limit Orders, not Market Orders?

Wed, 07/16/2014 - 01:58 | 4961793 theonewhowaskazu
theonewhowaskazu's picture

No, I missed the part where something bad happened. A limit order with a price above the price you're willing to buy may as well be a market order. If the order doesn't fully execute, then no harm done, now your order is on the books. The orders above you were just ghost orders anyway. The only people hurt by this are those who look at the order book as a way of forecasting price. These people are usually day traders or speculators themselves. Not that there is anything wrong with that, it's just an even better day trader had emerged. Day traders competing amongst themselves doesn't harm the efficiency of the market in any way.

Once again, it's very simple. Amount. Price. Buy or sell. That's what matters in a trade, and no amount of HFT affects that if you're not stupid. Only thing HFT can mess with is a very small portion of the orderbook, which is largely illusory anyway.

Wed, 07/16/2014 - 12:33 | 4962999 g speed
g speed's picture

If I raise chickens they depend on me for food--I go to the feed store ----there are 50 bags on the pallet that I've been waitinng for---I go to the counter and say I'll take those fifty -- Then some sleazy shit butts in front and says "I'll take thiry Im first in line"-- every one knows that the price will go up if there aren't enough bags and my chickens will suffer.----In wall street trader life the counter guy says OK and sells/promises my spoken for bags to the butt in guy -(who will be on the phone and claim there are a thousand bids and the price is up  -then try to get me to buy for a profit or say "I don't want them" after I left and didn't buy from him at the higher price he was going to sell it to me for. )---In real life I meet the shit in the parking lot and smash his fucking face - maybe throw gas in his pickup and torch it-maybe follow him home and kill his dog--you know--stuff he deserves-----

Wed, 07/16/2014 - 11:03 | 4962645 PrintemDano
PrintemDano's picture

You are defending fake bids and fake offers used to induce muppets to lose their shirts?   

Wed, 07/16/2014 - 01:11 | 4961739 JuliaS
JuliaS's picture

A fix to this specific situation would be a buyer's condition that unless the entire order is able to be fulfilled at the quoted price, no order takes place at all. Transaction is cancelled.

The HFT algo is making a bet that the buyer will chase a higher price on the remaining part of the package. If it was instead forced to sit on its own purchased half-order with the client vanishing, then there'd be fewer cases like these...

Though order volume in itself is often a good indication of the distance the buyer is willing to travel to have the shares. Few would turn on a dime and steer half a million $ in a different direction because an HFT decided to skim off pennies. It does become an extortion tax. Want to cross the bridge - pay money to the algo boys at GS or wherever.

Wed, 07/16/2014 - 09:52 | 4962391 BobPaulson
BobPaulson's picture

OK, so you stepped on one roach. The point is that you don't want to be piling on the rules just to chase each way the market rigging can happen. You need a general set of simple rules that stop all the scams.

Wed, 07/16/2014 - 01:50 | 4961788 MrTouchdown
MrTouchdown's picture

It seems there might be a way to make money off the front-running algos. I don't know how, but I'll postulate a bit:

Have a target "buy" quantity, but don't submit them all in one chunk. Let's say 20,000 shares. So you put in a buy for 300 shares, followed by a buy for 10k shares and immediately put in a sell for 300 shares. The first 300 should go through without a hiccup, but the the algos will front-run the 10k - hopefully buying some of the 300 which are now priced higher and are being sold by the company that just bought them. Rinse, repeat and keep going until the target 20k shares are met. Maybe a firm could steal back some of the service charge the algos put on trading.

Wed, 07/16/2014 - 02:19 | 4961807 PaperWillBurn
PaperWillBurn's picture

Former Head of U.S Mint says his friends are too busy inflating stocks with cheap fed $ to buy bitcoin. ha

 

https://twitter.com/EdmundCMoy/status/489203717355216897

Wed, 07/16/2014 - 03:36 | 4961855 MasterOfTheMult...
MasterOfTheMultiverse's picture

So would this be considered a real-time example of "spam-and-cancel"? There is some very interesting research done on this problem by Haim Bodek: http://haimbodek.com/research.html.

Wed, 07/16/2014 - 04:10 | 4961880 shinobi-7
shinobi-7's picture

Congratulation to ZH for a great article. We all know the market is rigged but few care to prove it. This article does it beyond any doubt, great job.

Wed, 07/16/2014 - 05:00 | 4961907 hedgiex
hedgiex's picture

Great job Nanex. Amazing that SEC said that the market is not rigged. Can this Mary Jo White and others define what is a "market rig" or are they so behind the curve and still grasping to understand HFT.

Wed, 07/16/2014 - 05:00 | 4961908 hedgiex
hedgiex's picture

Great job Nanex. Amazing that SEC said that the market is not rigged. Can this Mary Jo White and others define what is a "market rig" or are they so behind the curve and still grasping to understand HFT.

Wed, 07/16/2014 - 07:26 | 4961943 deflator
deflator's picture

Just another skimming operation by decree of government to the benefit of the anointed ones. I know a salesman who sold some floor cleaning soap to the government. He says he has received a $30 commission check every month for the past 22 years. Reminds me of the peanut subsidy scam.

Wed, 07/16/2014 - 06:36 | 4961956 craus
craus's picture

MS-13 and the SEC are similar.
With both you have to pay money to operate.
At least with MS-13 there are no favorites.

Wed, 07/16/2014 - 06:48 | 4961964 hugovanderbubble
hugovanderbubble's picture

Excellent piece. Appreciate

Wed, 07/16/2014 - 07:03 | 4961980 deflator
deflator's picture

 Anyone who has done their own trading for any length of time knows the markets are rigged. About 6 years ago I placed a market order for 10,000 shares of a $2 stock and for a millisecond it spiked .93. The spike didn't even show up as a solid line on the chart that day, it was shaded and then didn't even show up at all at the close. Cost me $3,000 to learn the lesson, "never place market orders on the Nasdaq."

Wed, 07/16/2014 - 07:36 | 4962016 dognamedabu
dognamedabu's picture

never place market orders..period. Never place orders in fact. 

Wed, 07/16/2014 - 13:17 | 4963257 g speed
g speed's picture

I learned about stops back in the day when blackberry was day trading (2006) I traded large cap stock in the $20 to $50 range--  in almost every case blocks of stocks between 100/200 where stopped out.  This in a trending up market.  I used Ameritrade, Scottade, Schwab and others of their ilk using lots of charts, graphs, fundamentals, and trend following to place orders on line for $7+ dollars a trade --In some cases the stops where set down to 10% below the buy price and I always had to wait for "settlement" to invest that money again--usually two days--I guess you guys know the "axe" will get you--most of the money I made in the markets was from big funds (Vangard etc) investing for me in an up market before ETFs----so yeah --if you aint in the club they are rigged against you and you have to be on your game and full time to win at all. I gave most of it up prior to 2008 so I missed the big down--but then again I missed the big up also---but sleep much better.  (remember a 10% loss is more that a 10% gain back)--

Wed, 07/16/2014 - 07:07 | 4961983 Dr.Engineer
Dr.Engineer's picture

Nanex, thank you.  As an engineer, I fully appreciate real world data.

To the SEC, you all deserve to be fired and put in jail for negligence. At the very least, I want my tax dollars back from you because you did not do your job.

End the SEC if we can't end the Fed.

Wed, 07/16/2014 - 07:32 | 4962004 Ban KKiller
Ban KKiller's picture

Even the market is operating as a tool for the fascist oligarchy. Perfect.

Wed, 07/16/2014 - 07:33 | 4962007 dognamedabu
dognamedabu's picture

The markets are rigged?!? No!! Say it ain't so! OK.. Now that I have reconciled this.. How can I make a buck or two to feed the kids? They getting sick of the crappy greasy chicken I'm bringing home from work.

 

 

Wed, 07/16/2014 - 08:24 | 4962108 satan2liberals
satan2liberals's picture

Clear demonstration that SIP quotes are nothing more than a  mirage.

Suggestions of waiting a few seconds to re-submit are useless because they'll just do it a again canceling their offers to sell it to you and buying the shares you're trying to buy, then selling you those very same shares.

Limit orders don't get around this problem either.

I wouldn't mind so much if it were  just a  tenth of a cent per share but what pisses me off is the fuckers won't fill the complete order.

If that how they want to play: Fine, then I'm done with equities and converting over to 100% option strategies where available.

Everything else will be routed to IEX from now on.

IEX is the answer to this problem let the HFT fight it out amonst themselves. 

Wed, 07/16/2014 - 09:27 | 4962266 Sivad UK
Sivad UK's picture

SIP quotes are not a mirage. You just have to understand exactly what they are meant to tell you. They do not represent demand and supply - they simply represent the aggregated totals of bids and offers across numerous exchanges and ECNs. To think they represent supply and demand is a dangerous misunderstanding of the constituant parts that make up the quote. 

Sorry there is a lot f'ed up in regards to these rigged markets and timing of various market data these days but Lewis and most of you guys seem to miss the issue here as it relates to the average retail trader.

HFTs aren't making any money off the little guy. They are simply taking a substantial share of the bid/offer the traditional brokers would have charged you 10 years ago. They are taking money from the brokerage industry, not the little guy. What they give the little guy are much tighter bid offer spreads and price transparency.

In 2000 equity retail bid/offer 12.5c -25c/share PLUS brokerage. Today it's 1c for almost any stock on the planet. Think about that.

One of the main things at issue here is a basic misunderstanding of order routing. That consolodated quote you are looking at is a compilled best bid/offer and aggregated market depth across multipl exchanges and ECNs. The total aggregated amount bid or offered at the best price is not one or multiple bids at a single exchange but the aggregated total of multiple bids across multiple exchanges. When you hit the bid or lift the offer your aggressive order will most certainly be filled at the first exchange you provider routes it to in the full amount bid or offered at that exchange. However that doesn't mean that colocated or cross connected HFTs might see that trade and be able to react on other exchanges before you provider has routed that order to the other exchanges that constituted that aggregated quote.

This is an issue between you and whomever provides your DMA - not you and the HFT shop that is more efficient than your DMA provider. Don't like how your orders are being routed? Change providers.

Regardless of what you do you are still in much better shape than you would have been in the old system where locals virtually raped you both ways on a trade and disappeared when the shlt really hit the fan.

The HFTs don't cost the market - they cost the brokerage industry. They have displaced replaced traditional "local" market makers and study after study has shown they have lowered overall retail tradex costs.

Now - there is some real f'ed up shlit going on with the HFTs and dark pools and people will probably go to jail - but that's an issue between the professionals and they are all in that game together. They also have nothing to do with the poor examples in this article. 

Great article breaking down economics of HFTs on equity markets:

http://pages.stern.nyu.edu/~jhasbrou/Teaching/2014%20Winter%20Markets/Re...

 

 

Wed, 07/16/2014 - 09:47 | 4962363 withglee
withglee's picture

Your link doesn't work.

But assuming you know what you're talking about I have two questions:

1) Would not adding a random number of seconds (or fractions thereof) to all creations of time stamps and then processing in strict time stamp order remove any advantage an HFT would have over a non HFT?

2) Assuming the answer to (1) is yes, then why wouldn't such a simple fix be instituted? What does such a fix break?

Wed, 07/16/2014 - 10:28 | 4962519 Tall Tom
Tall Tom's picture

Random Number Generators are also a product of an Algo. There is no such thing as a Computer generating a set of true Random Numbers.

 

Someone, somewhere, will have a copy of the code for that Algo. With the code it can also be exploited. There is a seed number (That is the key upon which the numbers are generated.) If you know that, and have the code for the algo,  then you can predict what numbers come up and when.

Wed, 07/16/2014 - 11:05 | 4962633 withglee
withglee's picture

Semi-true. Using an algorithm is just one method for generating random numbers. These are pseudo random numbers. It is a good method for studying stochastic processes as it is repeatable (same seed gives same sequence). However, random noise can be generated in many ways which are "not" repeatable but can be made to conform to any distribution. Using such a method and imposing a uniform distribution will do it.

See: https://en.wikipedia.org/wiki/Random_number_generation

Say it is indeed possible to create totally unpredictable unrepeatable random numbers. What is the next objection?

Wed, 07/16/2014 - 13:40 | 4963380 malek
malek's picture

Wow, now you really did a statement (on randomness generation) that is true and complete! Congratulations, I hope it was not random chance.

Now on your order de-rigging:

You proposal is so obviously poor, that it must be deliberate deception.
Why would someone want to add randomness to a "solution" that would work just as fine with a constant delay to all orders?
That has to be a conscious attempt to make it game-able.
Because if someone rigs the randomness assignment process (e.g. during critical market phases all Goldman Sachs orders get randomness mostly in the lower half of the possible range, so execute quicker),
the randomness works in favor of the fraud, as that process feature has no predictable result that outside observers may check.

(Only if the fraudulent party overdoes it, then over long times a very astute observer might end up with a high probability of fraud being in the system.)
As usual you violated the most basic rule "KISS"... (keep it simple stupid)

"An engineer knows he has achieved perfection not when there's nothing left to add, but when there is nothing left to TAKE AWAY."

Wed, 07/16/2014 - 14:13 | 4963519 withglee
withglee's picture

What do we know about how time stamps are assigned now? How are all the clocks synchronized so as not to give someone an advantage (assuming the processing is first come, first served)?

If you add a constant delay to all orders you just delay the entire sequence, but don't re-order it. I think the advantage remains with those who can put themselves at the head of the line.

When you consider that what should be being traded here are investments (rather than gaming) which have much longer life spans than the span of time in which the actual trade occurs, there should be no objection to a little randomness (unless your gaming the system and it makes your gaming ineffectual).

As they say, it takes a thief. I am definitely not a thief. I don't look for ways to game games ... or systems. Can we think of a system where the parties themselves do not, and can not, control the clock?

On our gas pumps we have meters. These are inspected regularly to assure the seller does not have his thumb on the scale (to mix a metaphor). There could just as easily be a cesium cube that sets the timestamps. The exchanges would have no control of it whatever. The regulators would have no control of it either. But it would be easy to confirm its functionality by injecting it with time stamps and observing the distribution of time stamps it outputs.

Remember, this isn't about accuracy. It's about making the type of gaming we're observing (and a never ending race to be faster) unworkable.

 

Wed, 07/16/2014 - 14:31 | 4963598 malek
malek's picture

I would think the rulemaker better should be able to think like a thief, when trying to design good rules.

You are confusing effect (trade reordering) with cause (shorter time between trade input and trade execution, for some, by using faster electronic communication paths).

You still don't get it.
True randomness for all involved parties is a non-verifiable feature. Only the generating authority knows for sure if all parties are treated equal or not.
Once again you try to solve something by introducing a fairytale never-corruptible entity that magically falls from the sky.

Distibution tells you exactly nothing about a single random number.
As random numbers can be generated limitless, you need to cover the fact that rigging entities could do things like "Oh, I don't like that random number, give me another! Oh that one is also not so good, another! Ok I take that one, thanks."

You confuse accuracy with accountability, i.e the ability the check and verify things. In a very important spot your proposal disables verification ability completey.

Wed, 07/16/2014 - 15:10 | 4963846 withglee
withglee's picture

I would think the rulemaker better should be able to think like a thief, when trying to design good rules.

That's what I aluded to with my comment "it takes a thief". Are you presenting yourself as such a thief?

You are confusing effect (trade reordering) with cause (shorter time between trade input and trade execution, for some, by using faster electronic communication paths).

No I'm not. I eliminating the advantage of being faster. There are many steps in the process. If you make the period taken by each step statistically unpredictable, you have great difficulty gaming that step and all the steps that make up the process.

You still don't get it.

Are we to assume you do?Enlighten us.

True randomness for all involved parties is a non-verifiable feature.

Oh really? Are there other non-verifiable features of the current system? Like proper assignment of a time-stamp and proper processing in first come, first served order? What's verifying that?

Only the generating authority knows for sure if all parties are treated equal or not.

And who (what) would that be?

Once again you try to solve something by introducing a fairytale never-corruptible entity that magically falls from the sky.

And we benefit from your contributions how? Once again you declare the solution invalid because it will be gamed ... not because it doesn't achieve the desired result.

Distibution tells you exactly nothing about a single random number.

Are you clueless about statistics and distributions? Are you clueless about methods of prediction and what makes them difficult?

As random numbers can be generated limitless, you need to cover the fact that rigging entities could do things like "Oh, I don't like that random number, give me another! Oh that one is also not so good, another! Ok I take that one, thanks."

You enter an order. You get a time stamp.

You confuse accuracy with accountability, i.e the ability the check and verify things. In a very important spot your proposal disables verification ability completey.

You provide your own details and attribute them to me. Perhaps you would like to describe to us the mechanisms that make assignment of time stamps verifiable today.

Wed, 07/16/2014 - 20:16 | 4965081 malek
malek's picture

Playing dumb again when you run out of arguments?

I eliminating the advantage of being faster.
If you read attentively, I did not dispute that.
I only pointed out that there are other ways to do that, which do not add new potential problems into the mix.

True randomness for all involved parties is a non-verifiable feature.
And it will forever stay a non-verifiable feature, while many other things can be made verifiable through public bookkeeping etc.
It seems you are clueless that randomness works both ways - you can never prove the same randomness is applied to all parties requesting random numbers, and it's very hard to disprove it.
But hey, just "trust us!"

You enter an order. You get a time stamp.
Assuming the giving entity is honest.
Will you ever learn to automatically ask the "what if" question: What if the entity is rigged?

Thu, 07/17/2014 - 12:18 | 4968067 withglee
withglee's picture

Playing dumb again when you run out of arguments?

I eliminating the advantage of being faster.
If you read attentively, I did not dispute that.
I only pointed out that there are other ways to do that, which do not add new potential problems into the mix.

And I pointed out to you your solution of delaying all orders by the same amount has no more relative effect than the tide that raises all boats.

True randomness for all involved parties is a non-verifiable feature.
And it will forever stay a non-verifiable feature, while many other things can be made verifiable through public bookkeeping etc.
It seems you are clueless that randomness works both ways - you can never prove the same randomness is applied to all parties requesting random numbers, and it's very hard to disprove it.
But hey, just "trust us!"

So, record the actual time stamp and the random delayed time stamp both. Now you have a complete audit trail. Kind of silly since you don't question the actual time stamp itself ... but does allow you to be argumentative.

You enter an order. You get a time stamp.
Assuming the giving entity is honest.

You're assuming it now.

Will you ever learn to automatically ask the "what if" question: What if the entity is rigged?

Have you been asking that? Ask away. Then solve. Your attack vector of predicting the random number generated by an algorithm is ridiculous ... but I let you have it. You are just being argumentative. Yet you avoid my main questions: (1) Do you have any knowledge of how timestamps are generated now and how the exchanges synchronize their clocks. (2) Assume the randomization can be accomplished verifiably and incontrovertibly. Does it make the HFT advantage go away? If not, then why not?

Fri, 07/18/2014 - 00:45 | 4971891 malek
malek's picture

You're really clueless on features of randomness. But accusing others of not knowing what a distribution is.

delaying all orders by the same amount has no more relative effect
Once again you ignore all second round effects. If all orders are delayed by the same amount, let's say one second, traders can cancel their order if they see they are are getting frontrun (as you have proposed nothing disallowing cancelling).
I personally don't like the delay approach, I'd prefer that orders get a minimum lifetime enforced, i.e. one cannot cancel an order before 2 seconds have passed.

"a complete audit trail [of randomness]"
Tell me of what help that would be. Auditing/accounting/bookkeeping only works on predictable things, so you can verify if outcome matches precise forecast.

implying I proposed predicting randomness as an attack
Once again you're so stupid it's beyond words.

Fri, 07/18/2014 - 10:22 | 4973056 withglee
withglee's picture

You're really clueless on features of randomness. But accusing others of not knowing what a distribution is.

Fine, it is I who am clueless. Give me clues. I've asked two specific questions. Let's see how the "clued" answer them.

delaying all orders by the same amount has no more relative effect
Once again you ignore all second round effects. If all orders are delayed by the same amount, let's say one second, traders can cancel their order if they see they are are getting frontrun (as you have proposed nothing disallowing cancelling).

I personally don't like the delay approach, I'd prefer that orders get a minimum lifetime enforced, i.e. one cannot cancel an order before 2 seconds have passed.

Perhaps you would like to dissect the process to prove your point ... or to prove mine. Start with a non distorted series of time stamped events that expose the chicanery. Then, transaction by transaction, delay (cumulatively) the following transactions. You will see that you have just given the system a constant latency. The HFT advantage remains and you will discover that their transactions still get in front of the others. Remember, you can not respond to an event before it occurs.

Next do the same exercise but using a random delay. See how the transactions get rearranged and how the information assumed as each transaction occurs is no longer as before ... and so the response must be different. The advantage of speed it completely lost. The HFT algos fail if they aren't assured of being able to predictably respond to events faster than the other players ... in "all" instances.

To undersatand the concept, look into how low level "random" noise is applied to analog signals before they are digitized by an analog to digital converter. This eliminates bias at quantization boundaries. Now consider the effect of adding a low level "constant" deviation. You don't solve the problem of quantization at boundaries at all ... you just move it.

Regarding cancellations, I don't think you have to impose any rules on them as they only work if they can be done before anyone can react to the original order. You might simulate the rules you would impose to prove your (or my) point.

"a complete audit trail [of randomness]"
Tell me of what help that would be. Auditing/accounting/bookkeeping only works on predictable things, so you can verify if outcome matches precise forecast.

You were the one complaining about verifiability. I thought your complaint was with the random delay being applied to the transaction. If you don't retain the original time stamp, the size of the delay is lost. So I'm just trying to accomodate your argumentation. I still don't see it adding an attack vector that doesn't already exist. And if it does exist and you move to mitigate it, you mitigate it for both the original time stamp and for the random delayed time stamp. I again ask, how are timestamps created now? Perhaps the problems you anticipate already exist ... or the mitigation is already in place.

implying I proposed predicting randomness as an attack
Once again you're so stupid it's beyond words.

Fine. But please keep trying to find the words. And in the meantime, prove your own competence by answering my questions. (1) Does a random delay mitigate HFT issues? (2) Does a constant delay have the same effect? (3) Does a constant delay mitigate the HFT issues? (4) How are timestamps currently attached and how are clocks synchronized?

Let's try to stay on point here. It's not often I get to interface with true genius. I admit I am not in the genius class. Please bear with me. True geniuses don't pull rank. They prove their points.

Fri, 07/18/2014 - 14:59 | 4974760 malek
malek's picture

Remember, you can not respond to an event before it occurs.

Maybe I was making too intelligent assumptions about your proposal.
I was presuming your delay is on the execution time, but the visibility of entered orders is not delayed. If you delay visibilty, then you have even two possible points of failure: that the randomness is skewed towards some parties, and in addition that the hiding towards all participants is uncovered earlier for some.

I think you should spend some time pondering the effects of cost-free cancellation of "orders", even when done so in vast numbers.

"a complete audit trail [of randomness]"
Tell me of what help that would be. Auditing/accounting/bookkeeping only works on predictable things, so you can verify if outcome matches precise forecast.

You were the one complaining about verifiability.

Maybe you should also read and understand why I was complaining about verifiability:

True randomness for all involved parties is a non-verifiable feature.
And it will forever stay a non-verifiable feature, while many other things can be made verifiable through public bookkeeping etc.

It seems you are clueless that randomness works both ways - you can never prove the same randomness is applied to all parties requesting random numbers, and it's very hard to disprove it.

And then you come and propose audit (a/k/a bookkeeping) as a "solution". Epic fail.
Do you now finally grasp that true randomness (especially in all dimensions: not just over time, but also over parties that received supposingly random numbers etc.)
cannot be proven, even with auditing?

Fri, 07/18/2014 - 17:53 | 4975490 withglee
withglee's picture

I surrender. You are just way too smart for me.

Fri, 07/18/2014 - 19:15 | 4975721 malek
malek's picture

I'm sure deep inside you're certain to be an unrecognized genius.
Keep on clownin'

And I will keep shooting you down if you go out too far on a limb...

Fri, 07/18/2014 - 21:50 | 4976181 withglee
withglee's picture

Get a life.

Sat, 07/19/2014 - 14:01 | 4977415 malek
malek's picture

Don't project from yourself onto others...

Wed, 07/16/2014 - 10:04 | 4962427 satan2liberals
satan2liberals's picture

uk whatever

1)SIP quotes are not a mirage. You just have to understand exactly what they are meant to tell you. They do not represent demand and supply

 

 

1) Not a mirage eh?

I just told you they don't represent supply or demand.

So what the fuck are they then? 

 

2) One of the main things at issue here is a basic misunderstanding of order routing. That consolodated quote you are looking at is a compilled best bid/offer and aggregated market depth across multipl exchanges and ECNs. 

2) Really?

 So you're claiming these compiled quotes cannot see my orders though when submitted? Meanwhile the price  just magically dances away eh?

LOL 

 

The  rest of your post is just attacking strawman arguments I didn't make.

I never said trading costs have not improved nor said I was being gouged.

I'm of the opinion that if you're gonna be so bold as to tender a bid or offer , you should be required to keep it in effect long enough to reasonably be executed against.  If you can't do that take your pussy ass somewhere else with your fake bids /asks and let those whoi actually want to engage in mutual commerce do so.

 

 


Wed, 07/16/2014 - 16:55 | 4964271 Sivad UK
Sivad UK's picture

.

Wed, 07/16/2014 - 16:52 | 4964272 Sivad UK
Sivad UK's picture

1) They represent the aggregated total of bids and offers across multiple exchanges and ECNs. With respect, you are applying the concept of supply and demand incorrectly here. Supply and demand doesn't hold across multiple discrete markets. They only apply to an individual market/exchange. If the universe of exchanges is A, B, and C then in our current market the order books of A, B, and C all have full integrity. What you are missing is that with a single DMA provider you can put a bid in in A, and then you can put the same bid in with B and C but make those bids one cancels other. You only wanna buy 10 shares, but you want to make sure you get them regardless of where they trade. Your aggregate demand is 10 shares. However in a consolodated SIP world your orders will show up as 30 shares, your aggregated bid across 3 discrete markets. If you don't understand that SIP data is aggregated you will misunderstand what your feed is telling you.

2) How your DMA provider routes orders is between you and your provider. It is your job to understand this. He may send your order to exclusively A. He may send it to A and B or all three. It's your job to learn how your DMA provider routes orders. Once you understand that you can adjust or get a new provider. It's not the HFT firms fault. They spend a lot of resources and capital accessing each exchange individually where you are paying a middle man to decide where to send your orders first. This is a big boy market. It's not Fischer Price my first Trading Platform. Figure out your needs and structure your access to the market accordingly. Blindly picking a middle man without understanding how he accesses the market doesn't give you a license to whine when your Fiat doesn't perform as well as the Porsche the other guy has. Is it fair? What's fair? 

 

Wed, 07/16/2014 - 09:39 | 4962327 withglee
withglee's picture

All they have to do is add a random number of seconds to all transaction time stamps and then process in time stamp order.  Poof! Problem goes away. HFT can continue but it can't work to any advantage except to market efficiency. It can't be used for front running because it has no way of getting in front.

Thu, 07/17/2014 - 15:47 | 4969666 MeelionDollerBogus
MeelionDollerBogus's picture

That doesn't address at all that one's trade is front-run so shares available are removed. A cancellation can always get there faster than an attempted fill & everyone would face the same random delay so you'd have to be very lucky to beat the machines.

Thu, 07/17/2014 - 15:56 | 4969673 MeelionDollerBogus
MeelionDollerBogus's picture

Also: this doesn't address how super-fast quoting of prices can crash parts of exchanges so some people will get no information at all, or their trade will be held up much longer than some random number of seconds.

Wed, 07/16/2014 - 09:18 | 4962231 Notsobadwlad
Notsobadwlad's picture

The article and Michael Lewis (intentionally) miss the real market rigging.

As long as there are market makers and specialist (firms, people or computer programs), WHO CONTROL PRICE, and a buyer of last resort that has infinite money (created out of thin air) to purchase all shares (or sell an infinite number of shares short without having to cover) THE MARKET WILL ALWAYS BE CORRUPT AND SCREW THOSE WHO DO NOT HAVE THE ABILITY TO CONTROL PRICE AND CREATE INFINITE MONEY.

The system, as is, is irreparably broken and criminal and getting rid of front running HFTs will do nothing except expose that the real fraud still exists.

Wed, 07/16/2014 - 10:14 | 4962468 rsnoble
rsnoble's picture

The gov't likes a lofty market, if the HFT's help then that's why they aren't doing anything.

Yep, everythings a farce that's for damn sure.  Of course i'm not sure if it always hasn't been that way just greater scale now.

Wed, 07/16/2014 - 11:38 | 4962729 chinaboy
chinaboy's picture

Market riggers laugh all the way to the bank, all the way to the SEC, all the way tp hell, and back.

Wed, 07/16/2014 - 12:14 | 4962879 satan2liberals
satan2liberals's picture

You might find this IB webinar detailing high frequency latency trading

informative.

https://interactivebrokers.webex.com/interactivebrokers/lsr.php?RCID=670...

Wed, 07/16/2014 - 12:17 | 4962881 satan2liberals
satan2liberals's picture

duplicate

Do NOT follow this link or you will be banned from the site!