How The Pursuit Of "Light Speed" Broke Equity Markets, Or Why The NBBO Is No Longer Relevant

Tyler Durden's picture

After clearly demonstrating that last year's flash crash was essentially a byproduct of massive quote stuffing-induced churn surge, which cascaded into a full blown liquidity collapse, coupled with the bulk of HFT "liquidity" providers simply turning their machines off and subsequently reverberating by ETF "amplifiers" to the nth degree, last week Nanex released what is probably one of the most critical white papers on why in its pursuit of ever higher speed (or, at least speed that is physically capped by the theory of relativity at 300,000km/sec) to gain a frontrunning advantage over everyone else, courtesy of a massively two tiered market in which there is the collocated Ph.D. braintrust focused on millisecond trading (day trading is so 20th century), and then everyone else, the core premise of the "fair and efficient market" - the National Best Bid Or Offer, which is at the heart of Regulation NMS which in turn sets forth the guiding principles behind modern "capital markets" is now an anachronism and is being overrun on a daily basis as the fastest to the market gets to set just what their own NBBO is, in the process literally destroying the premise of market fairness and efficiency, as those who have the newest and shiniest toys are guaranteed to win, while everyone else fights for a fraction of the scraps. Said otherwise, steroids are forbidden in sports but when it comes to capital markets, they are very much accepted.

The reason for this bold claim is that the Security Information Processor, the aggregator of NBBO data from the Consolidated Quote System is no longer used and instead exchanges, of which there are hundreds courtesy of fragmentation, make up rules literally on the fly. The implications of this are dramatic and imply that in order to restore market efficiency as envisioned by Reg NMS, the way modern market topology is evaluated will need to be gutted and overhauled completely, in the process eliminating the bulk of parasitic HFT, which in turn will reduce market volume to an even worse trickle, leading to more Wall Street layoffs, higher equity market volatility, a greater facility for market manipulation by the FRBNY, and an increased exodus by daytraders to other exchange and OTC-based markets such as commodities, rates and FX. In other words, the daily flash crashes we have so grown to love in stocks are coming to a 10 Year near you.

Full text from Nanex:

Executive Summary

The NBBO lies at the heart of Reg NMS and is the key concept that assures investors get the best price when buying or selling stocks. However, due to the industry trend that emphasizes speed at all costs, the NBBO, in practical terms, no longer exists. There is no audit trail that can show definitively whether an investor received the best price on their trade. The regulators do not appear to understand the root of the problem because they continue to promote new regulation, when a simple and effective solution exists: enforce Reg NMS. Likewise, getting rid of quotes that only serve to manipulate others is as easy as enforcing Section 9 of the Securities Exchange Act of 1934.

If any new regulations are needed after enforcing existing ones, we think a minimum quote life of 50 ms makes the most sense. Just the discussion of implementing such a rule would expose how deeply flawed the system is today and would be sure to raise a lot of eyebrows. But to be clear, we do not advocate new regulation.

The Speed Of Light

As the lifetime of a quote approaches zero, the arguments for and against a minimum-quote-life rule become more interesting. Let’s suppose, for example, that the day has arrived where a few of the top HFT systems are able to send and cancel quotes in 1 nanosecond (ns). For reference, light travels 30 cm (1 foot) in 1 ns. At this rate, we could see 1 billion quotes per second per stock (qpss). A thousand active stocks trading at this rate would generate 1 trillion quotes per second (qps).

OK, that was an extreme example, but with all the hyperbole from some HFT marketing groups, we couldn't resist. So let’s slow things down by a factor of 1,000 and imagine a world where HFT systems can send and cancel a quote in 1 microsecond (us). At this speed we could expect 1 million qpss and a thousand stocks would generate 1 billion qps.

Still extreme? Let's make things 1,000 times slower again and imagine a world where HFT systems can send and cancel a quote in 1 millisecond (ms). At this speed, we could expect 1,000 qpss and a thousand stocks would generate 1 million qps. At this rate, many quotes will expire before leaving exchange networks. Today qpss rates of 2,000 occur frequently, with peaks in the 5,000 - 30,000 range. These numbers are growing rapidly.

"No one uses the SIP for the NBBO anymore"

Perhaps the definition of a quote is what needs to be modernized, or maybe we need a new name for what used to be called a quote. Not long ago, when a trader (or auto-trading software) received a quote marked auto-execute, there was a reasonable expectation of hitting that quote — the only real exception being that another trader might beat them to it. Today, under the same circumstances, the same auto-execute quote would almost certainly have expired during the same time period.

The change in semantics becomes most important when we look at the definition of the NBBO. Per Reg NMS, the NBBO for a stock is defined to mean the best bid/offer sent by a market center to the Security Information Processor known as the SIP (CQS, UQDF). But no one uses the SIP for that anymore, we are often told, which would seem to violate the letter and spirit of Reg NMS.

So what do they use?

Each exchange computes the NBBO internally from their direct connections to other exchanges. As the speed of trading increases, the likelihood of two exchanges having the same NBBO decreases. Most of this is because of the pesky speed-of-light limitation.

So how does a trader know whether a trade was routed properly to the exchange with the best price?

He doesn't. It is impossible.

You see, each exchange’s view of the other exchange prices only exists in memory on that exchange's machines. It is not recorded. There is no audit trail. Sure, each exchange provides book-level data, but that only includes prices for that one exchange — not the prices that existed on the other exchanges at the time of each order.

"It is impossible to verify that a trade received the best price"

If we are going to allow machines to trade faster and faster, then at the very minimum, there must be audit trail data that includes each exchange’s view of top-of-book quotes for every linked exchange trading that stock. In other words, what now only exists in an exchange routing computer’s RAM, needs to be captured and made available. The exchange routing simulation video below may help you visualize this. In the video, each box represents one exchange and the price information it has from the other exchanges -- that is the information which needs to be recorded to assure trade through price protection. Essentially, each interlinked exchange would end up recording its view of every exchange's top-of-the-book prices -- exactly what the SIP (the box at the bottom) does now.

So why not use just use the SIP? Auditing, maintaining, and improving one system is much better than 14 (one for each exchange). Well, because, as some more informed people (including the SEC), have told us: The SIP is not as fast because it has to aggregate information from the other exchanges.

Wait a minute.

How can an exchange insure every order receives the best prices, if it doesn't aggregate information from all the other exchanges? Take a close look at the simulation video and note that every exchange must essentially replicate the SIP's aggregation function. There is no way around this. Which means the claim that aggregation causes the SIP to run slower is absurd. Quote rate overload is the primary cause of latency in the SIP and direct feeds.

"The claim that aggregation causes the SIP to run slower is absurd"

There is more.

A computer that aggregates information from other computers uses an aggregation policy that details how it selects between messages coming in at the same time from multiple computers (things like time-slice period, scheduling quantum, size of input queues, overflow behavior, etc.). Let's just say it's very complicated and has many dependencies. The aggregation policy is important because it affects the 3rd criterion in NBBO selection: price, size, time. The SIP's aggregation policy is something that is fairly easy to verify. How easy is it to obtain and verify the 14 aggregation policies used by the 14 other exchanges?

It gets worse.

We have discovered a few anomalies when analyzing aggregation characteristics of CQS, which is the SIP for stocks listed on NYSE, AMEX, and NYSE Arca (home of many ETFs). One disturbing anomaly is that the SIP appears to be ignoring the timestamp in the quotes it receives from an exchange, using instead the actual time the SIP receives the quote, even if the timestamp in the quote is over 5 minutes late. That is, we found an example of the SIP treating 5 minute old data as real-time, affecting the NBBO in thousands of stocks. We will be publishing the results of our analysis shortly. The main point, is that this type of detection is impossible to carry out for individual exchange aggregation behavior; and if it was, it would take 14 times more work.

"We found an example of the SIP treating 5 minute old data as real-time, affecting the NBBO in thousands of stocks"

We understand how difficult it can be to grasp these problems. Understanding complex networked systems where the speed-of-light is the dominant source of latency is hard. When the fate of your nation's financial infrastructure is at stake, we think regulators should have a solid understanding of these issues.

Finally, we'd just like to point out to those who say no one uses CQS, that last year, 2.5 million subscribers spent over $450 million in exchange subscription fees to receive and process CQS. We think they deserve better.

Exchange Routing

The video shows how orders flow between linked exchanges trading one stock. Orders are represented by triangles, and begin from the edge of the display and feed into one exchange. That exchange then updates and checks its internal table to determine whether to route the order to another exchange with a better price, execute it locally, or simply update the top of the book for that exchange. Only orders that affect the top of the exchange book are included to reduce clutter. The order is then transmitted to every other exchange making a market in the stock (via premium direct exchange feeds) as well as CQS which is shown at the bottom. When other exchanges receive the order, they update their internal tables in order to keep track of the other exchanges' top-of-the-book quotes.

The price data within each box therefore represents how that exchange views the prices on other exchanges trading that stock. Unlike this simulation which assumes a perfect world: due to the variations in distance and connection quality, system load, and other real-world imperfections, exchanges won't always have identical NBBO information. As update rates increase, the percentage of time that all exchanges have identical NBBO information will rapidly drop to zero.
Download the exchange routing simulation application. Simply unzip to an empty directory, and drag-and-drop one of the text files over the executable.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
HistorySquared's picture

maybe we should just give an exchange a monopoly, that'll probably help. or let's go back to getting screwed by the specialists and paying 1000 times more in commissions. or perhaps we should institute a tobin tax, drive all the business offshore, decrease volume, and increase volatility. or wait, maybe the algorithms are doing their job and arbing out the inefficiencies. 

malek's picture

You seem to have missed the point completely. They are arbing the "inefficiencies" all right, but they will never "arb them out" as the inefficiencies (unfairnesses) are written into stone (rules carefully constructed for that purpose.)

HistorySquared's picture

they have driven latency to from milliseconds to nanoseconds. Anyone can invest and have the same setup to try to reduce that further. what exactly is unfair. the only unfairness comes in when the government comes in and grants a monopoly to "protect us" from those nasty models that are the reason retail investors can't make money. please. the question remains, what do you want to do about this so called "problem?"  

Marco's picture

Take in bids and offers and execute them delayed and en blocke, remove the advantage of latency altogether.

Rynak's picture

Exactly, forget complicated patches - just address the root issue itself: That it is even possible for the price to change in ns frequency. Simply reduce the resolution.... to, let's say 5 seconds.... and then execute orders in 5sec quantums - all HFT problems solved, and no workarounds possible.

malek's picture

Minimum bid/offer lifetime of 50 milliseconds, unless filled.
Which includes originator timestamps, by design.

Marco's picture

We have exchanged spreads for volatility ... it seems a rather poor trade to me.

HistorySquared's picture

What facts and statistics are you using to say volatility is out of control right now? The fact is volatility sucks right now, and the Greek "flash crash," that was by the way initiated by a long only fund, not high frequency algorithms, was 1/4th the size of the October '87 crash.  From a day in and day outperspective, large orders temporariliy move stocks 25 cents when they used to gap 50 cents to a 1.00 as specialists sought to fleece institutions. This is exactly because there are so many bots that come in to play market maker. 

Regarding your solution, can you imagine what a chart would look like with your solution to trade once every 5 seconds. During volatile times, it would be print, then a massive gap in prices, print, massive gap in prices. It's just a mini version of closing the market, then opening up with a gap in the chart. You're basically taking away liquidity when the market wants and needs it, trapping traders in their positions - like a mini version of a runaway limit moves in the futures market. I can't even fathom all the unintended consequences of your solution, but whatever they are, it's certainly not a drop in volatility. The rest of the world is still open remember. 

If you really want to create volatility, try to artificially suppress it. 

Rynak's picture

Oh noes, i'm "trapping traders in their decisions", and "closing and opening the market"...... for 5 seconds.....

i think i'm gonna cry.

traders? WHAT TRADERS? Humans cannot react faster than in a matter of 5 seconds anyways. The only ones who are affected, are bots...... so what are you complaining about.... that i discriminate the poor bots, because they can no longer outrun humans? Uh, yeah, that's precisely the point - plus heterogenous latencies.

And as for volatility - as you said yourself, liquidity/volume decreases with lower resolutions.... therefore, the argument that the same volume would be concentrated in steeper steps makes no sense at all. When resolution is lower, you don't make 10000 orders per second. You make just one, WAIT for the reaction, and THEN respond. What this means, is that crash/boom like situations will be just as steep as they are now, but with larger steps.... but all the artificial volatility, resulting from milisecond bitwars is taken out. Thus, the only thing that would change, compared to now, is HFT-induced  volatility - everything else would stay the same (of course it would! after all, how did we manage to trade in an orderly fashion, before HFT? If you remove what makes HFT possible, the market turns back to the same as how it was before HFT).

Again, we're talking about 5 seconds here. This CANNOT affect human trading... it cannot matter for human-made decisions.... it can only matter for bots creating millisecond wars, and frontrunning humans.

TBH, at this point, considering your above posts, it is quite clear that you are simply a HFT advocate.

CalibratedConfidence's picture

did you ever hear of the Flash Crash....that little bit of bullshit cost a lot of people a lot of money...maybe if it happens to you, you'll see things differently.


If your defending this, your a fucking retarded and a drifter here at Zerohedge


Jw n FL....nice post :)

Rynak's picture

Uh, what? A little bit of context here? Or was the post to which you replied removed, because it got junked into oblivion?

aminorex's picture

Disagree, I do, but please don't junk this guy.  He's making the devil's point, which needs to be heard.

More_sellers_than_buyers's picture

This is what you get when the regulators listen to the loudest guys in the room ie. Goldman.  To hell with the public and the creation of the greatest tool in the history of mankind to raise capital(The NYSE)...The Goldmans of the world wanted no interferance with the public on their quest to squeeze every damn nano penny out of everyone else.  As a former specialist I can guarantee you that not all of us were screwing you.  I can't tell you haow many times I told the big firms to go screw themselves because they were looking to screw the public.  I used to be able to stop them...then they just changed the rules so they could do it anyway, and eventually they got rid of us all.  I stood up for the little guy every damn day and I was not alone.  Now, there is NO one to stop them.  Its a damn shame if you ask me.  Remember, we may have been middlemen, but no one ever knew if I was long or short.  I was an impartial middle man who had a fiduciary responsibility to stand up for all the players and I had to answer for every single trade I made.  Sure it took 2 or 3 seconds to execute a market order back then, but I ask everyone were we not a hell of alot better off then than now??

lolmao500's picture

And we finally an answer to the question... Were republicans full of it all this time about the debt ceiling?

House Republicans brace for compromise on debt
Oh regional Indian's picture

India had it's own little flash crash Friday. After hours at that.
Some 1,500 outsized trades were unwound.

The system is reaching out, testing, testing, relentlessly testing. HFT is already here....just warming up.

Of anecdotal interest perhaps:

"ICICI was India’s most prolific card issuer during the boom but halved its cards portfolio to about Rs 4,800 crore ($1.08 billion) at end-March 2011 versus 2008. Private lender HDFC Bank and foreign banks such as Citigroup and Standard Chartered are the most active card issuers in a country where state banks have 70 per cent of the overall loan market.

Card usage is low in India, with total spending of Rs 75,500 crore ($17 billion) in the year that ended in March, central bank data show, equivalent to just 4 per cent of retail sales in an economy where more than 90 per cent of shopping takes place in mom and pop stores.

Between 2005 and 2007 the number of cards in India jumped by 50 per cent, peaking that year at more than 26 million. After the financial crisis, it fell to 18 million. Indian consumers, on the whole, have not fully embraced the idea of using credit cards, preferring debit cards instead, with roughly 230 million in circulation."

The above quote from:

Trends are clear eh?


Imminent Crucible's picture

Tyler managed to cram 222 words into his opening sentence, a new record. A flash crash in high-frequency verbiage.

malek's picture

Full stops are for punters.

hbjork1's picture



lolmao500's picture

I love the Orlando police...

This guy understand what the new America is all about... Hilarious...

Bastiat's picture

Awesome!  Another good story I stumbled across today:  a guy took viagra and wore sweatpants with no underwear to the airport--he went through security 4 times until he got a full pat down.  The TSA officer resigned after that, saying:  "I'm going home to make love with my wife.  I never want to touch another stranger." 

JW n FL's picture


NASHVILLE INT’L AIRPORT — A Wyoming man walked through a TSA checkpoint with a raging erection on Tuesday, daring TSA officers and even fellow passengers to give him an invasive pat down.

“I’m next,” Warren Kelvin, 34, screamed as he pushed to the front of the security line. According to TSA officials, Kelvin had ingested two Viagra and wore sweatpants without boxers for his Southwest flight from Nashville to Phoenix.

“I thought he was carrying a baton in his pants,” said Amanda Watershed, second shift supervisor of the A Terminal at Nashville International Airport. “Nope… That was his penis.”


Thanks to the original poster!!!

lolmao500's picture

Yeah well that story is fake. Sorry.

Bastiat's picture

Oh damn.  It looks like an Onion sort of site.  Ah well, my optimistic bias led me to accept it too quickly.    On the other hand there is a true story, linked on Drudge about a Colorado woman who got arrested and charged with a felony for groping the breasts of a TSA agent.  It would be an interesting trial if it came down to what makes it OK for the TSA agent to do that all day but not OK for the defendant.  What ever happened to tit-for-tat?

Seriously:  is *anything* OK if you're wearing a uniform and following orders?  Molesting children clearly is, how about body cavity searches?  What if one was ordered to use his penis in a body cavity search because it is longer than a finger?  Would that be OK?

I hope someone actually does the Viagra/sweatpants routine--political guerilla theater.



John_Coltrane's picture

Still, its mighty funny indeed!  Stories like that and the fact that issues like the speed of light and violation of special relativity are mentioned in financial discussions distinguishes this site from any other and will have me coming back for more.  I believe the uncertainty principal, the Fermi exclusion rule that keeps us from falling through floors, Godel's undecideability theorm, and the equivalence of inertial and gravitation mass should be included in future articles and discussions. 

Waterfallsparkles's picture

Wonder if he was gay or just showing off.

JW n FL's picture
by lolmao500
on Sat, 07/16/2011 - 13:20

I love the Orlando police...

This guy understand what the new America is all about... Hilarious...





Fantastic!!! We need are more Police wearing more Body Armor than the Marines get to use on the Front Lines!! That will save America.



ebworthen's picture


This would explain the shake outs, the wildly oscillating spikes, the algos hitting the networks with different trade frequencies and amplitudes so they can feel out the speed and responses of other algos and ad-hoc control measures.


slewie the pi-rat's picture

LOL  i'll be out by the tree with a few silver dollars & FRNs if you wanna trade stuff...

i haven't "traded" since the 20th C.  can ya still use limit orders?  all or none? good for the day?

too old-fashioned?  unless you're all chromed out in hi freek, i mean. 

John_Coltrane's picture

I want more conditional orders where if the price goes down after I'm filled I get to cancel and rebid (oh- that's the HFT stuff isn't it)

cosmictrainwreck's picture

+100 you get my vote for new SEC chair...uh, spot's already taken, you say? shit

StychoKiller's picture

"A Trade Supreme!" -- paraphrase of John Coltrane

lolmao500's picture

What's gonna be nice if the US default is that the municipalities bond market will blow up, bankrupting hundreds if not thousands of cities... which will force them to fire tens of thousands of cops... YAY!

superflyguy's picture

That's not very good.

What will they do? Do you think they'll suddenly start farming? Or work somewhere else that doesn't involve arresting and treating people like criminals?

In my opinion, there are several jobs they're most likely to take first after they're kicked out:

- join TSA

- join private security companies (become contractors)

- organize into a criminal organization and terrorize everybody else

- join military (not likely, first it's too dangerous and second, they're laying off to - drones are better soldiers anyway)


delacroix's picture

they are already employees, of a criminal organization

pesamystik's picture

You have to love how our patriotic congress called congressional fucking hearings over steroid use in sports. I mean, we're talking about a few baseball players who juiced and we're going to get the U.S. Congress involved? Meanwhile, we have billions being scalped in rigged markets, and not a peep.

To be fair, average joe six pack is a fucking moron and probably cares more about Roger Clemens shooting HGH up his ass than whether some Wall Street shyster is scalping pennies off his paycheck every minute. You had Schumer a few years ago write some half-hearted letter about flash trading, but none of it really dealt with HFT. In fact, since then, the HFT lobby has made sure nobody is going to make a real issue of this. 

It's a Wild West darwinian battle, and everyone but the well capitalized whiz kids are winning. You are losing. This is just one more racket in a long line of rackets cooked up by the elite.

Slightly unrelated to HFT (but all these things are sort of interconnected now), here is an interesting bunch of graphs about the increasing workload of Americans (the article is good too).

oldmanagain's picture

The ultimate chart trend is the top percentages wll just hire robots. Animal spirits will have been fulfilled.

oldman's picture

Hey, bro',

I woke up two days ago with the sense that this has already occurred. Without humans in the loop the virtual reality works perfectly except, for 'animal spirits'; now all it has to do is feed us or---------------------------------------------?

I think that we are locked by our own minds in a place that no longer matters. Your last line in line with my sense of this is changed to:

Animal spirits have been fulfilled.

Thanks, oldmanagain- om

oldman's picture

"It is impossible to verify that a trade received the best price"


How can one be certain that a trade even occurred?

It seems more and more to this oldman that this virtual farce of a market is only evidence that the machine has self-destructed.

One of these days it will say, "we stopped you out and took the money from your account, the additional from your bank, but there is still a balance due to be paid now."

And you will say "----but, I had no position."

"Yes, you did-------------you lost---send us the balance due immediately We know where you live."

And the machine is always right---it can do anything it wants to because you entered the game and gave it credence. Are you going to argue with a gun to your head?

I used work these markets but they were almost real then-----now? good luck!

darkstar7646's picture

Why any sane person still "trades" in a "market" where the average share of stock is "held" for only eight seconds is beyond any sense of my comprehension!

AmCockerSpaniel's picture

So right you are. I'm of the opinion that buy and hold is the only sane thing. And holding paper is not an option. I hold PM, and read the mail here to get insight as to when to add to my holdings.

PulauHantu29's picture


In heaven you'll get "fairness" ...on earth you get Goldman Sachs.

billsykes's picture

So lets look at this info, they are frontrunning stocks able to manipulate buy/sell orders, are placing millions of trades a millisecond and account for the majority of action on most major exchanges.

My question is, if these HFT firms have so much advantage where is the profits from these firms, why aren't new billionaires being minted daily? Why are the owners not front and center of forbes or buying up entire cities made of gold?

You would think that these shadowy firms need to start out somewhere, raise money somewhere, be on the radar somewhere?

Where would info be to find out what type of return these guys are making?


Why doesn't someone go retro and have a pit traded exchange again?

JW n FL's picture
by billsykes
on Sat, 07/16/2011 - 14:50


My question is, if these HFT firms have so much advantage where is the profits from these firms, why aren't new billionaires being minted daily?



You are dumb fucker arent you? you dig ditches and live under a rock? is this your first day here at Zero Hedge? are you retarded?


come out from under your rock and read a lil you fucking short bus hermit!

billsykes's picture

SO cocksucker you are saying only the big banks are doing it?

why don't you school me and tell me that besides jp & gs who else is involved?

pesamystik's picture

Billy, you have to understand that there are many HFT operations and a lot of people involved in them. HFT is probably not producing billionaires, but it is producing plenty of millionaires. Remember, these algos make small profits that accumulate, but it takes away for serious money to add up. You can't lose but it takes time to win.