This page has been archived and commenting is disabled.
It's Official: HFT Breaks Speed-of-Light Barrier, Sets Trading Speed World Record
Submitted by Nanex
HFT Breaks Speed-of-Light Barrier, Sets Trading Speed World Record
Adds a new unit of time measurement to the lexicon: fantaseconds.
On September 15, 2011, beginning at 12:48:54.600, there was a time warp in the trading of Yahoo! (YHOO) stock. HFT has reached speeds faster than time itself. Up to 190 milliseconds into the future, or 0.19 fantaseconds is the record so far. It all happened in just over one second of trading, the evidence buried under an avalanche of about 19,000 quotations and 3,000 individual trade executions. The facts of the matter are indisputable. Based on official exchange timestamps, there is unmistakable proof that YHOO trades were executed on quotes that didn't exist until 190 milliseconds later!
Millions of traders depend on the accuracy of exchange timestamps -- especially after bad timestamps were found to be a key factor in the disastrous market crash known as the flash crash of May 2010. We are confident the exchange timestamp problem has been completely addressed by now: the SEC would have made sure of it. It's not like adding accurate timestamps is rocket science, or even considered a difficult problem. Based on recent marketing materials, the exchanges are practically experts on measuring time. And with hundreds of millions in annual data feed subscriptions paid by the same subscribers expecting quotes with accurate timestamps, there is no shortage of funds to make it happen.
So we can be certain the exchange timestamps were accurate, which means that HFT has truly entered the era of the fantasecond.
But let us suppose for a moment that in reality, quotes became queued (delayed) and were timestamped after leaving this queue. After detailed analysis of the UQDF data feed (see chart below) that transmits this information to traders, we find that the traffic rate for both the total of all output lines and specifically multicast line #6 which carries YHOO, were both well below peak rates. So it doesn't appear there were any capacity problems which have always been an excellent indication of feed delay.
This raises a few thorny questions.
Does this mean there are far more delays than we previously thought? Is there a delay every time we see an explosion of quotes in one stock? Because that sort of thing happens. All the time.
Regulation NMS is pretty clear that direct exchange feeds are prohibited from having a speed advantage over the UQDF data feed. UQDF computes the NBBO after all. So how does one ensure trade-through price protection if the price being protected hasn't even occurred yet? The NBBO lies at the heart of Regulation NMS (Reg. NMS) and is the key concept that assures investors are getting the best price when buying or selling stocks.
Maybe it would be better to just fantasize about fantaseconds after all.
The first chart is a 250 ms interval chart of the NBBO in YHOO which is plotted as vertical lines and colored red if the NBBO was crossed during the interval, yellow if it was locked, and gray if it was normal. The implied quote rate is shown as a histogram at the bottom -- and scaled in quotes/second. We will focus on the time shown in the black circle.
Zoomed in detail of above chart in 2 millisecond intervals.
The next chart is the same as the one above, but includes trade executions which are plotted as dots or squares and are sized according to trade size. Each reporting exchange uses a unique color and shape. An exchange code table appears at the end of this page. What is unusual about this chart is that trades are reported ahead of quotes. The dots/squares should trail after the quotes. Up to the point in time that is labeled "A", and after the point labeled "B", trades and quotes were in sync. Between these two points is where the quote timestamps began falling behind.
By plotting quotes and trades from just one of the active exchanges, we can easily measure how far in time the trade messages came before the quotes and therefore estimate the minimum amount of time the quotes were delayed. Below is a 1 millisecond interval chart of YHOO showing only Nasdaq trades (black circles) and the Nasdaq bid-ask spread (gray vertical bars).
Exchange Code Legend:
| 1 | NQEX | Nasdaq Exchange |
| 4 | AMEX | American Stock Exchange |
| 7 | PACF | NYSE ARCA |
| 8 | CINC | National Stock Exchange |
| 9 | PHIL | Philidelphia Stock Exchange |
| 11 | BOST | Boston Stock/Options Exchange |
| 57 | NQNX | NSX Trade Reporting Facility |
| 59 | NTRF | NYSE Trade Reporting Facility |
| 60 | BATS | BATS Trading |
| 63 | BATY | BATS Y Exchange |
| 64 | EDGE | Direct Edge A |
| 65 | EDGX | Direct Edge X |
The chart below shows quote message rates for UQDF and multicast line #6 which is the line that carries YHOO quotes. The time of the event is shown at the lower left. Although traffic from YHOO was a significant percentage of all traffic on UQDF, it was still not high enough to indicate any problems. Note that the surge on the right side of the chart is much higher, and there weren't any known problems at that time in YHOO.
»
- 29205 reads
- Printer-friendly version
- Send to friend
- advertisements -
My High frequency trading is faster than your high frequency trading blah blah fucking blah.
Adds up to the square root of fuck all if the power goes out!
Power blackouts BITCHEZ...................EMP BITCHEZ.
"Oh whats that i see there off of the coast of L.A..............must be another bottle rocket!"
http://www.youtube.com/watch?v=_rOssVrhQmg&feature=related
Einstein says, fuck that, nothing can achieve any faster speed than the speed of light.
Yesterday I was broke tomorrow!
THIS TIME there must be a PERP WALK. And as soon as they find the computer programmer responsible, I'm sure there will be.
Great scott. This is heavy.
HFT has reached speeds faster than time itself ? Really?.....so has my Smiths Pocket watch......
like i said: these guest post are getting fucking worse all the time. If it was physically at all possible to be faster than the speed of light, ( which has not yet been achieved ) you would actually arrive before you left, you would become younger, you would actually travel back in time, back into the past and not into the future...but never let basic facts of scientific evidence interfere with your wishful thinking, fiction and megalomanic hopium fantasyland of HFT time-travel...
perhaps a financial transaction tax for each HFT trade would restore some sense of space and time ?
wr;)
Actually, IIRC, it has been officially reported with reproducible demonstrations that a certain type of lightwave will exit a certain type of prism before it enters the prism. However, such technology has not been incorporated into the vast majority of electron-pushing computing/networking mechanisms; e.g., ICs, motherboards, plug-in adapters, used by the markets. While fiber optics are used for long-distance communication (between boxes), it seems highly unlikely such technology, if it is even out of the laboratory yet, could be deployed within the market infrastucture without that coming to (dare I say it?) light. Of course, it is not impossible.
Likewise, deployment of technology based upon quantum effects such as entanglement are still very much in the laboratory and unlikely to be deployed in the market infrastructure, but there is nonetheless some possibility of that, too. The last time I looked into progress there, instantaneous transmission of quantum state between entangled matter has been demonstrated across 18 km of separation and the experiments have been published and peer-reviewed. But the supporting apparatus is hard to miss, so if it were introduced into certain data centers that provision the markets, I cannot imagine such deployment could stay completely confidential for a long time.
Lastly, there is the question of how much further highly classified R&D has progressed beyond what the unclassified R&D efforts have been discovering. Could classified technology be involved in this reported market anomaly? It, too, must be possible to some degree, but I would not think it likely.
dlc, curious, does the stuff you refer to have anything to do with spintronics?
http://www.physics.umd.edu/rgroups/spin/intro.html
I do believe what Spintronics is up to is unrelated to the two other breakthroughs I mentioned. Unfortunately, I have temporarily been unable to recover the specific URLs for them and it may take a while to be able to make the time needed to do so. However, I will post them when I finally suceed (unless someone else finds them and posts them first). I do not see that what Spintronics is attempting to do will reduce latency anything like the technology I was discussing, however. I need more time to pursue that question. Thanks for posting that link!
oh no worries, mate. thanks for the reply. quantum computing is mindblowing stuff for sure. even the little bit that i can grasp makes my perception of 'reality' stretch like taffy.
Entanglement maintained at 18 km:
http://en.wikipedia.org/wiki/Bell_test_experiments
http://www.physorg.com/news132830327.html
http://www.nature.com/news/2008/080813/full/news.2008.1038.html
Alas, I have yet to uncover the write-ups of the prism that emits a light pulse before the pulse enters the prism. In the meantime, here are some articles that provide a framework for understanding such experimentation is not rooted in a lack of understanding of physics.
Retrocausality
http://articles.sfgate.com/2007-01-21/opinion/17227872_1_quantum-mechanics-universe-particles
Time Reversal Experiments
http://www.iitk.ac.in/infocell/Archive/dirjuly3/science_light.html (state of the art July 2000)
http://en.wikipedia.org/wiki/Time_Reversal_Signal_Processing
http://www.hindawi.com/journals/ijmst/aip/425710.pdf
Kennt die weisse Rose den Begriff "Humor"? Genauer gefragt, hat sie überhaupt den Artikel weiter als die ersten paar Zeilen gelesen?
Nein Rynak, hab ich nicht, die ersten paar Zeilen waren mir schon genug.
Habe leider keine Zeit für dumme Witze. (HFT = Fraud)
i guess this makes "stealing from the future" figuratively incorrect.
it has now become literal.
The left hand sells too the right hand where the fastest algo is always late.
Our alien over-masters have tipped their hand. Time to move the plan up before the humans get organized and resist.
All this HFT, its just sad. Truly destructive innovation for capitalism.
Yes, the latest destructive innovation at the top of a list of truly destructive innovations as long as your arm.
What BS people, trading happens in the NOW, not in the past or in the future but now, now is all there is, the past is gone and the future has NOT happened regardless of how much crack you have smoked.
Gretings Professor Falken.
A strange game... The only winning move is not to play.
I say its time to give them the raspberry.
Ah yes, the old bilabial fricative.
I'm a pretty sharp cookie yet I understand about 10% of virtual Tyler's articles on this crucial subject. Can someone direct me to a reference where I can get more than the MSM HFT For Dummies treatment?
If 70+% of all trading is now HFT, wouldn't that call into question the applicability of all the "human psychology of buyers and sellers" principles behind many trading strategies?
humans write the algorithms.
Time isn't that fast.
Especially when you're getting butt fucked by China.
Could this be why gold bullion is trading like it hasn't left the ground yet?
There is an idea called 'vector clocks' in computer science that could be used to make this impossible. Even if all the times are set wrong. It's how large distributed systems resovle concurrent update conflicts.
http://en.wikipedia.org/wiki/Vector_clock
Events are marked with relative time, so it wouldn't have to actually map to a real date or time. It instead means 'this event happened at this point relative to these others.' Each time any single machine does something, anything that is considered to be an atomic event, it would tick up it's part of the vector clock. Two things can NEVER, at any precision, happen at the same vector time, so it's always possible to order them.
EDIT: regular clocks on the systems providing the readouts for the input to this graph may be out of synch, ie, you get your input from two systems whose clocks are not the same... internally it probably IS using some kind of vector clock, which is why this would probably amount to what can be called a 'display bug.' Depending on which source your actual algorithms are using, it could be a problem for you and your robot trading.
Ain't technology grand? Now we wait for the "real" story, like how the timestamps are, ah, somewhat less than certain, after all. Maybe the NTP and/or IBM Parallel SystemPlex clock synchronization theories/implementations are insufficiently robust? Well, it may become the "official" explanation, because it would never do to allow the appearance of a jiggered free market to remain unrefuted. Assuming, that is, that they don't just sweep this under the rug because nobody really cares any more if the masses start to wonder about that possibility. Say, isn't that rug starting to look a bit high in some places? Not a problem, be at peace.
Retail investors will have access to this technology soon...your broker's margin desk will call to bitch you out about Regulation T margin violations caused by trades you haven't placed yet. The ultimate step will be "forced buy ins" to close not-yet-existing short positions.
Unfortunately this pent up pre-emptive demand (a "soon short covering rally") is already priced into the market.
Seriously, the time stamp issue is addressed in this hacker talk from DEFCON 19 conference.
Programmers can mess with the microsecond timer, something like that, to fuck up the HFT program. Disgruntled employee?
http://vimeo.com/28846996here's that Admiral Grace Hopper interview w/ Letterman he was referring to:
http://www.youtube.com/watch?v=RZ0g5_NgRao
amazing woman
That video is now bookmarked. Thank you!
He glossed over the tachyon-transmission possibility very quickly, giving the impression he doesn't believe there are any feasibile faster-than-light approaches on the immediate horizon and thus they are unworthy of concern at this time. But he also pointed out these are the companies that can cost-justify useful life of gear measured in weeks and fund custom hard/software development continuously. As long as it reduces latency, it is always going to be cost-justified. So it stands to reason they will be the true bleeding-edge early adopters of serious technological advance, and maybe even enabled to utilize classified tech. They may indeed already have those faster-than-light implementations deployed. Have you looked inside any of these data centers? I sure haven't.
The exposure to sabotage is probably an order of magnitude or two more serious, however. It would therefore seem to me outlawing HFTs and diligently enforcing such a ban is the only hope for provisioning free markets again.
Somebody figured out quantum computing.... all bets are off.
Brantz figured the plan, Yang programmed it.
Take that - board bitzez!
After months of reading these Nanex reports the HFT story is now clear to me. Trading in the future! Why didn't I think of that!!??
This is all Marty McFly's fault!! If he hadn't lost that Sports Almanac to Biff then NONE of this would have happened!! SOMEONE HAS TO GO INTO THE FUTURE AND GET THAT DAMNED MAGAZINE BACK!!!
Don't confuse reg. NFS, which applies to the NBBO and retail transactions, but not internal commercial transactions (the house is playing with its own money). Brokers trade on behalf of their clients using NBBO, but they aren't required to use NBBO for their own internal trades, dark pools etc.
The 'bots construct a synthetic top of book using only a few high volume direct feeds rather than wait for the matching engine to pump out NBBO (constructed from all avail. feeds). The 'bots sacrifice a small amount of liquidity in order to arrive at top of book faster than NBBO can. If the client has a limit order, for example, the bot can front-run while selling to the client at NBBO (thus satisfying reg. NMS) and gaining a few pennies/share in the process.
This explains how a bot can know and act on NBBO before it arrives on NBBO.
There are also OS settings that will affect results for a getTime() call. Depending on the setting, the clock can return jittery results rather than linear results.
Nanex doesn't calculate the timestamps . They provide the timestamps sent through CQS/CTS/UQDF. This is what they have been harping about for so long, that the system is f@cked up and providing the timestamp when an order is dissimiated from the que (as opposed to when the order was placed) is so wrong they need to write blatantly sarcastic articles to even get your attention. Do you think they base all this off a getTime() function? please, read the body of work they have done.
HFT = Frontrunning any buy or sell orders placed by any Retail Clients...
Therefore defrauding Retail Clients of any chance of ever getting the better trade.
That's why HFT = Fraud sanctioned by the SEC and Regulators to defraud so-called "Investors".
There are no more Investors left, just Speculators and minute - traders and Hedge-fund managers that gamble with other peoples money to take a punt on what has become the biggest casino on the planet.
HFT just leverages this whole ponzi-scheme to the complete and total fraud financial markets have become.
wr;)
I am surprised that no one corrected the "fantasecond" name, although it is all a fantasy.
Could the writer is reaching for "femtoseconds"?
(A femtosecond is the SI unit of time equal to 10-15 of a second. That is one quadrillionth, or one millionth of one billionth of a second. For context, a femtosecond is to a second what a second is to about 31.7 million years.)
I don't think so. A femtosecond is still a positive unit of time, tho very small. The fantasecnod is a negative unit of time. Reverses causality and all that.
Exactly. If the velocity of time is variable, as predicted by Einstein and proven by clock behavior on space vehicles relative to clock behavior on the surface of the Earth, the "obvious" truth that time is a one-dimemsional, one-directional, and constant-rate phenomenon is less obvious, even if Einstein still maintained the speed of light is always constant (because light is energy, not matter). Therefore, it might be theoretically possible that the velocity of time can be reduced to zero (full-stop, electrons cease to orbit protons) or even reversed (the electrons start retracing their movements). That a unit of measure of negative time units has been defined so we don't have to say, for example, -30 nanoseconds would say a lot about the plausibility of the concept. However, Google seems to have encountered "fantasecond" only in this article and its comments.
I really need to make the time to find those URLs again. Of course, they may have become mysteriously unavailable since I was studying them and I'm not certain I actually downloaded the material. I hope that statement doesn't cause me and my records to become mysteriously unavailable. ;-)?
But even without that odd prism phenomenon, with instantaneous transmission of quantum state improved to the point of maintaining entanglement between matter on an Earth-resident computer and a computer resident on a spacecraft moving at, say, 50% of the speed of light, the spacecraft's computer could process the trades exceptionally faster than Earth-resident computers having comparable technology and instantly receive from and transmit to the Earth-resident trading infrastructure. That seems to be possible theoretically, at any rate, based upon the 18 km demo, but it remains to be seen if they can improve the technology to enable maintaining entanglement at such orders of magnitude. If so, watch for the frantic development of unmanned spacecraft capable of ever faster acceleration to ever faster velocity containing ever-faster quantum-entangled computers; i.e., the real space race. Apparently they can cost-justify it.
Actually, that spacecraft concept is backwards. The closer things get to the velocity of light, the SLOWER their time velocity appears from POVs moving further from the speed of light. But that's a mere technicality. The solution is to get the Earth moving closer to the speed of light than the spacecraft. Or, easier still, cause the spacecraft to move along further from the speed of light than the Earth is; i.e., the race is for ever-increasing deceleration to ever-slower velocities. :-)
I think you're looking at the title (and new (Sarc) terms) a little too literally. They are making a joke about how fucked up the system is.
That is the likely truth of the matter from what I understand re: the states-of-the-arts. You appear to think a probability exceeding zero regarding the technologies I mentioned actually being involved is inconceivable. I do not follow the reasoning for such a conclusion, however. The joke they made may be upon them, and you as well.
"But wait! There's MORE!" Just out from CERN today: Neutrinos (which are matter) appear to have been caught moving at superlight velocities, and the researchers justifiably want corroboration of their analysis of the raw data before they dare to publish the findings. http://www.chron.com/news/article/Roll-over-Einstein-Law-of-physics-chal... So, we may have yet another blow to the hubris of the scientific community who thinks it knows so much (and ignores the fact some of their "facts" get proven later on to actually be "fiction").
Actually, that spacecraft concept is backwards. The closer things get to the velocity of light, the SLOWER their time velocity appears from POVs moving further from the speed of light. But that's a mere technicality.
The obvious solution is to put all the HFT...hell...ALL traders (especially banksters) on the outbound ships...
Solves it for me.
hehe
But it is probably not a cost-justifiable solution. ;-)
Love the charts.