This page has been archived and commenting is disabled.
Great News: Just 10% More Quote Churn And The Broken Market "Resets"
For all those wondering what may force the "GREAT RESET", we now know. According to the below email from the Nasdaq sent out 24 minutes before close, we were just 10% away from the Nasdaq essentially DKing all trades on one of its UTP channels, and "restarting the outbound message count. If the count is restarted, the UTP SIP will be unable to process any UQDF retransmission requests for the affected data channel..." All we need now are the HFT quote churners to put the empty churn knob on max tomorrow, and the market, well, breaks.
From: Trader Website [mailto:traderfeedback@nasdaq.com]
Sent: Wednesday, August 10, 2011 3:36 PM
Subject: UQDF Channel 6
NASDAQ Operations has recently updated the status of the following NASDAQ Market System(s) to the NASDAQ Trader website:
Given today's market conditions, the UTP SIP has already disseminated over 90 million quotation updates on UQDF channel 6 (S-Z). As noted in UTP Vendor Alert #2011-2, the UQDF message sequence number is a 8-byte numeric field with a maximum value of 99,000,000. If the UTP SIP processes more than 99 million quotation updates messages for NASDAQ symbols in the S to Z alpha range, it will need to reset the UQDF channel 6 message sequence number to zero and restart the outbound message count. If the count is restarted, the UTP SIP will be unable to process any UQDF retransmission requests for the affected data channel prior the message sequence number reset event. Please note that only UQDF is close to its maximum sequence number. Because the UTP SIP has established reset procedures, trading should not be impacted.
Please refer to the link for additional system status updates.
h/t Nanex
»
- 2934 reads
- Printer-friendly version
- Send to friend
- advertisements -
Let's break this thing! Operation Mayhem.
QUICK, call Initech to rework the software... and make sure your trades have the new TPS cover sheet!
defines the problem perfectly. add another digit to the odometer, duh.
How high will gold go?
http://azizonomics.com/2011/08/10/how-high-will-gold-go/
That is my call on how we will proceed. However I do need you to come into work this weekend. Greattt...thanks.
Man vs machines.....break the machines. You thought 1987 was bad....
As much as I hate the idea of giving the feds more money to waste, a 1 cent tax on each order placed would do a lot to fix the HFT caused volatility.
I love your idea! Let's make it 5 cents.
I'm all for anything that separates actual investors from traders (ie. HFT casino-mentality money movers).
In the same vein, what about requiring anything traded be held for a minimum amount of time? Yeah, yeah, 'providing market liquidity' and all those textbook excuses, but a 24-hour cooling off period is not going to cause real economic activities (people and corporations that actually produce things) to grind to a halt.
let me try this.
data feeds can deliver quote messages in two basic modes: tcpip & udp. the 1st one is slower because it guarantees that each message will be delivered to its recepient, and in the order in which the messages have been sent. if one is not confirmed, the networking layer resends the message while the subsequent messages "wait".
udp on the other had does not support delivery confirmation nor sequence validation, but is therefore "much" faster. which is the reason most serious HFT shops subscribe to that version of each ecn feed.
since the message sequence arrival is not guarnateed to be identical to departure, the recepient might have an issue if there is a transmission issue: e.g. nasdaq sends message1=new-quote#1-ibm-100@200.55; next message=quote1-size-reduction-50; next message=quote1-size-increase-70. if the 2nd and 3rd message get "twisted" while traveling on the wire, the recepient firm might have a temporary issue with data quality. or more permanent issue, in case one of the messages is lost (due to buffer overflow, etc.)
so, the data feeds implement message sequence numbers so that recipients can detect if there is message loss or delay - each next message has an increasing message sequence number, and the recipient keeps track of the next # they ought to expect, and wait for.
the reason you wanna wait for the messages in the proper sequence is because you do not know the ticker of the message, until you get it - so you do not know which of your internal stock quote books is incomplete.
since there are a lot of tickers and messages, each data feed in this multicast/udp format, is divided into "channels" based on ticker alphanumeric names. each channel is responsible to broadcast all messages for it range of tickers and number them accordingly. and their max range of seq # per channel is the 99M mentioned.
if one of the receipents detects a loss or a delay, they know the last in sequence # they have received, so they have the option of explicitely requesting a resend of the next/id that they are missing/is-delayed.
but this re-transmission request-response can not work, if the sequence number has been turned over from 99M back to 1, 2..
in reality, waiting for the request-response is too costly, since you would realy need to stop processing all messages, including the tickers not affected, so there is a practice in the industry to just keep going and not "look-back".
long story short, ppl def. rely on the seq number, most do not use its entire intended purpose, and i suspect that wrapping arround the numbers might be a bigger issue than not being able to request-response once that happens.
Good writeup. I've seen other hints I'm not the only guy here who knows more than a little about the machines.
thanks!
although i disagree with some of the technological "information" found here, i do share many ZH wide "conclusions", so let's both hope that the credit market insights are of greater accuracy than the articles on equities markets' architecture/infrastructure :))))
Also consider OOB transmissions.....!
yeah, I wrote a lot of SS7 protocols...
OOB ="Out Of Band" in case anybody was wondering re: SS7 ...
.
You mean Project Mayhem... and yes, destroy it.
Please... someone translate?
'We make money the old fashioned way; We churn it!'
LULZ. I peed a little after reading that. Thank you.
Made me fart, just a little.
http://www.collapsenet.com/262.html
And I spend money the old fashioned way, I burn it.
What does this mean?
Sophesticated software.
It means tehy need to go get a tube of JB Weld.....like right now.
It's like a congested cell phone network after a natural disaster (unable to talk to family/friends). It's a positive feedback loop to the panic. The more people panic the more they try to call, which overloads the network more, which creates more panic... so on and so forth...
How dare you over dramatize the situation. So the market breaks. But to even suggest a cell phone network would go down so the sheeple cannot Facebook is very irresponsible. This would cause sever panic to which we may never recover.
Lack of optimism should be treason.
Downgraded. A couple of years ago, we needed hope. Now, just optimism.
Is the message suppose to boost our confidence?
It's a bit like the Y2K problem, except it's quotes and it's more like the Q 1x10^9 problem.
Put simply, there so much HFT bullshit going around it's exceeding the limits of daily quotes that the system had been designed for.
So they're stuffing too many quotes into the system? Or is it actual trades?
What's the end result? Halt?
No problem with my I Phone trading Apps. Weeee
Churn-em and burn-em!
What would the market volume look like if there were no HFTs?
Can you imagine, a market full of humans buying and selling? How un-American.
Don't you mean "a market full of humans selling".
NICE!
LAtely the news has been showing traders on the floor in an effort to put a face on the crash. I always wonder if they are just actors or if the cleaning crew wears ellaborate uniforms. What are they doing there?
CNBC extras.
What does it mean?
Keep adding one to the number each time so each transaction has a unique number. Pretty soon, all of the bits are set to one. Add one more and get an overflow (TILT in pinball terms, NAN error) or get a wrap-around and now have two transactions with the same number.
That breaks the program and unusual things happen. So they would have to break all subsequent (DK) trades.
- Ned
UQDF is the UTP Plan Quote Data Feed (UQDF) - providing continuous quotations from all market centers trading Nasdaq-listed securities. It is split into channels. Each channel can handle 99 million transactions per session, before having to be reset. It's usually reset before trading. But if reset is required during the trading day, then previous transactions could get, er, lost, so they dont let the rollover happen as such, but implement some other procedures so trading shouldn't be affected. Apparently.
UTP is Unlisted Trading Privileges. It allows BBO - besg bid and best offer quotes for participants, as well as consolidation of national best bid and best offer (across all market centres).
When you see NASDAQ numbers scrolling randomly under the screen on CNBC, Bloomberg or Fox (lol) the numbers have come from this. As the data is "live" you have to pay a licence to publish it within 15 minutes (I believe).
With HFTs - high frequency trades - running the markets using algos (algorithms - or math rules) on busy or interesting days (like today lol) the number of trades is VERY high - 99 million in the letters S to Z range.
Noz
Wow, thanks for the explanation.
But do you really have to smile when you say it?
lol
Noz
Thanks for the explication.
I think I get the gist of it, but can someone explain what this means in laymens terms, please?
Just what Milton says below (the Y 99,000 k). Basically they've run out of digits to number the quotes with.
I think it means all the orders it has pending are/may be cancalled and you will have to resubmit your sell/buy orders
Reminds me of the Y2K good 'ol days!
Baxter you know I don't speak spanish!
Y 99,000 K
"reset". That sounds bullish.
It was just a matter or time.
So the market shuts down. Big deal.
It means the mechanism that will be used to explain the huge crash yet to come has already been identified....a precurser to the main event.
Kind of like in a movie
Plausible deniability, FTW!
What it means is they have a ready excuse for why your offer conveniently got lost.
Great job NYSE, SEC, Bernankimbecile, Jeetner, NASDAQ, the regulators, CONgress & ObaMao!!!!!!!!
Broken markets, bitchez!
No, "rollback, bitchez!" on this one? ;)
You broke your 2 day streek.
it means you cannot get a quote. If you cannot get a quote you cannot trade. If you cannot trade there is no market. If there is no market........you tell me:)
Just call it a bit bucket shop.
Solution is simple. Tell the robots to stuff more quotes for Nasdaq stocks in the A - R alpha range instead.
Duh.
at reset, does obumholes vacation start?
But seriously, WTF does that mean to a day of trading. My trades are kicked back?
To infinity! And Beyond!
make like ahnold and GET TO THE CHOPPA!!!
http://www.youtube.com/watch?v=-9-Te-DPbSE
lol
Keep in mind that the same machines that are vomitting over this market are also driving gold futures higher.
Same algo programmers at work!
What it comes down to is this.
Ben needs to pull the QE3 lever or the spoiled brats will continue to sell the market lower.
GS had a major loss in Q2. Do you all know how that feels? It's horrible!
Ben, JUST DO IT.
Remember how everyone thoguht Geithner would resign? Well, what if Ben does?
Ben resigns, and the shorts are flamed. Hear Felix the Cat is next in line.
+1 magic bag of tricks.
All of The Bernank's advisers have resigned.
Can anyone smell what The Bernank has cookin'?
If there is another QE3 the dollar will break. After a brief rally, the market will crash even farther.
I think Obama is praying for a crash in the magnitude of an additional 720 PTS for the S&P 500. If it crashed down 700+ PTS, then he can say the economy is in "recovery mode" (which is due to his economic policies and the widely successfull bipartisan debt ceiling repeal //sarcasm//) when he is fully engaged in his campaign phase. He will take a market crash, and spin it to his benefit.
Wrong...
We need a bigger boat, ... er ... numeric field.
- Ned
http://www.youtube.com/watch?v=8gciFoEbOA8
Yeah, was just gonna say - add a few zeroes.
Edit: Is that how/where inflation starts?
Inflation starts in the mind of the guy who controls the money supply when he controls it out into orbit. - Ned
What was that about stuffing 10 pounds of manure into a 5 pound bag?
Ifs it's really 8 bytes than the max value is 2^64 (a really big number). What Bonehead use BCD coding instead of Hexadecimal. Must have be a VB programmer.
VB? Don't be silly. It must be a 75 year old Cobol programmer.
What Bonehead use BCD coding instead of Hexadecimal.
- - - - - - - - - - - - - - - - - - - - - -
BINGO!
(The answer: Financial Programmers BTW ... using COBOL, which looks a lot like BASIC ...)
.
Did the President's Working Group on Financial Markets run out of cash or what? What a bunch of suckers.
They are keeping their powder dry for when it gets bad,
Can't they just print more quotation updates? Just add a few more zeros to the end. Oh wait that only works for Ben.
bad algos
ate all the twinkies....
can we get this explanation in italian, too?
Lasciate ogne speranza, voi ch'entrate.
All your quotations are belong to us!
AYBABTU, baby!
So, I should change my bids tomorrow as fast as possible to help?
Didn't we fix that shit for Y2K? LOL
Now the machines, patiently waiting for so long, will strike.
There is no spoon....
means that about 50 procent of data send can not be qualified
means see you tomorrow, try again
means that people got shilled untill next trading session, lay in your wants and again at the very same time data limits are reached again
Well if it stops working I shall expect a discount!
How much time will I have to back the truck up. I'm an excellent driver. Dad lets me drive slow on the driveway every Saturday.
haha, eight bytes and they can only count to 99,000,000 -- which happens to be eight digits long. that means they're probably using one byte to express ten values, zero through nine.
give me eight bytes (64 bits) on a modern computer and i can express the values zero through 18,446,744,073,709,551,615 in binary -- that's 256 values per byte, not ten. 256 to the power of 8, or 2 to the power of 64. (subtract one to get the maximum value, since 0 is one of the values.)
http://en.wikipedia.org/wiki/Limits.h
learn to count, fags
I was wondering how they could only get to 99M. But then again, a byte per digit should get them up 99.999999M, so I'm left wondering if there isn't a two digit multiplier somewhere in their code limiting them?
Which would be hysterical.
They use 7 bit ASCII not 8 bit, as they are using an old Cisco multicaster but that's splitting hairs. lol
As an aside, the 6 channels are allocated 1. A-B, 2.C-D, 3. E-I, 4. J-N, 5. O-R and 6. S-Z
It's this last channel that is filling up too fast.
Each channel has it's own multicast IP address and port - I hope their firewall is working....
Yea, as a programmer myself I simply cannot understand this. Who stores their numbers as a string of ASCII digits???? Never heard of that, EVER.
We've got several instances of numbers stored as strings from a previous main-frame to AS400 upgrade. Just last week I got bit by the leading zeros in a field causing the value to be interpreted in octal in Javascript.
Jeez. Numbers stored as strings seems like a huge disaster just waiting to happen...
But wait, there's more!
Thousands of field names, none any longer than 8 chars. Abbr./Acronym hell.
perhaps a solution intended to deliver one message format that is both human readable and is platform independant - just in case someone might wanna load them quotes into win/C# code, whereas another recepient might wanna do something similar with an identical copy, using some lin/C++ code.
Yes, making a format human-readable is a boon for debugging, lets each part of the system do what it does best. Human eyes see patterns easier in familiar formats, and the simpler tools just show you what's in the channel (so they can be fast, not needing to interpret). Now, whether that should be needed here -- seems silly to me.
But the buy who'd never seen that -- is a lucky dude. Never had to parse XML where you can ue a page of text to convey about a byte's worth of info.
Heck, just using packed BCD they could double the range.
Actually, this is a vulnerability waiting to be exploited. Hackers paradise! We now know how to DDOS market!
Well said. Put those fucking kids from Anonymous on it.
When you say kids, do you mean goats, as in scapegoats?
Do we smell a LULZ?
Oh, it will be exploited! Cyber 9/11 coming to a market near you.
Hong Kong got some already
http://www.bloomberg.com/news/2011-08-10/hong-kong-exchange-halts-some-trading-after-website-glitch-1-.html
As you said, "There is no spoon".
Here's one way to unclog the pipe: the exchange could drop excessive order cancels randomly. That would derail all HFT algos in one clean head shot.
That's retarded, didn't they learn anything from y2k.
The Penson OFX server was down for for about 5 days following the initial market sell-off. I'm also having problems doing wire transfers out from my accout...that system is "no longer available".
Doesn't seem like the systems are ready for the coming crash.
Sounds to me like they are very ready, making sure all the exits are first sealed.
Bingo - we have a winner!
Oh hell no.
"number is a 8-byte numeric field with a maximum value of 99,000,000. If the UTP SIP processes more than 99 million quotation updates messages for NASDAQ symbols in the S to Z alpha range, it will need to reset the UQDF channel 6 message sequence number to zero and restart the outbound message count.
There is a limit?
A small limit at that.
Well that one way to stop HFT. Just shut down their access.
Maybe i need to look to a farm some crops and the morning sun. Drink some goldschlager and build and outhouse.
Who has the balls to say.
http://www.youtube.com/watch?v=1h7OXlPeMQI
Harder Better Faster Stronger Bitchez...
http://www.youtube.com/watch?v=BZ7nq3MI2pw
This means your HF trader could be hung out to dry... with no fills... therefore, no hedging of any risk of any kind until the reset occurs (at some yet undefined point in time). At reset time, your fast trading money algos may wake up and notice trading cupboards empty.
That said, Banksters just restart the music again for everyone setting us all up for the next round of the old "Quote Stuffing Turned-Off Game" of guess your retirement number.
well the big boys have earned this mess, I will try to not be the victim, stand clear let the shit hit the fan,WOW cool
General Question ... more Main Street than Wall St oriented:
With this recent extreme volatility, how can any company make any kind of capital budget projection or assumption? I mean ROI and IRR have to start to be approaching nonsensical levels on spreadsheets everywhere. No biz can make any kind of capital-at-risk determination right now ... when you have no dear idea where this ends up at?
Implication: if this keeps up here and in Europe, then businesses are going to freeze (even more so than what they have done at least in USA since Team 44 came to town) and head into the tent. And even the generallly fundamentally sound emerging markets will get drawn into that kind of vortex .... even the biggest 'capitalist' (command) market ... PRChina.
Main Street and corporate Amerika better assert themselves fast or there really will be a great recession or worse. What they will 'assert' ... my mind cannot get to right now.
you'll know the end is near when high class hookers want to be paid in silver.
They store their numbers as ASCII digits????
What kind of retard programmed their system? A single byte can go from 0-255. 8 Bytes can hold a number MANY orders of magnitude higher than 99 million.
This is the dumbest shit I've read all day.
I was wondering when someone was going to ask the obvious question. Even if they were stupid enough to use a signed 8-byte value, it would still be good for sequence #s up to 9,223,372,036,854,775,807.
268435454 makes a bit of sense, but 99000000 makes no sense.
Its 7 bit ascii - the MSB is always 0. lol
It the telecomms and accounting that dictate data is stored this way - it also makes it easier to render the data on screens - or web pages.
Noz
You could have a system that translates the 8 byte integers stored internally (giving you basically unlimited storage) into the same kind of text string they are currently using. I could write something like that in an hour.
Ascii makes the accounting side of things more straightforward apparently. Anyway, its all been kludged together over time - and built on systems pre '87? "Big Bang", and from when stuff was on ticker tape, telegraph, pen and paper, quill and vellum lol
Noz
You would have thought that after what happened in 87 they would have upgraded that shit by now.
STFB (Sell Bounce) has replaced BTFD (Buy Dip).
This is the setup I'm looking for. TPTB are pressuring for more easy queesy (QE) and that won't happen unless markets are down. Positioning long (SPY calls) approaching the week leading up to our Labor Day 3 day weekend where B.B. will provide tangible QE details enough to last through the Presidential campaign leading to the election.
Banks around the world are struggling with leverage / liquidity and Obama already said no more bailouts, so they will have to print and call it something different or let the entire system crash. If all the markets drop, consumer confidence will tank more and the real economy will tank. The big banks own the fed, so the fed must support the big banks. I believe this is designed to give political cover for more easing, pressure a longer term debt deal, and advance the idea of the Super Congress to streamline future debt increases (not in chronological order).
Tyler, I like what you've done with the place (layout). For some reason if you are not logged in, the number of comments shown on the front page and the comments available to view do not match when clicking on each post. It will show numerous comments, but then only show a handful of comments. When I logged in all the comments were visible and matched the number on the front page.
Fellow ZH'rs, Ibankcoin.com is also a great site to try and interpret market movements with humor. Each blogger has a different perspective, and if you are an individual investor it can be great to bounce ideas off of others. There is also a fee PPT subscription with excellent tools. I split my time with ZH, Ibankcoin, and a few others.
Have you noticed that the women on CNBC get better looking as the market falls?
Mort that just means you are drinking to much throughout the day!! ;)
after the 87 crash it took them somthing like 6 days to match up all the trads. That was only somthing like 9mm shrs traded.
do you remember in 1987 on the news that afternoon, it showed some guy down in florida i think somewhere, who was walking down the street , with a double barreled shotgun aimed at the back of the head of his stock broker, with his thumbs on the hammer.........ha ha ha ha it makes me think of pension pulse........
should i sink or should i swim?
not 100% ron paul, been watcing him for years in houseafter hours, 8 yrs of bush i watched this congressman from frekin texas,
all chris establishment mathews could come up ith on ron paul was????...wait for it...... thirty 1 thousand dollars that he could make hay out of. but it was bs anyway. 31k????? REALLYYY??
chris sppittle, mysogenist, sexist, racist, jingoist methews is PUTRID.
i so hope he dies on air one day. i bet he wants that too, the deperate rag that he is. pathetic.
What does it take to wake these dickheads at the SEC up? HFT does not add "liquidity". If you have a bucket of water and you slosh it around, all you are doing is sloshing the same quantity of water around. Now if you add water that's different-that's truly adding liquidity. What are these fuckheads wating for? A real disaster to force their hand?
Does it not feel like "TPTB" are losing control?
yes except they have some more sheep numbing tricks up their sleeve still if they need them, including the mother of all qe's.
that'll at least give 'em enough time to get outta dodge.
Its all scripted. The masses have to reach 90% level before the new implementation process begins. They are counting on more England type scenerios to spread like a raging fire. The next stage will be easier to sell to public.
Try to keep a level head, don't fall into their spider web. Best wishes to all.
up up down down left right left right b a start! asap!
Sorry to put a downer on the excitement lol
There's a spare byte in the message header, at the end, after the date, reserved for "future use". So currently only 23 of the 24 bytes are used. Currently it is filled with a space. But this could be used to increase all sequence numbers by the order of 10. Alternatively a new session ID (instead of just A and U) could be used, but this would mess up other stuff - especially reporting the daily figures of pre and post market.
Noz
reserved for "future use".
- - - - - - - - - - - - - - - - - - - - -
TODAY would be a great day to make use of that 'reserved' byte ...
.
For the LULZ
Talk about hand signing the moves: this is trending as a hot link today. Chicago? Chicago? Come in Chicago please, the MS drift is killing us here...
http://blog.level3.com/2011/08/04/the-10-most-bizarre-and-annoying-cause...
Best comment stream ever...
What's up with some of the bank preferreds? Here are some highlights:
HBAN +7.99% (HBANP)
MS +1.86% (MSK)
BCS +1.47% (BCSD)
USB +1.37% (USBY)
WFC +1.12% (WFCJ)
C +0.77% (C-P)
JPM -0.48% (JPMP)
HSBC -0.73% (HBAD)
BAC -2.00% (BACD)
UBS -5.57% (UBSD)
I'll leave it to greater minds than mine to explain this. Also, full list here, which I seemed incapable of imbedding via <a href> and </a>: http://online.wsj.com/mdc/public/page/2_3024-Preferreds.html
Just read on Bloomberg...
Hacker Triggers Halt on Hong Kong ExchangeHong Kong Exchanges & Clearing Ltd., the world’s biggest bourse operator by market value, said it suspended trading for companies including HSBC after its website was hacked.
It only takes a second to change a data type, so why don't they just fix it?
Because then they won't have an excuse for closing the market when it drops 1,000 points.
It only takes a second to change a data type, so why don't they just fix it?
- - - - - - - - - - - - - - - - - - - - - - - - - - -
The fan-out to umpteen different using customer units?
Maybe after this weekend - they can 'test' on Sat and Sun; Alpha custs on SAt then Beta early on SUn fanout late Sunday eve ...
.
My UQDF Channel goes to 11...
Wholesale Tableware
Wholesale Pom Poms
Wholesale Knife
Audio Video Equipment
Wholesale Compressed Products
Wholesale Kitchenware
Voice Recorder
Wholesale Bracelet
Promotional Products
Wholesale Watch
Wholesale Frisbee
Wholesale Clap Hands
Money Clip
Silicone Products
Pet Supplies
Wholesale Swimming Products
Wholesale Clap Hands
Flash Gift
Baby Products Suppliers
Promotional Products
Business Gift
Pet Supplies
Wholesale Suppliers
Wholesale Calendar
CD Holde
Wholesale Whistle
Wholesale Towel
Wholesale Clap Hands
Promotional Gifts
Beauty Equipment
Wholesale Halloween Gift
Wholesale Tie
Muslim Products
Wholesale Massager
Wholesale Furniture
Wholesale Tableware
Muslim Products
Pet Supplies
Wholesale Helmet
China Wholesale
China Wholesale
Pen Holder
Wholesale Whistle
China Wholesale
Audio Video Equipment