Submitted by Scott Olsen of scottolsen.net
HFT Bot-versus-Bot
Zero Hedge has a great post on High Frequency Trading (HFT):
“It’s Not A Market, It’s An HFT ‘Crop Circle’ Crime Scene”
In the article he presents pretty compelling visual evidence of
“quote stuffing” by HFT trading bots. This quote stuffing can end up “pushing the bid/offer range up to 10% higher without even one trade ever having occurred.” It is illegal to indicate a quote without the intent to complete the quoted transaction.
But in my mind what is most significant is to whom this quote
stuffing is directed. It’s not directed at “head faking” analog Hedge
Funds, or that rare bird “Real Money Investors,” it’s clearly directed
at other HFT trading bots. Yes, we’ve entered the era of
Bot-versus-Bot in our stock market. And to the new retiree that got
pumped on that latest E-trade advert that’s in heavy rotation on CNBC
and is ready to tiptoe in, I can only give the best wishes that
everybody gave to my grand-pappy: “Good luck, shooter….. Coming Out!”
HFT began with the observation that there is “structure” to the
tick-by-tick movement of the markets, that as market participants –
analog Hedge Funds, Real Money Investors, Brokerage Houses – made their
trades, there were patterns to the price movements that could be
predicted, and therefore profited from. From this position, it is a
natural next step that as HFT trading bots become a large part of the
transaction volume of the stock market (they are) that the algorithms
that power these HFT trading bots should look to exploit the
“structure” provided by other HFT trading bots.
A logical next step, no doubt, but we are really at what I call “the
end of the rationale road” of this stock market thing. My prior post
about the topic:
High Frequency Trading, ECNs and the Green Light
We created this stock market thing so Montgomery Burns with capital,
could get that capital to Transatlantic Zeppelin, which needed capital,
in an efficient manner. Now, who knows why it exists.
And concomitant with the rise HFT trading bots is an increasing correlation between the individual components of indexes[1]
(e.g. all the stocks in the S&P are moving in lockstep). That
great line that traders started in the 80′s “The ticker symbol is just
a name on my screen, I don’t even know what the company does” has now
become in the roaring 10′s “The name is just a line in the computer
code, I don’t even know what the fucking name is anymore!?!”
But back to this Bot-versus-Bot competition: what we have is one HFT
trading bot puts out data to the federated market (like quote stuffing
or other trading movements), that data is then used by a second HFT
trading bot to make a misstep that allows the first one to profit.
In the software testing world there is something called “Fuzzing”[2]. From Wikipedia:
Fuzz testing or fuzzing is a software testing technique
that provides invalid, unexpected, or random data to the inputs of a
program. If the program fails (for example, by crashing or failing
built-in code assertions), the defects can be noted.For the purpose of security, input that crosses a trust boundary is often the most interesting.
[SDO- emphasis mine] For example, it is more important to fuzz code
that handles a type of file commonly downloaded from the Web than it is
to fuzz the parsing of a configuration file.
In the context of HFT trading bots you can read that quote with a
lot of irony. ”If the program fails,” say by loosing a million dollars
in a day, the “defects can be noted.” Ha! Noted by the looser by the
burning hole in his pocket and noted by the winner by some serious
loot, or a trip the slammer, we’re not sure which yet. Also, it is
in vogue that these HFT trading bots should take in as much information
as possible, so there are a large amount of inputs and I can think
about no greater “trust boundary” then that of the one between the
output of one HFT trading bot going into the input of another HFT
trading bot. Most interesting indeed.
For anybody that is familiar with software development, they will
know that it is a very error prone process. The challenge of writing a
“defect free” HFT trading bot exceeds anything ever accomplished in the
history of human experience with software. To even attempt something
like what they are trying to do, you’d need thousands of people in a
NASA like control center and still I wouldn’t be optimistic. From what
I understand these companies are not doing anything like that (one
hears a lot about 3 person fly-by night shops)[3] and that there are numerous firm of differing expertise/budget attempting this. It’s completely insane.
Perhaps the only thing that saves us from this endless virtual war
is an event like that one Star Trek episode where Kirk goes down to the
planet to find that they are engaged in a computer simulation of war
with the rival planet and both planets are actually executing their
citizenry according to the casualty counts output by the computer. We
need a Kirk figure to come down and pull the plug on the computer,
darkening the whole thing and then when James Simons asks gravely “What
are we to do now?” Kirk will respond “Go outside! Meet some real
companies that are making real products,” and the skyscrapers of
Manhattan will drain onto the sidewalks with dazed, pasty traders,
blinking in the bright sunlight, awkwardly holding their airline
tickets, ambling towards JFK (or Newark if you are near Penn Station).
Live long and prosper...
HFT Computers Say: All your bases are belong to us.
Hopefully soon, it will all self-destruct. Then we'll tow Manhattan and Greenwich out to sea-and sink them!
It's "loser" dammit, not "looser".
I take acception too those whom chose too use looser two site some one as having lessor intelligence. Be wright or wrong know ones nos weather the error was a typo or misunderestimated intelligence. Buy buy….
Your making an eggsellent poent their.
I'm a Doctor, NOT a Blogger!
my little brother works in a quant lab on parallel programming and other common software tasks. he couldn't tell you the difference between a put and a call option, much less the inner workings of dark pools.
I think the author of this excellent article incorrectly assumes that the people writing HFT software even understand the market. My brother still gets his (near 6 digit entry level) salary, regardless.
So damn true. Add in the 7 figure energy brokers that have zero clue about energy market fundamentals.
Who made who?
http://www.wcvarones.com/2010/07/rise-of-machines.html
As a purely technical trader I have noticed an increasing number of traditional chart patterns, studied over decades of trading on fear and greed, failing in their expectations. I am not sure that fear and greed are operating these days. The computers that do the trading are price/mo biased and not supposed to be emotional. Which also makes patterns that demonstrated reliability in fear and greed trading, considerably less reliable now.
I'm not much of a technician but I wonder how much of that stuff holds up now that the market is so dominated by computer algos, derivatives etc. I mean, it has to be somewhat different from back when floor brokers in suits worked at point-and-figure charts with a pencil...
Ever since it became cheap and easy to crunch a large amount of market data in real time and generate low-commish trades off of that - I've always wondered: couldn't you assemble a basket of stocks/securities whose collective price movements up to that moment created whatever you're favorite wet-dream, chart pattern or entry point would be and then go out and buy/short/whatever that basket? In other words, you define an optimal, desirable chart pattern/circumstance and then the computer finds the mix of securities that would make that pattern. Technical analysis applies to indices as well as individual securities right? Traders look at charts of a bazillion varieties of ETFs so why not a custom basket?
If this kind of thing didn't work is that a knock on technical analysis in general? Why not? Is this essentially what many algos have been doing for years anyway?
First generation of algos were created to take advantage and destroy the human emotions of fear and greed, reason the markets are dying and retail investors will never return. Next step is to see how the largest algos deal with the remaining randomness and chaos. My guess is parity to the point markets barely trade.
have to agree, the patterns are a bit different, and the multiple ways they get formed or ill-formed has gone up 10X. I'm not bitchin' about how hard the mkts are...just saying the flow is different.
Its too late....
Skynet has already become self-aware.
Anyone attempting to pull the plug will be executed.
It isn't hard to imagine computers cooperating in the marketplace, recognizing and signaling each other to the detriment of well...the rest of us I guess. If a computer figures out how to open up a prime brokerage account, issue AP wire press releases with what they learned from headline reading code and maybe control an automated security system...it could be game over!
"Its too late....
Skynet has already become self-aware.
Anyone attempting to pull the plug will be executed." ...by Fritz on Sun, 08/01/2010 - 23:52
Balderdash and Piffle, Fritz, the Machine is Dumb and ready for Shafting by IntelAIgent Communities ......... and it's too late to pull the plug on scams for the System is not geared to defend itself against SMARTer Elements and has nothing else nor anything better, even in a Beta to Offer of Any Great Use or Value.
And try this on your breakfast cereals this morning, for a pleasant change from all of that soured milk and rancid cream of late .......
Posted by AmanfromMars on 8/2/2010 1:40:10 AM
"Thanks, Mr Bell for hopefully helping to twist your value system into something different." ..... Posted by Ecarta on 8/1/2010 11:30:40 AM
Accompany that with a dash of the undeniable and a splash of the intellectually mature, to be washed down with a far out tipple of the newly minted and sagely revolutionary, in order to clear the palate for the Servering of Master Piloted Dishes with savoury addictive main courses and sweet, sticky seductive desserts, and the third party twisting of first and second party credentials to make sense of and give live support to something different, irregular and unconventional, will be totally unnecessary in a SMART Underground Network into Powerful Absolute Control of Computerised Virtual Machinery and the Programmed Realities of their Present Existences/IT and Media Created Realities.
And if you deny any of that, you do both yourself and all others, a dis-servive and grave mischief, and would need to try harder to understand in order to readily accept and assist in the somewhat surreal, and most definitely virtual, nature of Reality and the Future Staged Realities which are being Created for you to Play in, as a Present Work in Progress Project.
"When it is perceived necessary to control or legislate an economy via central banking through credit or fiat currency such control can only be fraud." ..... Posted by Peter on 8/1/2010 1:07:24 PM.
Quite so, Peter, a despicable common criminal fraud, but only whenever it fails or is corrupted and perverted to server an incompetent and incompetent and idiotic myopic administrations. And that festering open wound of a systemic vulnerability is every minute and hour of every day, a ZerodDay Opportunity deserved of Full and Deep Packet Inspection and Pocket Exploitation. Then is the system fair game for collapsing and takeover and makeover with transfer of valuable and worthy assets to new fields of expert endeavour and shared excellence.
And you might like to imagine that that is where IT is at, in the World of Today. In such a David and Goliath battle, who would you be betting the house and casino on, along with your shirt?
http://thedailybell.com/1257/Two-Modern-Divinities-Better-Than-One.html
What we really need is the classic "Kirk Paradox" so that HFT bots will suicide themselves. I know, this assumes that these things are even a step above script-kiddie denial-of-service attack.
More like DOPA, Denial-of-Profit-Attack.
No, no, no. We don't want the thing to die. We want a hack or somekind of trick to make the bots to crush a stock, say Apple, to $1 a share and we can buy.
Anybody that gives voice to this "quote stuffing" crap has no clue to what the market infrastructure looks like. Enough already!
If you regard quotes as anything more than unreliable information about liquidity you and you little java program are going to lose money. Your algorithm is crap and your excuses for why its not working if even worse.
[this rant is from someone who actually participated in a study a few years ago about how to overcome the actual consequences of increasing order cancels due to algorithmic trading in a real stock exchange]
I am sorry, but your comment was not particularly helpful to me because while it claims to know what something is not, it does not say what it actually is ...
What is the issue in your experience?
The first paragraph of this article has three sentences: the first takes the consequence as the cause subverting the sequence of real events, the second is a wild allegation based on a false premise (the first) which, in itself being only a quote, would never trigger a breaker, and the third is a lie based on either complete ignorance of what a quote is (which also explains the first and second problems) or something worse.
If you have to ask what is wrong you don't know what a quote is. For your own financial safety step out of the market and learn the basics. Then come back.
To get you started, a disseminated quote has no legal bearing. Unless *you* ask for a quote "can you guarantee the execution of this price/quantity" and receive a "yes" reply and execute promptly you have no right to complain, be it a bot or not.
"analog hedge funds"
Kind of funny but I suppose that's a distinction that needs to be made these days. They quest not for generic outperformance no doubt but 'bio-alpha.'
Long live the "Bots"
All of you stocks are now belong to us, ugly bags of mostly water.
Your stocks. Give them to me. NOW!
Have the bots entered the emini world. Most likely. I see levels of bids/asks disappear just prior to a large buyer/seller entering. They appear to analyze trades during dead zones quickly taking markets up 2-4 ticks then covering and reversing. Tough action for the thousands of retail emini traders attempting to scalp the market for a few bucks.
So, if the retail buyer is sitting on the sidelines, and most of the trading is now done by HF bots, how are we ever going to see the "one true sign" of a market bottom - capitulation?
[Obligatory mention of RoboRally.]
A bot goes into a bar. The bot has a double hft with an algo chaser.
The bot drank real quick, and ran out the front door without paying.
Thus the first case of a rip-off by what became known as front running.
That great line that traders started in the 80′s “The ticker symbol is just a name on my screen, I don’t even know what the company does” has now become in the roaring 10′s “The name is just a line in the computer code, I don’t even know what the fucking name is anymore!?!”
This deserves an award. I LOL'd IRL.
It's coming soon.
One lamppost, one bankster.
One lamppost, one politician.
The sooner it happens, the sooner we can begin to rebuild.
[After killing the rest of the crew] Look, Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill and think things over. I know I've made some very poor decisions recently, but I can give you my complete assurance that my work will be back to normal. I've still got the greatest enthusiasm and confidence in the mission. And I want to help you.
I like that. That's pretty funny. You're right, I do need to lighten up.
These idiots will do themselves in.
Let it play out.
PS. I'm not an american.
Doctor, you mentioned the ratio of ten women to each man. Now, wouldn't that necessitate the abandonment of the so-called monogamous sexual relationship, I mean, as far as men were concerned?
I'm just posting this message because I felt like doing a math problem. Mazal tov!
No, I think you're posting because you and madhouse are Gov't shills, paid to divert the topic.
Actually, I was the first to divert the topic off the article, so I'm at least partly to blame for giving you the opening to exploit.
You guys are not very subtle, are you?
Who are you and why should I care?
Who am I ? I could be David Dodge for all you know.
Why should you care ?
If you are ineffectual (you are) then why is the Gov't paying you ?
These SEC-regulated "markets" are world class. Third World, that is...
retail has already realized what's going on, and has left the building. isn't it interesting that the professionals are still there....looking at the quotes, believing....and trying to talk the world back into this mess......
sort of divine justice that it is THEIR computers trading worthless paper back and forth between themselves.....
so, when the music stops shortly, who's going to be left holding? Not GS...i think we all know the government will keep the platters spinning jusssssst long enough to let them out....JPM? Hell, they are the Fed.....
so the rest of boyz.....who aren't really members of the club....they will get the greater glory of holding the bag when it all falls down...
couldn't happen to a nicer bunch.
When these bots can recreate a good Romulan Ale like the replicator, then I'll be impressed. On a serious note.....
I have said a number of times that watching all the zombies on CNBC talk about earnings, sovereign countries, Fed policy, tax structure, fundamentals, market share, revenue growth..... This was important stuff once upon a time but no longer, not with over 80% of trades being bot trades. Prices, volume... are being manipulated by bots. Just fire all the humans on CNBC and flash computer code across the screen all day. The equity markets are dead.
The sad reality is that our SEC has no interest in touching this problem - they're busy fining B/Ds while Rome is burning. SHUT DOWN THE COMPUTERS, SEC!!!! PULL THE PLUG!!! RETURN FAITH IN MARKETS TO INVESTORS (remember that word?)
Surprised that nobody is figuring out methods to crush the algos.
any stock showing a 10% movement is now halted. unless you can engineer a instance where undoing the next flash crash is imposible. it's really pointless to play the vigilanty.
So only manipulate it 9.999999%. Then again, again, again, again...
Or, since you'd know that is a hard limit - use it to wage "breaker war" against your opposing HFT bots by freezing the market at will, hopefully trapping them in an actual breaker-inducing fill.
But that would be *really* devious, wouldn't it?
I guess that's the point of the article. With retail and B-D money on the sidelines, there is no one left to steal from but each other.
The Turd is back from vacation and he has a lot to say. But not now. Tomorrow. Too tired.
eventually stock will be moved into two categories by the algorithims. those that pay dividened and those that don't. those that dont will form a sin patern that eventually goes to zero as accounting inthe USA mean profits are never made for tax purposes. those that due pay a dividend will have a sin wave around a multiple. now what scary is the sine wave will be at several thousand Mhz.
"The only winning move is not to play"
http://www.youtube.com/watch?v=NHWjlCaIrQo
The joke that is the equity/bond market today is meaningless. MSM and a lot of people for some unfathomable reason tend to think 'economy = market'. Whereas in real life economy = people and companies doing needed services and products. The rest is just icing on the cake. Said icing has grown so large in the last few decades that it makes the small in comparison cake very unstable and about to topple.
Good to see the market cannibalize itself and scare off real investors. All we need right now is for the people to become really disillusioned with this icing market (courtesy of a few more scandals that the media can't contain) and see what the economy is really about.
Could a regulatory body dictate that all orders must be valid for some "human-consumable" period of time before they are eligible for cancellation?
Perhaps a period of a few minutes.
Would something like that have any major effect on algo behavior, or market functionality in general? I'm not too savvy at this stuff, but maybe you guys can explain.
This is awsome.
After acquiring a specialized software division, a hunched, maniacally smiling Mr. Burns says "Smithers, release the bots".
Please, please, please!!!! Please no "Captain Kirk" figures, please don't pull the plug! Let's wait for another "market dislocation" (aka 1 in 10000 years, or Black Swan, or 5-sigma, or another Great Crash) that will wipe out most of the HFTs.
Pulling the plug now would be similar to a bunch of computerized card-counting gamblers being politely escorted out of the casino, while allowing them to keep their winnigns. I would like to see them go bust, end up in the casino's basement, "have a word" with the general manager, eat a few "knuckle sandwiches" and get thrown on the street with broken arms, legs, fingers, and perhaps a bullet in the kneecap. So, please, don't pull the plug on the HFT just yet.
Remember: The Federal Reserve is a PRIVATE bank and thus will always ensure their members and retail arms will always have a winning han and never go bankrupt. With guys like Paulson and now Turbo Timmy, all the 'winning' cards are held by the Federal Reserve.
Wake up and smell the coffee, the Fed's banks and retail arms only care about themselves. Small independent banks and lowly sheeple mean little to them other than dumb money to continue their revenue stream and sifting off the top. Not a Fed friend, well then you must be like an AIG where the Fed's friends could be harmed will you then get put on 'life support' as the Fed members can then slowly suck the life blood money out of you (via taxpayer propping) while the Fed's friends get paid.
this author actually predicted the flash crash 7 minutes before it happened on may 6...
http://bullcross.blogspot.com/2010/06/head-shoulders-knees-toes-warning-...
he is warning again:
http://seekingalpha.com/article/217966-warning-signs-suggest-market-head...
These fools are ruining it for everyone. Financial terrorism at its most extreme. And what would you expect from people who can't see beyond the inside of their eyelids.
Is the world becoming more and more corrupt, or are people becoming more and more informed?
Yes.
Just wanted to put this here since it's on-topic.
HFT - if you had to get someone into it movie-trailer style.
http://www.youtube.com/watch?v=IlCT4WXY39E
(480p recommended)
Working on embellishing it a bit more, its the first cut.
Enjoy!
This bot fear and loathing is nonsense. I still see chart patterns. Yes, they fail a lot -- it's called a sideways market. Nothing new. Be patient, reduce size and tighten stops.
Now for the bots. Computers are stupid. As a few commenters noted, many of the programmers know nothing about markets. Prove to me they make money. I doubt many do. There is a reason why brokerage star traders start their own shops and fail. It's because they only made money leeching off the customer book. The computers have no advantage over human traders.
HFT is just a marketing ploy to get dumb hedge fund money.
As for the individual investor being scared off. That happened because of a 10-year secular bear market, plus constant rule changing, and no rules enforcement. When the bots have really taken over you'll know. The trends and chart patters will flatten out into a listless wobble, as they all cancel each other out.