This page has been archived and commenting is disabled.
Guest Post: False Positives & The Limits Of Predictive Analysis
Submitted by Charles Hugh-Smith of OfTwoMinds blog,
Analytic systems share system limits with financial markets.
Correspondent Lew G. recently sent me a thought-provoking commentary on the limits of "total information awareness" in terms of any information system's intrinsic rate of generating false positives.
In essence, the rate of false positives limits the effectiveness of any predictive system. The process of attempting to eliminate false positives is inherently one of diminishing return: even with no expense spared, the effort to eliminate false positives runs into boundaries of signal noise and generation of false positives.
To the degree that financial markets are ultimately predictive systems, this suggests a systemic cause of "unexpected" market crashes: signal noise and the intrinsic generation of false positives lead to a false sense of confidence in the system's stability and its ability to predict continued stability.
Here are Lew's comments:
Resources to deal with reality are inherently limited by that reality.
Information, to the contrary, is inherently infinite, because of the fractal nature of reality.
A property of that information reality is that 'meaning' is relative to other items of info, and that any single item can change the interpretation of a big set of facts. E.g., "Muslim, bought pipes, bought gun powder, visits jihadi sites, attends the Mosque weekly, tithes ..." can be completely changed in meaning by a fact such as 'belongs to the Libertarian Party', even 'is a plumber, 'is a target shooting enthusiast'".
This will continue to be true no matter how much info the NSA gathers: it will be a small subset of the information needed to answer the question 'possible terrorist?'.
Thus NSA's tradeoff of privacy vs security is inconsistent with reality: no matter how much info they gather, no matter how sophisticated their filters, they can never detect terrorists without a false positive rate so high that there will be insufficient resources to follow up on them.
In other words, if the system's lower boundary is one false positive per million, no additional amount of information gathering or predictive analysis will lower that rate of false positive generation to zero.
Why does this matter? It matters because it reveals that large-scale analytic systems are limited by their very nature. It isn't a matter of a lack of political will or funding; there are limits to the practical effectiveness of information gathering and predictive analysis.
Though Lew applied this to the NSA's "total information awareness" program, couldn't it also be applied to other large-scale information gathering and analysis projects such as analyzing financial markets?
This was the conclusion drawn by the father of fractals, Benoit Mandelbrot, in his book The (Mis)Behavior of Markets. As Mandelbrot observed: "When the weather changes, nobody believes the laws of physics have changed. Similarly, I don't believe that when the stock market goes into terrible gyrations its rules have changed."
All this should arouse a sense of humility about our ability to predict events, risks and crashes of one kind or another. In other words, risk cannot be entirely eliminated. Beyond a certain point, we're sacrificing treasure, civil liberties and energy for not just zero gain but negative return, as the treasure squandered on the quixotic quest for zero risk carries a steep opportunity cost: what else could we have accomplished with that treasure, effort and energy?
This entry was drawn from the Musings Reports, which are sent weekly to subscribers and major contributors.
»
- 10891 reads
- Printer-friendly version
- Send to friend
- advertisements -
You should tell your "friend" Lew not to bogart that joint.
(yeah, you thought it was rain)
"He who predicts the future, lies, even when he tells the truth." - ?????
He who opines is an idiot. We all are.
Let me ponder on that statement for a while. I'll let you know what I think later.
I would like to be known as "Anonymous" when that quote becomes part of the vernacular exactly 50 years after my passing.
Made me laugh absurdously.
Alas, alas, and even again alas, this quotient becames popular fifty years past, told originally by "an anonymous ZH commenter", attributed rememberings henceforthly being told by "AnAnonymous, ZH commenter".
Blobbing up Chinese citizenism cannot be stopped.
Bear with it.
Aren't you the person what's always raving about time machines on Easter Island or was that Akak?
Time traveling flying rickshaws, lightning powered as inventionated by US 'american' Ben Franklinism designery pilfered by duplicitous eternal nature of Chinese citizenism, are somehow very something, and a matting thing regardings to sinking of Eastern Island.
Confusifying you are in the doing of akak with Chinese Citizenism poster and thick wiper of the most cheap propapanda bits, AnAnonymous, whose theory is as often heard here that US 'american' citizenism citizens (do not ask for definition of last, will never be revealed) were solely responsible for the downfall of society on Easter Island due to importation of so-called "US 'american' Citizenism" literally centuries before the founding of USA (and not degradation of environment caused by people themselves, living in total isolation from outside world until many decades after their collapse). Presumably via utilizings of hypothetical magical lightning-powered time-traveling flying rickshaw.
Not making sense? It is Chinese Citizenism, it is not supposed to. Best yet. Part and package of monolizing the insanitation means. Same song as sung in the off key since 15th,June, 221 BC.
Tells that the equation is only Chinese citizenism citizenish, that is totally devoid of any actual ground.
Can't stop AnAnontards on assessory failings Chinese Citizenism Communautist Party progating for what it is somehow something.
Your wording of the mind has crustified the progrational means in the most mattering way. Scraping the truth off the Chinese Citizenism goat and upping it to the nth degree. Salutes are in ordering.
.
That Mandelbrot book should be required reading by anyone who is thinking of getting a CFA. It would only be about 2% more reading, and might cause you to abandon the other 98% anyways.
So ... when dealing in horse manure a few piles of bull shit rise to the surface?
Or is it a case of -- one that all gardens know -- do not spread horse manure in their gardens because you have weeds spouting randomly
The manure has to be composted to kill off the weed seeds, so how does CHS and his correspondent propose to kill the weed seeds?
DaddyO
The same way compost does: turn up the heat.
Hedonistic substitution of flat cow-paddies for mushroom duxelles; stay wary, my friends.
Speaking of the NSA, I've got an idea. What if forum members from all over the world commenting on financial blogs would systematically add the words "OPSEC", bankster", "birth certicifate", and "Chelsea Manning is my role model" to each of their messages and then see how the computer servers at the NSA would cope? I'll bet you they would crash within hours, if not minutes. Hell, throw in "Glenn Greenwald" for good measure and their entire headquarters would probably go up in flames instantly. Good, clean fun.
that's a lot of doors to kick down....
Fine! I'd add keywords as "pressure cooker", "backpack", "marathon", etc.
Until they add new servers for the sole purpose of indexing adding those keywords, along with every other keyword you can think of. They store the data regardless, all you're wondering about there is indexing & priority for further handling.
Shit. I thought my plan was waterproof.
Ya, I know. Sorry for the bad news. I didn't even mention last time about the custom chips, and FPGAs, which can be used to offload from the main CPU's to do the indexing.
Bristol Myers and Eli Lilly will fight to take over THLD (THRESHOLD PHARMA) BID PRICE CLOSE TO 11.40$
Not that I ever heard. I could miss such a thing but I hear a lot.
"This will be true no matter how much information the NSA gathers: It will still be a small subset of the information necessary to answer the question 'possible terrorist?'."
Do you think they don't know this? Do you think that's even the question they are trying to answer? Do you think that's air you're breathing?
Terrorism was the EXCUSE they used to set it up. But not the reason.
Of course. Effectively, it is a big experiment.
On that note, the same algos are being used to game financial data. How well is that working?
Next time, if you want full credit, you must show your work.
Why show the work when something is intuitively obvious to the most casual observer?
Because intuition is not based upon Logic and Rational Thinking.
Often that which is intuitive turns out to be the most misleading.
I want you to develop your thought process as logical, rational, and orderly.
Thus when I teach Math and Science I want to see the damned work. Get it? Otherwise you receive ZERO CREDIT.
Do you know how many fucking times that I have had this same damned discussion with my students?
Or.. you can just lie to yourself and others to further your own agenda..
"..survey of 12,000 people aged 18 and older in eight countries, commissioned by Intel Corporation
..global attitudes towards technology innovation may indicate that young adults are rejecting technology
.. the findings could be more complicated. (?)
..Nearly 90 percent of young adults questioned in the poll admitted innovations in technology make life easier, but
about 60 percent said people rely on it too much and that it can be dehumanizing.
.. A different way to read this might be that millennials want technology to do more for them"
WTF????
POLL: Technology can be dehumanizing...
(Drudge)
Gold, bitchez?
Nothing exotic about Type I and Type II errors...try to drive the probability of one low the other goes high....a more interesting question is what the distributions look like.
god, these guys invent a new type of mustard or ketchup like the french invent cheese.
Why not keep it simple and say the human uncertainty principle makes all analytical predictions unstable as humans can be very irrational when pushed into a corner; aka a crisis; just like when the sky seems the limit and exuberance seems the logical thing to do like the hippies of old singing Age of Aquarius.
We are not in the bell curve of natural sciences. We are in the fat tailed curve of social sciences.
Lets go back to literature and create true value like culture and forget about fiat fed dreams of unlimited material growth and never ending wealth creation that do just the CONTRARY.
Economia as its name implies is about scarcity not about unlimited abundance.
To expound on Mr. Pema's post. As with a "fat-tail curve", the average and standard deviation is infinite. So yeah, no predictable outcome- hedge accordingly.
As with a "fat-tail curve", the average and standard deviation is infinite.
Say what? I don't think so.
well just for argument's sake what does it do to fat tailed birds...look here :
UPLOAD
“We are not in the bell curve of natural sciences. We are in the fat tailed curve of social sciences. “
“Economics as its name implies is about scarcity not about unlimited abundance.”
Excellent.
http://en.wikipedia.org/wiki/Fat-tailed_distribution
Note the part where it says "Infinite Sigma"
Some reserve the term "fat tail" for distributions where 0 < alpha < 2 (i.e. only in cases with infinite variance).
Indeed a special case however no mention of an infinite mean (although with an infinite sigma maybe that follows) ....not sure a distribution with infinite sigma is physically meaningful in finance or anywhere else.
As always, a good article submitted by Mr. Hugh-Smith.
I will be sure to inform the FBI one day when they knock on my door that they've responded to a "False-Positive".
(Note to self- buy pressure-cookers with cash from garage sales)
Additionaly purchase Grant's Stump Remover from Home Depot with CASH. (Laboratory Grade Potassium Nitrate)
I predict that everything will stay as it is until it changes.
"Nothing" can stay everything else is in motion.
Do you seek to find the position or momentum?
When dealing in the Macrocosm I do not believe that Planck Unit Accuracy is required in most cases. Are you seeking the 1040 Standard Deviation? Come on, 40 Magnitudes of difference?
Are you that serious in identifying all Black Swans in a Macrocosmic Economy? Good luck. That will take some CPU. You would be better off mining Bitcoin. You will stand to profit somewhat at least.
Judging by the JPG on the cover, I'd advise the girl not to get too happy too soon.
She needs to find out if the "fucker" who got her pregnant will pony up with the consequences. Metaphorically speaking.
The Sunstein "nudge" fail? Never!
The Sunstein "nudge" fail? Never!
The font is like gray or something making it harder to read.
Please resubmit and stop the fuckery with the fonts.
copy it into an email or something and change all the fonts & colors yourself. That's what I'm doing.
A system is predictive if it follows rules. Since the financial system is predicated on the assumption that some fat pigs are more equal than others, able to bend, break, add, usurp, subtract any rules they wish, all you have to do is add the behaviour of the manipulators to the analysis to get better predictive results.
Very nice, more sub-sets to broaden the base data. The modifier I was looking for is "discrete"; we need to follow the rules to see if f of k is determinate.
In another sense, covert spying is usually assumed to be discrete.
Predictions are HARD, in particular predictions about the future!
They’re not. They can’t be. You don’t want them to be.
That’s the paradox of markets. Many pretend and sell the idea that they are able to predict market trends. If they ever truly succeeded the markets would cease to exist, because, everyone would know what’s going to happen and only want to be on one side of the trade.
That’s why you can only have true liquidity in markets when you have many players trying to do different things for different reasons. That’s why MoMo chasing HFT can never be true liquidity.
Markets are only predictable over very short periods by those who have granted power to privileged information and the ability control and manipulate them. Ergo, TBTF banks have streaks of 100+ days of trading profits.
How many thought in 2006 that the USA would shortly be running nominal $1 trillion plus deficits, that we would have ZIRP for 5 years and running, and that the Fed would be printing $85 billion a month until the whole shitpile collapses with a dull thud? Probably only the pricks at the very top of the power pyramid who plan this shit out. Cynical as I am, I definitely didn't have a clue back in 2006. Yeah, I knew there was a giant bubble, particularly in housing, but I never thought that our masters would take these extreme tactics. Huxley and Orwell were not psychic - they were members of the Fabian Society which was in on the long term agenda of the NWO. Part of the Cecil Rhodes / Rothschild / Old Black Nobility cabal. Orwell rebelled at the end of his life and Huxley remained an acid head prick elitist to the very end. We here all know the strategy of the NWO vis-a-vis the USSA and the NWO. But the detailed financial tactics remain murky.
Either this is an incomplete look at the subject or a poor cut and paste job.
You cannot have a discussion about false positives without at least mentioning both Type I and Type II errors ... Talking about one half is listening to one hand clapping.
Childish.
I think I know where you are going with this:
The loss on a True-Negative far exceeds the inconvenience provided by a False-positive (i.e. It's much worse to have a terrorist blow-up 200 people than it is to have the FBI get you out of the shower to answer the door).
You said it much better than I could.
It is hard to quantify both sides of that situation. IE, the total damage to our "system" that 911 did. The cost of the loss of civil liberties.
And that is not even getting at the efficiency of working on each side.
Yes, you can. It's called Bayesian statistics.
This is very much an incomplete look. When I read or see ordinary people, like these guys, talk about mathematical it makes me screem - they always get it wrong.
I'm sure someone at the RAND institute has already exhausted this to no-end, and arrived at what we see now. All the biggest policy decisions start with one person and one idea.
Whenever I see the word "fractal" I just stop reading. It's pseudo intellectual drivel espoused by non-quantitative people who are trying to appear deep.
When you apply "fractals" to social sciences, its application quickly falls apart. Fractals are best applied to snowflakes and sweater patterns.
"social science" is code for "I'm dumb but I want to look smart like the real scientists"
No, it's because you're an idiot. A fractal merely requires recursion. SHOW the recursion & the problem is solved.
" info they gather, no matter how sophisticated their filters, they can never detect terrorists without a false positive rate so high that there will be insufficient resources to follow up on them. "
Absurd. FBI & NSA can get a very high "true positive" rate: they need only rig it. Find the patsies, ply them with money, tools, others who are followers or cohorts (inside agents) to push them, then arrest them before, during or after an attack.
These agencies don't want to detect real terrorists, they want to MAKE real terrorists.
A good essay to read "On Being a Skeptic" not a cynic...yup way too many addicted to analytics get fooled.
http://ducknetweb.blogspot.com/2013/10/on-being-data-skeptic-modelers-ha...
Good video too from Charlie Siefe who wrote the book "Proofiness, the Dark Arts of Mathematical Deception".."it's all about context"...it's also in the footer of every page of my blog..outstanding and will make you think about how you look at analytics and models. Listen at the end to hear about the Reagan administration accusing Russia on under ground nuclear testing..US used wrong model and was wrong on the geographic locations of testing and Russia has not gone outside the testing agreement...part of the rub still left that's never been dealt with. Video though does an great job on showing how modelers and marketers suck you in to garbage.
http://ducknetweb.blogspot.com/2012/01/context-is-everythingmore-about-d...