This page has been archived and commenting is disabled.
Guest Post: The Unknown Unknowns And Survivor Bias
Submitted by Charles Hugh-Smith of OfTwoMinds blog,
Survivorship bias helps us understand why success stories are not what actually helps us succeed.
Former Secretary of Defense Donald Rumsfeld is famous for uttering a koan-like description of the epistomological ambiguity of human experience:
There are known knowns; there are things we know that we know.There are known unknowns; that is to say, there are things that we now know we don't know.But there are also unknown unknowns – there are things we do not know we don’t know.
(Interestingly, it appears Rumsfeld did not pen the koan himself; correspondent J.S.S. noted that the original source may be Landmark Education of Seattle, Washington.)
I recently read two fascinating accounts of why we have such a difficult time knowing what we don't know: it's called survivor bias, and what that means is we only get information from the survivors, not those who perished and vanished from the records.
A mid-list author recently explained how listening to the handful of authors who make it big financially is completely misleading: Survivorship bias: why 90% of the advice about writing is BS right now.
Here's how this works: big-bucks Author Z says, here are the 10 steps you need to take to become as successful as me. The list always mentions perseverance, being nice to your readers, writing 1,000 words a day and so on.
The 99.9% of writers/authors who make less than $10,000 a year from their writing (and the 99% who make less than $1,000) take this list as a script or program that if followed, will yield great success. But the 99% follow the script and do the 10 things and discover they are still unknown and not making any money.
The point is that nobody asks the 99% who fail to make it big what they did, or try to analyze what they did that prevented their success.
The numbers of people who become successful in these sorts of high-competition careers is vanishingly small--hedge fund managers are an example, along with musical acts, artists and writers. A handful at the top make most of the money, a relative few make some money (let's say a middle-class income) and then 99% make near-zero.
Longtime correspondent B.C. recently sent me an analysis of how many mutual fund managers outperformed a "dumb" low-cost index fund. B.C. found that 66 managers out of 26,507 outperformed the SPX over 5 years. That mirrors the distribution of outsized success among hedge fund managers, a few of whom net $500 million each annually while the rest underperform a plain old index fund.
The conclusion is that luck is the ultimate factor in these signal-noise levels of success, and being in the right place at the right time with the right product/idea can't be replicated by following a script or list of tips.
This essay is longer and well worth a read: Survivorship Bias The Misconception: You should study the successful if you wish to become successful.
The basic idea here is that studying the aircraft that come back riddled with holes from bombing missions leads us astray when we try to analyze the damage: what would really help us is studying the planes that were shot down, but they are unavailable for study.
The same principle applies to restaurants: the few that become roaring successes are endlessly studied, but the causes of success and failure are actually buried in the stories of all those that failed and close their doors. But nobody collects that data, for a number of reasons, including the study of failure isn't sexy and won't sell magazines.
We know that we learn from mistakes and failures, yet the study of failure is never recorded or saved unless the company or individual "came back from the dead," for example, Apple after Steve Jobs returned to lead the company from the abyss in 1996.
Survivorship bias is a profound insight into how we confuse the known unknowns and the unknown unknowns.
»
- 17572 reads
- Printer-friendly version
- Send to friend
- advertisements -
ok i'll start aaaaaaaaaaaaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhhhhhhh! known or unknown scream?
The more I know I know more, the more I know I know less.
Half the battle is learning to ask the right questions.
There is also a fourth category called "unknown knowns", which are things you know so well that you aren't even consciously aware that you know it.
Like the fact that the banks and government are run by crooks.
or that you breathe
the non-survivors are black holes from which no communication emerges
all you can know about them is their mass, their angular momentum, and the area of their event horizon
whatever that means in analagous terms for a person
We try very hard to study bearing failures in Peru to check on the quality of our product vs. how the pieces are mounted and/or maintained.
its the things you learn after you know it all that counts!
Knowing is half the battle GI Joe.
I know, I love being an idiot.
Not the dancing with the hockey players type of idiot though.
I say, hey, I be gone today
But I be back around the way
Seems like everywhere I go
The more I see, the less I know
Who wants to live any longer anyhow.........
"The known is a drop. The unknown is an ocean. He who knows himself knows all."
Someone said: " I don't know nothing, I'm not even sure I don't know nothing"
Life is like a box of chocolates. Ya never know whatchurgonnaget.
Next.
Re: Restaurants
They can burn the steak, but if you've a reasonable chance of nailing a waitress... oh, never mind.
Who farts in church, sits in his own pew.
i do think people study failure! - especially the aeronautics industry - since almost day one before the black box - airplanes that fell from the sky were methodicallly taken apart and put back together to understand the failure whether it be hardware or pilot error
this approach of studying failure and not success is why the incidence of airline and private aircraft failure today is so low -
+ 1 True! You beat me to a great example. But, the author's point is still a very good one: studying failure is hard, but probably quite worthwhile. I will try thinking more about what can be learned from failure.
One company I helped start used an interesting program to predict failures of various advanced composite parts...naturally, one also compared actual failures to the predicted ones to refresh the program.
It's a common approach in engineering.
In social "sciences", not so much.
I've been involved in the technology startup world and it's pretty obvious that not only is survivorship bias huge in that, but also that entire businesses are setup to make money out of getting the young, naive, would-be-entrepreneurs to pay to hear those successful ones (the survivors) in person, with quite a strong dosage of personality cult.
Ok, it is who you know, not what you know that advances your career, that much I do know.
Man who fuck on ground get piece on earth.
The FAA studies the failures.
One of Zero Hedge's recurring themes is to remind, ad nauseum, that
"the definition of madness is to keep doing the same thing over and over again, hoping for a different result."
However, it is also the definition of greatness ...
it seems to me that what defines greatness is that utmost faith in one's self and one's destiny,
that one will keep on trying and trying when most others would falter, give up, quit ....
it all depends, though, the only difference between the great man and the mad man may be, i nfact,
that one finally got a lucky break but the other never did .....
greatness becomes madness when the fallout for the collective species becomes negative. Its a tradeoff in history between short term gain and long term decay; only the guy who crosses the Rubicon or rides to the frontiers of the world, in name of greatness, does not see it as such as his favorite drink is personal hubris; nectar of the Gods.
One historical phrase summarises this very well : It was not a crime, it was worse : it was a fault; aka if you kill all the enemy but then ride off in your genocidal frenzy over the cliff...taking your whole army with you, like Napoleon or Hitler in Russia!
You lose all.
Quitters never win and winners never quit. That is what I teach my children, if nothing else they learn from me, other than the Golden Rule, love thy neighbor and your enemy...do unto others and you would have them do unto you.
Christ Jesus died for your sin, what could be better than that?
A probable corallary is that, as much as you might think you are in control of your destiny, its an illusion, except for some very very few.
But noone wants to hear that.
El wrongo! We don't control all the variables but we are in control of most of them. Get up in the morning and MOVE, study while the frat boys and the sorostitutes are out puking their guts up on the street at 1:00am, be careful who you get involved in and marry, don't think with your nuts, check out your honey's family first and foremost and don't worry about wforking more than your eight hours...do the job and forget the hours. Always look at what unsuccessful people do as a life style and then DON'T DO IT!
I have found in my life that while it is good to have positive role models to perhaps emulate, it is equally, if not more important, to look at my friends who were "losers" so to speak and to not do what they did. Use both data sets.
Just stay away from things people did and failed.
I have no mouth and I must Scream
Your real issue is that you only test the Human on certain parts of your populations.
Top tip: if you test the unknowing, and they pass, they will revolt (once having understood your entire weaponry). So, you're either looking for the shock of the new, or being muppets. That's the lesson.
You inform everyone of the test, and let it be voluntary. That is all. You might throw power to the ones that pass, but you make sure you look after the rest.
My Blood Line is My Strength and I will Awaken Again. The sleeper awakens. I was just playing Nice (and boy, you 1.0's are bad at this Game, I thought you were faking it or something, this last three years was me just going "Naaaah, totally, they're running a >OMFG look at that NEW bubble<"). Man, no clothes.
I do believe, mein freund, that the time has come for change.
Dude,
Are you losing it?
In a insane world you must insane in order to retain your sanity.
Fiction and fact from WAMO's almanac.
that kind of brought me down. i think i need a hug ;-)
you picked me up with Inva Mula...amazing voice
did i just notice daniella lugassy lip-syncing, or is it just an A/V sync problem?
whoever is singing this great song of dvorak's rusalka, you still have to respect the classics...
change needs stasis
fear is the mind killer
Case in point: I lost 53K in a stock pick & never told anyone. Except you...
The NSA loves you: they were the counter-party.
"The basic idea here is that studying the aircraft that come back riddled with holes from bombing missions leads us astray when we try to analyze the damage: what would really help us is studying the planes that were shot down, but they are unavailable for study."
In WWII they studied the aircraft that came back and put armor plates in all the vulnerable places that had no holes figuring that the non-survivors had been hit there. You can learn a lot from survivors if you look in the right place with the right mindset.
One thing you will never see is the CASUALTY LISTS of the SF warriors in the spec ops community.
There is a reason for this.
"Survivor Bias"
I had always heard that term applied to indexes such as the DOW....companies that fail are dropped out while those making (survivors) it remain which ultimately leads to stock indexes that are really inaccurate when looked at for longer time periods.
This point is never discussed by the sell side....they are happy to show you 10, 20, or 30 year charts proving that the market always goes up over time. The losers are dropped from the indexes and replaced with new companies.
Survivorship bias is the logical error of concentrating on the people or things that "survived" some process and inadvertently overlooking those that didn't because of their lack of visibility. This can lead to false conclusions in several different ways. The survivors may literally be people, as in a medical study, or could be companies or research subjects or applicants for a job, or anything that must make it past some selection process to be considered further
wait didnt Bernank said the same shit?
This wasn't developed by Donald Rumsfeld, at least not when he was SecDef and I don't think he's ever taken credit for having coined it. It's been around for years. This concept was part of a Leadership 101 course I took in ROTC given by an artilleryman as part of any dynamic planning procedure. No doubt he learned it from somewhere too.
Xenophon and Anabase...best example of survivorship; the march of the 10000.
From victory as mercenary on battlefield to becoming the enemy of hereditary enemy, 'cos the king you fought for for monetary advantage, having won battle dies on battlefield; and crown of empire then goes to his son who lost the battle in the first place and now promises vengeance to those who invaded his lands as mercenaries in dead cause. (fate of battle an unknown unknown before battle).
The irony of history as the hunter mercenary becomes the hunted. Back to Greece you go!
A lesson there for the mercenary soldiers of today's private armies fighting for Corporate-Empire power in third world; notably MENA/Africa. (Many unknown unknowns there which are probably similar to known unknowns from past experience; aka blowback of unknown dimensions).
The world marches on to a framework of fight between people and Oligarchy, just like in the age of Renaissance.
As we face the challenges of Theocracy in Islam; just like we faced Theocracy in Christianity then. (known known).
Or confronting the redoutable banksta conquistadors today making hay in the interties of their devilish financial plays using the "free market" front to rape "Peru", land of Oil today as of gold yesterday. (known known).
Our kings of "divine rights of man" today are as two faced in name of this NWO meme as were the machiavellian monarchs of the previous age fighting for "nation's prerogatives". ---Our way of life is non negotiable; come what may! If you fight that rule of survival you are a traitor sir!--- (Dick Cheney). (typical known known shill mouthing hubris disguised as wisdom).
---And don't talk to me about golden rule and according others the same rights as yourself, as a man's inalienable rights ends there where another's inalienable rights begin.--- Whence Prism and cyber wars to defend ourselves come what may. (ditto, travesty of what history teaches us).
Swing low sweet chariot of time from west to east, from north to south, as Columbus's voyager legacy gets wiped out by civilization turned barbarian in its own denial of truth, in its embracement of global hubris.
Survivor bias : was it the Trojan Horse to trick the other guy to win a predatory war or was it the declaration of Independence to affirm one's own freedom; eternal question in history. (known known, the essence of "looking back" to understand "looking forward", known unknown and unknown unknown combined).
DoChen, Studying how bearings are "mounted and maintained", sounds more to me how US citizens are being managed by the gooberment. Janet Napalatono,,, promising us that the US government is not Orwellian. LOL,,, wow.
As to studying the past and succesful people, plans, businesses. There is nothing I have found that contributes more to my success than hard work and perserverance. Having said that we learn all the time if we are willing to admit that we don't know it all. I was talking with my wife about why some would choose to be homeless. End of discussion was this, the world wants to eat us. It takes work to make a stand. It takes work to be successful. It takes work to keep your head above water. It takes work to raise kids that turn out to be worth a damn to society. It takes work to say, " I was wrong and I don't know everything." One thing my dad told me when I was young and it seemed to stick was, "It is easier to learn from others mistakes but some things we only learn by making our own." We learn every day. When we decide we have nothing more to learn we die by degree. I will always want to look forward more than I look back and that I think is the key to staying motivated. Stay thirsty.
No.1 unknown unknown is what you didn't learn by showing up and pretended that more study gets it done. Rumsfeld is a corporate tool and will say anything to confuse the enemy, e.g. transparency.
Historians from England will say I am a liar, but history is written by those who have hanged heroes.
The problem with analyzing mutual funds is they are already index funds, 90% of the mutual funds are not trying to be the next Magellan, they just mirror the S&P.