The Survivor Fallacy

Survivor Bias

Survivor bias is a particular kind of logical fallacy – if you only recount the experience of the survivor, you leave out the victim. It is part of a wider sort of fallacy that I call survivor fallacy – basing your theory of the world on only looking at evidence of survival.

In a certain sense, there is value in only looking at who survived because that represents a kind of Darwinism – those who survived, you would think, are the best role models. But this is fallacious reasoning. You also have to look at those who did not survive to see if the survivors are a lucky minority out of those who did something bad for survival. What you really need to look at is whether those who survived beat the odds – are they over-represented compared with similar individuals who didn’t survive?

Some numbers. If you do something that has a 1 in a thousand chance of survival and you are part of a subgroup of humanity who have a 1 in a hundred chance of survival, you are onto something.  Without this missing piece of analysis, you have nothing.

Here’s an example. I heard a news report of someone who was sitting in a car at a red light, unable to move, when he saw a huge truck hurtling towards him. He prayed mightily and somehow survived the complete and utter wreck of his car without a scratch. This, he claimed, was a miracle. The problem is, if you have a one in thousand chance of surviving such an event, you have to assume – if it is not just that you got lucky – that the other 999 unlucky sods did not pray just as fervently as you did. Of course, since they died, there is no way to ask them. But in a society with a high level of religious belief (up to 90% in tribal societies like the United States), you have to assume that a similarly high level of those who were crushed were at least as religious as the survivor.

So did prayer work? Not likely. If 90% of the population is religious and only one in a thousand survives this sort of crash, being religious doesn’t really help your odds. Unless you can show that more than 90% of the survivors were also religious.

Another car crash variant: the person who survived a horror smash because he didn’t put on his seat belt. The other 999 out of a thousand who die in this scenario don’t get to tell the tale.

Likewise people who drink like a fish and smoke like a fish (smoked fish is tasty) and live to be 90 are not typical – and you tend not to run into people who did likewise and died before they reached 50 because they didn’t live long enough to meet a lot of new people.

Anthropic principle

Another variant on the same kind of bias is the anthropic principle. Let’s say for argument’s sake (since we don’t actually know) that it is extremely, wildly improbable that intelligent life develops on a planet. After all it took some 4-billion years on this one, and we have not discovered any incontrovertible evidence of alien intelligent life. So does that mean life on this planet had to be have been created by a deity? No. Even if it is stupendously unlikely for the conditions for not only life but intelligence to develop, if it happens in one place, the creatures so developed (us, for example) would be around to ask this sort of question. That is the essence of the anthropic principle: if the universe was not set up to support life (for whatever reason – even for no reason) and it did not develop as it did here, we would not be here to wonder how it all happened.

You can’t make this stuff up – or can you?

Here’s a fun story that illustrates survivor bias.

Imagine a pre-industrial civilization that develops the false myth that pigs have an instinct to swim towards land, so all ships carry a pig. If the ship is slowly sinking or supplies are running short, the pig is tossed overboard, and the ship sets sail whichever direction the pig chooses to swim. Every now and then a ship is lost at sea and is never seen or heard from again. Every now and then a ship arrives safe and sound after the pig toss. After much merriment and celebration (the pig’s role is unstated at this point), the pig myth is considered confirmed.

What of the ships that are lost? Who knows … no one can report back if they tossed a pig overboard and everyone died.

Curiously, I made up this story a while back to illustrate the survivor fallacy and don’t recall telling it widely, but if you search on “pig swims towards land myth” you find some people quoting just such a myth. You just can’t make this stuff up. Or, rather: I did but it’s hard to be original where superstition is concerned.