The Survivor Fallacy

Survivor Bias

Survivor bias is a particular kind of logical fallacy – if you only recount the experience of the survivor, you leave out the victim. It is part of a wider sort of fallacy that I call survivor fallacy – basing your theory of the world on only looking at evidence of survival.

In a certain sense, there is value in only looking at who survived because that represents a kind of Darwinism – those who survived, you would think, are the best role models. But this is fallacious reasoning. You also have to look at those who did not survive to see if the survivors are a lucky minority out of those who did something bad for survival. What you really need to look at is whether those who survived beat the odds – are they over-represented compared with similar individuals who didn’t survive?

Some numbers. If you do something that has a 1 in a thousand chance of survival and you are part of a subgroup of humanity who have a 1 in a hundred chance of survival, you are onto something.  Without this missing piece of analysis, you have nothing.

Here’s an example. I heard a news report of someone who was sitting in a car at a red light, unable to move, when he saw a huge truck hurtling towards him. He prayed mightily and somehow survived the complete and utter wreck of his car without a scratch. This, he claimed, was a miracle. The problem is, if you have a one in thousand chance of surviving such an event, you have to assume – if it is not just that you got lucky – that the other 999 unlucky sods did not pray just as fervently as you did. Of course, since they died, there is no way to ask them. But in a society with a high level of religious belief (up to 90% in tribal societies like the United States), you have to assume that a similarly high level of those who were crushed were at least as religious as the survivor.

So did prayer work? Not likely. If 90% of the population is religious and only one in a thousand survives this sort of crash, being religious doesn’t really help your odds. Unless you can show that more than 90% of the survivors were also religious.

Another car crash variant: the person who survived a horror smash because he didn’t put on his seat belt. The other 999 out of a thousand who die in this scenario don’t get to tell the tale.

Likewise people who drink like a fish and smoke like a fish (smoked fish is tasty) and live to be 90 are not typical – and you tend not to run into people who did likewise and died before they reached 50 because they didn’t live long enough to meet a lot of new people.

Anthropic principle

Another variant on the same kind of bias is the anthropic principle. Let’s say for argument’s sake (since we don’t actually know) that it is extremely, wildly improbable that intelligent life develops on a planet. After all it took some 4-billion years on this one, and we have not discovered any incontrovertible evidence of alien intelligent life. So does that mean life on this planet had to be have been created by a deity? No. Even if it is stupendously unlikely for the conditions for not only life but intelligence to develop, if it happens in one place, the creatures so developed (us, for example) would be around to ask this sort of question. That is the essence of the anthropic principle: if the universe was not set up to support life (for whatever reason – even for no reason) and it did not develop as it did here, we would not be here to wonder how it all happened.

You can’t make this stuff up – or can you?

Here’s a fun story that illustrates survivor bias.

Imagine a pre-industrial civilization that develops the false myth that pigs have an instinct to swim towards land, so all ships carry a pig. If the ship is slowly sinking or supplies are running short, the pig is tossed overboard, and the ship sets sail whichever direction the pig chooses to swim. Every now and then a ship is lost at sea and is never seen or heard from again. Every now and then a ship arrives safe and sound after the pig toss. After much merriment and celebration (the pig’s role is unstated at this point), the pig myth is considered confirmed.

What of the ships that are lost? Who knows … no one can report back if they tossed a pig overboard and everyone died.

Curiously, I made up this story a while back to illustrate the survivor fallacy and don’t recall telling it widely, but if you search on “pig swims towards land myth” you find some people quoting just such a myth. You just can’t make this stuff up. Or, rather: I did but it’s hard to be original where superstition is concerned.

The Missing Middle

I present some thoughts on why political opposition to the ANC is failing to make much of an impact on the ANC’s support base.

You need to take care not to lump all ANC supporters together.

The rural poor have had genuine advances: running water, RDP houses, fuller schools. That the water supply is unreliable, the RDP houses fall apart and most of the schools are little better than day-care centres reduces the value of these gains a lot, but they are gains. Add in social grants and food parcels, and the ANC makes a pretence at caring about this section of the community and no one did before, so they get away with it.

The group more likely to switch are the emerging middle class who are finding that they earn more than the NSFAS cut-off yet can’t afford university fees, so their prospects for improving their next generation are frustrated, and frustrated hope is a huge driver of change. This also is a group that is more likely to read forums like this, and to feel they are not at home with the people they find here. Would they vote EFF? Maybe, maybe not. Agang should have been attractive to this group, but failed for a bunch of reasons I won’t bore you with.

If you step back and look at the big picture, new political movements arise from new alignments of class interests. Old movements decay into patronage networks. Old movements’ survival game is thwarting any realignment of class interests. The ANC has that covered with workers because unions have been absorbed into the patronage system. This works because the unions have a deep hierarchy and a small number of leaders near the top can be bought off relatively cheaply (cabinet posts, provincial government etc. – Shilowa was a good example, while that lasted). The emerging middle class is harder to buy off because it does not have a hierarchical structure. Hence the government’s fearful response to #FeesMustFall.

Race is the government’s key weapon to stop a realignment of middle class interests into a powerful political movement. They failed to split the EToll protest movement this way, but they did manage to split #ZumaMustFall to a large extent because of racist responses by supporters of this campaign.

Any opposition movement to be successful must tap into this growing demographic and firmly condemn racism not only in its ranks but also in its wider support base.

So why not the DA? The DA has the same neoliberal economic agenda as the ANC and has a general arrogance about it that makes it a difficult choice for voters who have not already bought into it. Its history also makes it unattractive.

Neoliberalism is the agenda of the rich: in every country that has bought into it, the result has been increasing inequality.

Though the DP grew out of long-time anti-apartheid predecessors, its 1999 “Fight Back” campaign under Tony Leon was designed to attract pro-apartheid voters (“fight back” against what?). Then when the DA was formed by merging with the “New” National Party, the suspicion was deepened that this was a party of closet apartheid apologists. When the NNP split off and merged with the ANC, somehow this was not seen as negative for the ANC.

Because of this baggage, the DA has trouble cutting through. For this reason, I still see a case for a new movement. If anyone is interested let me know and I can start you off with what went wrong with Agang…

A Tale of two Window 10s

Rhodes University donated a very old PC to a local NGO and I set it up for the NGO. Though my preference is the free software route the practical reality is that I am not an NGO supporting NGOs so the easiest thing is to install what they are used to and that enables them to share documents with others, and that means Windows and MS Office.

The machine came with XP and that is a concern because it is not being actively maintained and there is a possibility that new software may not run on it. I was surprised at the modest hardware requirements of Windows 10, probably because Microsoft went the route of converging their desktop and mobile OS. While this effort has flopped in the mobile market, being able to run the latest OS on a machine that old is a plus. So how easy was it?

The hardest part was preparing the system and installation media.

First, I needed to put the installation media onto a USB flash stick, since the machine has no optical drive. A nice person in out tech support department helped with that.

Then, the Rhodes computing service set a password on BIOS to prevent changing the boot order, which I needed to do to start off the USB flash stick. Since it was heading for a weekend, I had to try to work out the BIOS thing. I found a web site that proposed popping out the CMOS battery and leaving it out for long enough to the capacitors to drain. That worked so well that the machine refused to start until the next day when I intimidated it by approaching it with a voltmeter – it started instantly but the BIOS password was still set. Another try: I found that it has an Intel logic board and there is a trick involving a jumper to put the BIOS into maintenance mode. I took out the password and then discovered that the BIOS would not let me set the boot order to include a removable device unless the removable device was actually attached at boot time.

Finally, I was able to start off the USB flash drive (after 2 restarts – perhaps it was slowly warming to the idea) and Windows 10 installed without any dramas. It took quite a long time, subtly disguised by appearing to finish fast, then needing a reboot. I was then able to install Office off another flash stick and do some minor configuration stuff, including persuading it I wanted to log in as a local user rather than via a Microsoft identity. This latter bit was not too hard for me to work out, but could have confused someone without tech knowledge.

On to machine number 2. A pensioner recently bought a notebook from a local dealer (upgraded by the dealer to Windows 10) and was having difficulty reading email attachments. The attachments are Excel documents, and I found him a free reader via Microsoft’s app store. When I launched it, it had some weird requirement for installing a license that made no sense to me, suggesting something had not been installed right in the base system. I gave up on that and found an older version of Microsoft’s free Excel reader, which installed just fine, but the system refused to find it when I tried to set it up as the default for opening Excel files.

When I installed everything from scratch, things were reasonably straightforward. Though I have extensive computing background, I am not a Windows user, so it is pleasing that it should be relatively easy to set up from scratch without too many bad surprises (the BIOS password thing would not be something the average home user would run into).

I wonder what the dealer did that made the other one more of an ugly beast to install software.

On the whole, though the first install I did went well enough, I am not tempted to shift from Mac OS. Though Apple has lost a lot of their edge in usability (not so much because Microsoft has improved but because they have stopped bothering with evidence-based usability and focus instead on making their systems look good), Apple still has one huge advantage for my purposes: UNIX-style development.

Muddle East

Some have responded to the recent Paris atrocities by saying, “So what? The People of the Middle East are suffering far worse at the hands of the West.”

You can feel bad for the people of Paris and still point out the hypocrisy of allowing Syria to be torn apart without a whimper of public protest, or 300 African schoolgirls disappearing with only moderate protest, or so many other appalling events around the world that do not excite Western news media.

Once we start down the road of justifying the unjustifiable because something even worse happened somewhere else where do we stop?

And when we start taking sides, we become blind to the fact that it is not just one side fueling the flames. Russia, the US (and allies), and local Islamic states all to varying degrees are fueling the conflict, providing weapons and money to back their very narrowly-defined very short-term interests. What is the role of Iran or Saudi Arabia in stoking up conflict? What is Turkey doing? Would the Syrian crisis have ended peacefully long ago if the Assad regime had not been guaranteed outside support?

The US and other Western countries blundering into this whole sorry mess is just one part of the equation. That on its own can’t explain all the instability. You think the Russians would have learnt from the horrible mess they got themselves into in Afghanistan, but no.

In the current Middle East conflict, it is very difficult to make sense of anything if you support any side as all sides have made terrible decisions.

Trace through the sequence of events in Egypt, as just one example:

• Arab Spring protests – huge crowds in the streets, soldiers who refused to take orders to mow them down
• a democratic election returning a Muslim Brotherhood government
• follow-on protests as the government failed to meet expectations
• a military coup

Some of this you can put down to Western interference. The Arab Spring protests were genuine as far as I can tell, as were the protests against the newly elected government. Where Western interference kicked in was the fact that the military coup was tolerated where such a takeover in most other parts of the world would be condemned. So it is a mistake to explain all problems in the Middle East through a lens of Western imperialism. It is a factor, but not the only one.

Looking more widely, putting everything down simply to Western malevolence does not explain the deep animosities between the different strands of Islam who are doing each other far more damage than they are doing to foreigners. Nor does it explain the roles of Russia and other regional powers like Iran and Turkey.

Beware the logic fail of “enemy of my enemy is my friend”. The biggest enemy in this conflict is lack of moral clarity arising from taking a side and sticking with it.