Chapter Fourteen Popper’s Wager

Karl Popper was frightened of certainty. He had reason to be. Certainty had nearly killed him.

He was born in Vienna in 1902, into a house with ten thousand books. His father was a lawyer, his mother a pianist, both of Jewish descent and converted to Lutheranism to assimilate into a city that would not, in the end, let them. He found school an agony, hours of hopeless torture, and left at sixteen to sit in on university lectures he actually wanted. The war had just ended; Austria was in ruins; the family savings evaporated in the inflation. And in the spring of 1919, at seventeen, Popper became a Marxist, and for a few months a Communist, because the theory explained everything, which is the property he would spend the rest of his life learning to distrust.

Then, on the fifteenth of June, 1919, a Communist-organised demonstration in Vienna turned violent, police fired on the crowd, and eight of Popper’s young comrades were killed. He had believed the doctrine that told him such deaths were a necessary intensification of the class struggle, and standing in the aftermath he asked himself whether any such calculation could honestly be called science, and something in him turned over for good. That same year he volunteered at Alfred Adler’s clinics, and one day described to Adler a child’s case that struck him as not fitting the theory at all, and Adler explained it effortlessly in terms of his own doctrine, a child he had never seen. How can you be so sure, Popper asked. Because of my thousandfold experience, Adler said. And Popper heard himself reply: with this new case, I suppose, your experience is now thousand-and-one-fold.

Two shocks. One year. From them, the question that became his life. In the street, he had seen a theory claiming the authority of science while people bled for it. In the clinic, he had seen a theory so elastic it could swallow any fact whatever and call it confirmation. What, then, separates the real thing from the counterfeit? He worked construction, apprenticed as a cabinetmaker, taught school, married Josefine Henninger who would type every word he wrote for the rest of their lives, and in the evenings composed the book that would answer his question, driven by an urgency that was not only intellectual: he needed a publication to win a post in some country that would still be safe for a man of Jewish descent, and the clock on that safety was running. He and Hennie reached New Zealand in 1937, one jump ahead of the annexation. The people who did not jump are part of what Popper’s fear of certainty was about.

The answer he had found was falsifiability. A genuine scientific theory sticks its neck out; it forbids something; it makes predictions that could turn out wrong. Evolution is science: a rabbit fossil in Precambrian rock would wreck it. Astrology is not. Whatever happens, the astrologer explains it afterwards, and is never once at risk. And Popper went further, into a picture of science as an adventure: bold conjectures thrown as far as they will go, then subjected to the fiercest attempts at refutation the community can devise, the best scientist being the one who tries hardest to destroy his own idea. What survives brutal testing has earned something; what fails teaches something; either way the enterprise moves.

Nearly all of this, the framework vindicates outright, and it is worth saying so plainly before the one correction, because Popper got more right than almost anyone. Severe testing is not merely good manners; it is forced. A belief that has never been exposed to the possibility of refutation has no more claim on you than one pulled from the air, because credence is earned only where evidence could have gone the other way and did not. Fallibilism is likewise not a mood but a theorem: every empirical belief is doxa, provisional, revisable, and to claim otherwise is to claim the immune kind of knowledge where it cannot live. Falsifiability itself, stated exactly, is the demand that a hypothesis make different predictions from its negation, for a claim compatible with every observation tells you nothing and cannot be learned from; it is epistemically inert, a wheel turning nothing. And the romance of the bold conjecture has a precise mathematical shadow: a hypothesis that commits to something specific and surprising starts with a lower prior, because it risks more, and is rewarded with a sharper rise in credence when its risk comes good. The scientist who predicts an exact number rather than a comfortable range wagers more and, vindicated, wins more. So far the framework and Popper are one voice.

Here is the single place he went wrong, and it is instructive precisely because so much around it was right. Popper denied that passing a test could ever confirm a theory. We can falsify, he insisted, never verify; a theory that survives is corroborated but never made more probable, and he clung to this because confirmation looked to him like the first step onto the road of dogmatism and, worse, like a concession that induction was legitimate, which he, following Hume, believed it could never be. But the derived update rule is not optional. When a theory predicts an observation and the observation arrives, the theory’s probability rises only if the theory expected it more than its rivals did. The comparative condition is forced by the same consistency as everything else. Applause that every theory predicted moves nothing, and evidence a rival expected more strongly can cost a theory ground even when its own prediction came true. If a theory assigns X a higher likelihood than its rivals do, then X’s arrival raises its relative standing; prediction alone, shared by every rival, moves nothing.

That is confirmation. It is real. And it follows from the same mathematics whose other half, the downward move when prediction fails, Popper embraced without hesitation. His own “corroboration” is confirmation wearing a disguise chosen to avoid a word he feared: to call a theory well-corroborated is to say it has survived severe tests, and to prefer it, to fly in the aircraft it certifies, is to treat that record as evidence of reliability, which is induction under a quieter name. He was right about the method and wrong about the logic beneath it, and the two had come apart in his hands without his seeing it.

The framework keeps everything he saw and restores what he discarded, and the restored shape is simply the whole of the update rule instead of half. A hypothesis is proposed; it makes predictions; the predictions are tested; agreement raises its probability and disagreement lowers it, both moves governed by one mathematics, and the severe testing Popper prized proves to be valuable for exactly the reason the framework would predict, that a test which could easily have gone the other way delivers, when it goes yours, the largest gain of belief available. His methodology, seen clearly, is a strategy for maximising information, which is MU in working clothes. His error was to imagine that the refutation half could stand alone, and you cannot keep the valley while denying the mountain that makes it a valley; the shape needs both slopes, and so does consistency.

There is an old story in the Taoist tradition. A farmer’s horse ran off, and the neighbours said, what terrible luck, and the farmer said, perhaps. The horse returned leading wild horses, and the neighbours said, what wonderful luck, and the farmer said, perhaps. His son tried to ride one, and fell, and broke his leg, and the neighbours said, what terrible luck, and the farmer said, perhaps.

Then the army came conscripting young men for a war, and passed the son by for his broken leg, and the neighbours said, what wonderful luck, and the farmer said, perhaps. The farmer held every report as doxa. He updated when evidence arrived and refused to overcommit, reading no single turn as proof of a kind or a cruel universe, and he was neither paralyzed nor deceived, only calibrated. Popper knew a great part of this wisdom; he knew that certainty is the enemy of inquiry, that a theory gripped too tightly blinds you to what would refute it. But he drew one wrong conclusion from it, that the cure for overcommitment was to deny that evidence could ever support at all, when the farmer’s subtler medicine was to let evidence both support and undermine, and to hold the whole of it the way water holds a shape, completely, and ready to flow the instant the vessel changes.

The trials are over now. The ground has held under every one, sometimes narrowed, once scarred, never broken. One test remains, the strangest of all, where the verdict is not that the principle answers but that it declines to. A foundation that will say I do not know, and say it lawfully, with a proof that the silence is forced, is a rarer thing than a foundation that answers everything.