Chapter Ten The Best Explanation

In July 1984, a thirty-two-year-old Australian doctor had his own healthy stomach scoped, waited three weeks, and then, without telling his wife, drank a beaker of bacteria cultured from a sick patient’s gut.

This is a cross-examination, and Barry Marshall is our first witness, because the form of reasoning on trial is the one he staked his stomach on. The medical orthodoxy of his day held that ulcers came from stress and spice; a multibillion-dollar industry sold the acid-suppressing drugs; and the idea that bacteria could live in the stomach’s acid bath was considered somewhere between wrong and silly. Then the pathologist Robin Warren noticed spiral bacteria in gastric biopsies, and he and Marshall, his young trainee, biopsied a hundred patients and found the organism in every single one with a duodenal ulcer. Every one.

Notice the kind of reasoning that follows. Marshall did not deduce; no logic delivers “the bacterium causes ulcers” from any list of premises he possessed. He did not merely generalise; the pattern was already total. He asked a different question, the question at issue: what would best explain a hundred out of a hundred? And when the establishment refused the answer, when his contract lapsed and the animal models failed, he made himself the experiment: baseline endoscopy clean, then the beaker, and within days the bloating and the foul breath, colleagues quietly relocating down the hall, and on day eight an endoscope showing his stomach in open revolt, the bacterium culturable from his own tissue. Gastritis, note, and not yet an ulcer; the demonstration proved colonisation and injury, and the full causal case took years more of trials, which is the discipline at issue. Antibiotics on day fourteen, a confession to his wife, and, twenty-one years later, a Nobel Prize shared with Warren.

The form of reasoning Marshall used has a name, and the man who named it deserves his hundred words. Philosophers call it abduction, or inference to the best explanation, and it was first isolated by Charles Sanders Peirce, one of the profoundest thinkers America has produced and certainly its most tragic: a prodigy who read Kant at thirteen, ruined by scandal and academic vendetta, who spent his last decades in a freezing Pennsylvania farmhouse writing on the backs of old manuscripts and surviving on bread the local baker donated, and who, in 1903, destitute, delivered at Harvard the lectures in which the third form of inference got its definitive formula, the one at the head of this chapter. Aristotle had given us deduction, rule and case to result. Science had given us induction, cases and results to rule. Peirce saw the neglected third movement, result and rule to case: given what I see, what would make it a matter of course? He called it the only logical operation that introduces any new idea, and you have performed it today. The crash from the kitchen and the guilty cat. The symptoms and the diagnosis. The engine’s rattle and the mechanic’s verdict. Reasoning backwards from what is to what would explain it is most of what a mind does before lunch.

And it is treacherous, which is why we are in a courtroom, because the same leap that found Neptune has hanged the innocent. The glory first: for decades Uranus wandered off its computed orbit, and two mathematicians, Le Verrier in Paris and Adams in Cambridge, independently asked what would best explain the misbehavior and answered: an unseen planet, there. Le Verrier posted his coordinates to Berlin, and Johann Galle found Neptune the same night, within a degree of the prediction, in under half an hour, and wrote back to say that the planet Le Verrier had computed was really there. A star discovered with a pen.

But now the danger, and it sits in one small word. Best. Best by what measure? Juries convict on inference to the best explanation every day; the prosecution’s story explains the fingerprints, the timeline, the motive, and “no other explanation fits” carries the room. The trouble is that explanatory goodness, left undisciplined, is a feeling, and feelings can be manufactured: a vivid story explains everything because vividness feels like fit, a conspiracy explains everything because it was built to, and the explanation that forbids nothing, the trader learned the hard way, is the one that ruins you. There is even a courtroom at the root of the statistics of this problem: in 1867, in a celebrated American will-forgery case, Peirce and his father testified on the improbability of a disputed signature’s matching strokes, one of the first attempts to put explanatory inference under arithmetic in front of a jury, and the arguments about whether they did it rightly have never fully ended. The leap is powerful. The leap needs law.

Here is the law. It is not new; it is the derived architecture, recognised in different clothes. Ask what you are actually claiming when you call H the best explanation of evidence E. You are claiming, first, that if H were true, E would be a matter of course: Peirce’s own phrase, and it is the statement that H makes E probable, what the mathematics calls likelihood. You are claiming, second, that H is not itself a monster of contrivance, that it was plausible before this evidence arrived, and that is exactly its prior, where the last trial taught us simplicity already lives: fewer moving parts, thicker prior, by theorem rather than taste. Put the two together, how likely the hypothesis made the evidence, weighed by how believable the hypothesis was to begin with, and you have not invented a new logic of explanation. You have written down the update rule. Inference to the best explanation, done honestly, is the consistent inference of Part Two, described in the vocabulary of stories rather than the vocabulary of probability, and every legitimate explanatory virtue cashes into one of the two factors or it is contraband. Fit with the data is likelihood. Simplicity is prior. Elegance, where it means anything, is the absence of unnecessary structure, which is prior again. And loveliness, sweep, the click of narrative satisfaction, the feeling that it all makes sense: cash it or convict it. A story’s beauty that cannot be redeemed as fit-you-would-bet-on or plausibility-you-held-before is the smuggled cargo the whole search exists to find, and courtroom history is a ledger of convictions purchased with that currency.

One distinction rescues what is genuinely special in Peirce, and it is worth ending on. Abduction contains two acts that his own formula fuses. There is the generation of the hypothesis, the creative conjuring of “an unseen planet” or “a bacterium in the acid” out of the dark space of things nobody had thought, and about generation the mathematics is silent; no rule produces the candidate, and the leap is real, and it is art. And there is the evaluation of the hypothesis once conjured, and evaluation is not art at all; it is the forced weighing of likelihood against prior that any consistent mind must perform the same way. Generation is art. Evaluation is law. Marshall’s genius was in the conjuring and in the courage; his warrant was in the evaluation, the hundred-for-hundred likelihood no rival story could match, and the years of disciplined trials that followed the beaker. Holmes, for all the theatre, is doing nothing epistemically exotic: better priors from vaster knowledge, better likelihoods from finer observation, the same arithmetic as the child who solves the mystery of the missing biscuit. A machine can fail this test in a distinctive way, generating fluent explanations while skipping the evaluation entirely: loveliness without law. The Sufis say you do not add sight; you remove what obscures it. Explanation needed no new theory of itself. It needed the smudges named, and they are named: the mirror was always the same mirror.

Next witness. She is holding a lottery ticket, and behind her, in a British courtroom, a mother is being convicted by a number that was never true.