Bayesian Reasoning
In Lessons 2 and 3 we studied forms of inference that deal in certainties or probabilities without a systematic method for updating. But real-world reasoning almost always involves partial information that accumulates over time: a doctor receives test results one by one, an investigator gathers evidence piece by piece, a self-driving car integrates sensor readings at 60 frames per second. How should rational beliefs change as evidence arrives? The answer is Bayesian reasoning — a framework named for the 18th-century mathematician Thomas Bayes — and it is both mathematically precise and deeply counterintuitive.
Prior, Likelihood, and Posterior
Bayesian reasoning rests on three quantities. The prior probability P(H) is your degree of belief in a hypothesis H before seeing any new evidence. It encodes background knowledge: how common is this disease in the general population? How often do suspects with this profile commit this crime? The likelihood P(E|H) is the probability of observing the evidence E if the hypothesis H is true. This is the discriminating power of the evidence: how likely is a positive test result if the patient actually has the disease? The posterior probability P(H|E) is your updated degree of belief in H after observing E. It is what you actually want: given that I saw this evidence, how likely is the hypothesis? These quantities are related by Bayes' Theorem: P(H|E) = P(E|H) x P(H) / P(E) where P(E) is the total probability of observing E across all hypotheses. In practice, with two competing hypotheses H and not-H: P(E) = P(E|H) x P(H) + P(E|not-H) x P(not-H) The result is that your posterior is your prior, updated by the ratio of how likely the evidence is under H versus under not-H. Strong evidence — evidence far more likely under H than under not-H — shifts the posterior dramatically toward H.
The most common Bayesian mistake is ignoring the prior — treating a test result as if the base rate of the condition does not matter. A test that is 99% accurate for a disease that affects 1 in 10,000 people produces mostly false positives even with a positive result. The prior forces you to weigh how rare the hypothesis is before updating.
To make Bayes' Theorem concrete, work through the classic medical testing example. Situation: A rare disease affects 1% of the population. A diagnostic test correctly identifies the disease in 95% of people who have it (sensitivity = 95%). The test incorrectly flags 10% of healthy people as positive (false positive rate = 10%). A patient tests positive. What is the probability they actually have the disease? Step 1 — Prior: P(disease) = 0.01, P(no disease) = 0.99. Step 2 — Likelihoods: P(positive | disease) = 0.95. P(positive | no disease) = 0.10. Step 3 — Total probability of a positive result: P(positive) = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085. Step 4 — Posterior: P(disease | positive) = (0.95)(0.01) / 0.1085 = 0.0095 / 0.1085 ≈ 0.088 = 8.8%. Despite a 95% sensitive test, a positive result means only about an 8.8% chance of actually having the disease. This is the base rate effect: the disease is rare enough that false positives swamp true positives. This counterintuitive result explains why medical screening for rare conditions requires confirmation tests.
Match each Bayesian term to its precise definition.
Terms
Definitions
Drag terms onto their definitions, or click a term then click a definition to match.
You do not need to compute exact probabilities to think in a Bayesian way. The core habit is asking three questions whenever you encounter evidence: How likely was I to see this evidence if my hypothesis were true? How likely was I to see it if my hypothesis were false? How common is my hypothesis to begin with? Holding all three questions in mind calibrates your belief far better than gut reaction.
Flashcards — click each card to reveal the answer
A spam filter is 99% accurate: it correctly labels 99% of spam as spam, and 99% of legitimate email as legitimate. Email is 30% spam in your inbox. An email is flagged as spam. What is the approximate probability it is actually spam?
A medical test for a very rare genetic condition (1 in 1,000,000 people) has a 99.9% true positive rate and a 0.1% false positive rate. A random person tests positive. Which statement is correct?
Bayesian Update Chain
- You are an investigator. A theft occurred at a school. You know that 5% of students have been in trouble before (your prior for any one student being the thief, ignoring other evidence).
- Evidence 1: A witness says they saw a student near the scene. The probability of being seen near the scene given guilt is 80%. The probability of being near the scene given innocence is 20%.
- Step 1: Apply Bayes' Theorem to compute the posterior probability after Evidence 1.
- Evidence 2: Fingerprints are found. The probability of matching fingerprints given guilt is 90%. The probability of matching fingerprints given innocence is 5%.
- Step 2: Use your posterior from Step 1 as the new prior. Apply Bayes' Theorem again.
- Step 3: Compare your final posterior to your original 5% prior. How much did the evidence shift your belief?
- Step 4: Discuss: at what posterior probability level would you consider the evidence strong enough to take action? What non-mathematical considerations (fairness, consequences of error) affect that threshold?