Module 05 of 15

Bayes' Theorem

How to update confidence in a hypothesis when new evidence arrives.

AMathematical Foundation

Bayes' theorem says how a rational degree of belief in a hypothesis H should change after observing evidence E. The posterior P(H | E) is proportional to the prior P(H) times the likelihood P(E | H). The strength of evidence is captured by the likelihood ratio LR = P(E | H) / P(E | ¬H): evidence is diagnostic only if it is more likely when H is true than when H is false. In odds form, posterior odds = prior odds × LR, which makes repeated updating simple.

P(H\mid E)=\frac{P(E\mid H)\,P(H)}{P(E\mid H)\,P(H)+P(E\mid\neg H)\,P(\neg H)}

In words: The posterior equals the likelihood times the prior, divided by the total probability of the evidence.

$P(H)$: prior probability of the hypothesis
$P(E\mid H)$: probability of seeing the evidence if H is true
$P(E\mid\neg H)$: probability of seeing the evidence if H is false

\underbrace{\frac{P(H\mid E)}{P(\neg H\mid E)}}_{\text{posterior odds}}=\underbrace{\frac{P(H)}{P(\neg H)}}_{\text{prior odds}}\times\underbrace{\frac{P(E\mid H)}{P(E\mid\neg H)}}_{\text{likelihood ratio}}

In words: Posterior odds equal prior odds times the likelihood ratio.

Assumptions

Probabilities are interpreted as degrees of belief; different people may start with different priors.
Updating on several pieces of evidence by multiplying likelihood ratios assumes they are conditionally independent given H.

BIntuitive Explanation

A positive pilot study is good news for a policy only to the extent that positive pilots are rarer when policies do not work. If weak designs and selective publication mean that even ineffective policies often produce positive pilots, a positive result should move you only a little. The question is never just “is this evidence consistent with H?” but “how much more consistent with H than with not-H?”

CConnection to Debate

Bayesian thinking gives debaters a vocabulary for weighing evidence instead of treating every study as decisive. A single striking study with a small sample in a field with low replication rates should shift belief less than a pre-registered replication. Evidence that both sides' hypotheses predict equally well (for example, “some graduates thrive remotely”) has a likelihood ratio near one and is not worth much in the round, however vivid.

DReal-World Application

Weighing a single striking finding
Competing interpretation
When the Open Science Collaboration replicated 100 published psychology experiments, roughly one-third to one-half reproduced, depending on the criterion. And 96% of sampled standard psychology articles reported support for their first hypothesis, against 44% of Registered Reports, where the decision to publish is made before results are known (Scheel et al., 2021). Both findings imply that a published positive result is fairly likely even when the hypothesis is false (a high P(E | ¬H)), so a single unreplicated result should move belief less than its significance level suggests.
Sources
- Estimating the reproducibility of psychological scienceOpen Science Collaboration (Science 349(6251), aac4716), 2015-08-28(abstract only)
- An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered ReportsAnne M. Scheel, Mitchell R. M. J. Schijen, Daniël Lakens (Advances in Methods and Practices in Psychological Science 4(2)), 2021-04-16(abstract only)

These examples describe real policies and research. The numbers in the interactive below and in the practice questions are hypothetical teaching data unless a question says otherwise.

EInteractive Visualization

Set a prior and the two likelihoods, add independent pieces of evidence one at a time, and track the posterior in natural frequencies.

Illustrative example — hypothetical data.Hypothetical example: H = “this policy reduces the harm”, E = “a pilot study reports a positive result”.

Prior probability P(H)20%

How plausible H is before this evidence.

P(E | H): positive result if H is true80%

P(E | not H): positive result even if H is false30%

Publication bias, weak designs and chance all raise this.

Independent positive results observed: 1

P(H\mid E)=\frac{P(E\mid H)\,P(H)}{P(E\mid H)\,P(H)+P(E\mid \neg H)\,P(\neg H)}

In words: The probability of H given E equals P of E given H times P of H, divided by that same product plus P of E given not H times P of not H.

Posterior P(H | evidence)

40.0%

Likelihood ratio P(E|H) / P(E|¬H)

2.67

E favours H

In natural frequencies (latest update, imagine 1,000 similar situations):

In 200, H is true; 160 of these produce a positive result.
In 800, H is false; 240 of these still produce a positive result.
So of 400 positive results, 160 come from cases where H is true: 40.0%.

Repeated updating assumes each result is independent given H. Five studies that share the same flawed design are closer to one piece of evidence than to five.

FDebate Practice

GLimitations

Bayesian updating is a model of rational belief about facts. It does not tell you which values should take priority, and its results depend on priors and likelihoods that are often estimates. It also assumes evidence is reported honestly and that the hypotheses are well defined; in debates, the hypotheses themselves are often vague and need sharpening first.

AMathematical Foundation

P(H\mid E)=\frac{P(E\mid H)\,P(H)}{P(E\mid H)\,P(H)+P(E\mid\neg H)\,P(\neg H)}

In words: The posterior equals the likelihood times the prior, divided by the total probability of the evidence.

$P(H)$: prior probability of the hypothesis
$P(E\mid H)$: probability of seeing the evidence if H is true
$P(E\mid\neg H)$: probability of seeing the evidence if H is false

\underbrace{\frac{P(H\mid E)}{P(\neg H\mid E)}}_{\text{posterior odds}}=\underbrace{\frac{P(H)}{P(\neg H)}}_{\text{prior odds}}\times\underbrace{\frac{P(E\mid H)}{P(E\mid\neg H)}}_{\text{likelihood ratio}}

In words: Posterior odds equal prior odds times the likelihood ratio.

Assumptions

Probabilities are interpreted as degrees of belief; different people may start with different priors.
Updating on several pieces of evidence by multiplying likelihood ratios assumes they are conditionally independent given H.

BIntuitive Explanation

CConnection to Debate

DReal-World Application

Weighing a single striking finding
Competing interpretation
When the Open Science Collaboration replicated 100 published psychology experiments, roughly one-third to one-half reproduced, depending on the criterion. And 96% of sampled standard psychology articles reported support for their first hypothesis, against 44% of Registered Reports, where the decision to publish is made before results are known (Scheel et al., 2021). Both findings imply that a published positive result is fairly likely even when the hypothesis is false (a high P(E | ¬H)), so a single unreplicated result should move belief less than its significance level suggests.
Sources
- Estimating the reproducibility of psychological scienceOpen Science Collaboration (Science 349(6251), aac4716), 2015-08-28(abstract only)
- An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered ReportsAnne M. Scheel, Mitchell R. M. J. Schijen, Daniël Lakens (Advances in Methods and Practices in Psychological Science 4(2)), 2021-04-16(abstract only)

These examples describe real policies and research. The numbers in the interactive below and in the practice questions are hypothetical teaching data unless a question says otherwise.

EInteractive Visualization

Set a prior and the two likelihoods, add independent pieces of evidence one at a time, and track the posterior in natural frequencies.

Illustrative example — hypothetical data.Hypothetical example: H = “this policy reduces the harm”, E = “a pilot study reports a positive result”.

Prior probability P(H)20%

How plausible H is before this evidence.

P(E | H): positive result if H is true80%

P(E | not H): positive result even if H is false30%

Publication bias, weak designs and chance all raise this.

Independent positive results observed: 1

P(H\mid E)=\frac{P(E\mid H)\,P(H)}{P(E\mid H)\,P(H)+P(E\mid \neg H)\,P(\neg H)}

In words: The probability of H given E equals P of E given H times P of H, divided by that same product plus P of E given not H times P of not H.

Posterior P(H | evidence)

40.0%

Likelihood ratio P(E|H) / P(E|¬H)

2.67

E favours H

In natural frequencies (latest update, imagine 1,000 similar situations):

In 200, H is true; 160 of these produce a positive result.
In 800, H is false; 240 of these still produce a positive result.
So of 400 positive results, 160 come from cases where H is true: 40.0%.

Repeated updating assumes each result is independent given H. Five studies that share the same flawed design are closer to one piece of evidence than to five.

GLimitations