Module 04 of 15

Base Rate Fallacy

Why a highly accurate test can still produce mostly false alarms.

AMathematical Foundation

The base rate is the proportion of a population that has a characteristic before any test or evidence is considered. The base rate fallacy is judging the probability that a flagged case is genuine from the test's accuracy alone, ignoring the base rate. When a condition is rare, even a small false positive rate applied to the large unaffected majority produces more false alarms than the true cases found among the small affected minority. The positive predictive value (PPV) combines all three quantities.

\text{PPV}=\frac{\text{Se}\cdot p}{\text{Se}\cdot p+\text{FPR}\cdot(1-p)}

In words: The positive predictive value equals sensitivity times prevalence, divided by that plus the false positive rate times one minus prevalence.

$p$: base rate (prevalence)
$\text{Se}$: sensitivity: P(positive | condition)
$\text{FPR}$: false positive rate: P(positive | no condition)

Assumptions

Sensitivity and false positive rate are the same in the population being screened as where they were measured.
The base rate refers to the population actually being tested, which may differ from the general population.

BIntuitive Explanation

Imagine 10,000 people and a condition affecting 1%: 100 people. A test catches 90 of them. Of the 9,900 without the condition, it wrongly flags 5%: 495 people. So 585 people are flagged and only 90 have the condition, about 15%. The test is “90% accurate” in one sense and wrong about most flagged people in another.

CConnection to Debate

Base rates matter whenever a debate uses an exceptional case or a screening rule. “Esports players can earn millions” ignores the base rate of aspiring players who earn little. “Televised trials let the public spot wrongful convictions” must be weighed against how often broadcasts lead to wrongful inferences. In prenuptial debates, the probability of divorce for a specific couple is not the same as a headline divorce rate. Ask: out of how many?

DReal-World Application

Aspiring professionals vs. visible stars
Competing interpretation
For careers with very few top positions (professional esports, elite football academies, influencer careers), the relevant base rate is the share of aspirants who reach a sustainable income, not the income of those who did. Data on the very top are easy to find; the top esports earners each have several million dollars in recorded prize money. The base rate for all aspirants is rarely published. When it is missing, the defensible debate claim is that the probability is unknown, not that it is high or low.
Sources
- Top Players by Overall EarningsEsports Earnings (esportsearnings.com)
Divorce probability for a given marriage
Competing interpretation
CDC's provisional 2023 figures give 2.4 divorces and 6.1 marriages per 1,000 population (45 reporting states and D.C.). That is a crude rate per head of population, not the probability that a given marriage ends in divorce, which depends on cohort, age at marriage and how long the marriage has lasted. Arguments for mandatory prenuptial agreements need the base rate for the couples the policy would affect, and that cannot be read off the crude rate.
Sources
- FastStats: Marriage and DivorceU.S. Centers for Disease Control and Prevention, National Center for Health Statistics, 2023

These examples describe real policies and research. The numbers in the interactive below and in the practice questions are hypothetical teaching data unless a question says otherwise.

EInteractive Visualization

Set prevalence, sensitivity and false positive rate, and see natural-frequency counts and the chance a flagged person really has the condition.

Illustrative example — hypothetical data.Adjustable teaching numbers for a hypothetical screening test applied to 10,000 people.

Base rate (prevalence)1.0%

Share of people who actually have the condition.

Sensitivity90%

Of people with the condition, the share the test flags.

False positive rate5.0%

Of people without the condition, the share wrongly flagged.

P(\text{condition}\mid +) = \frac{\text{true positives}}{\text{true positives} + \text{false positives}}

In words: the chance someone flagged really has the condition equals true positives divided by everyone flagged.

True positives

False positives

495

False negatives

True negatives

9,405

Chance a flagged person really has the condition

15.4%

Of 585 people flagged, 90 have the condition and 495 do not.

FDebate Practice

GLimitations

Base rates are often contested: which reference class is right for this case? A couple marrying at 35 with no prior marriages belongs to a different reference class than the national average. Base-rate reasoning also says nothing about whether a policy is justified; a test with low PPV may still be worth using if false alarms are cheap and missed cases are severe.

AMathematical Foundation

\text{PPV}=\frac{\text{Se}\cdot p}{\text{Se}\cdot p+\text{FPR}\cdot(1-p)}

In words: The positive predictive value equals sensitivity times prevalence, divided by that plus the false positive rate times one minus prevalence.

$p$: base rate (prevalence)
$\text{Se}$: sensitivity: P(positive | condition)
$\text{FPR}$: false positive rate: P(positive | no condition)

Assumptions

Sensitivity and false positive rate are the same in the population being screened as where they were measured.
The base rate refers to the population actually being tested, which may differ from the general population.

BIntuitive Explanation

CConnection to Debate

DReal-World Application

Aspiring professionals vs. visible stars
Competing interpretation
For careers with very few top positions (professional esports, elite football academies, influencer careers), the relevant base rate is the share of aspirants who reach a sustainable income, not the income of those who did. Data on the very top are easy to find; the top esports earners each have several million dollars in recorded prize money. The base rate for all aspirants is rarely published. When it is missing, the defensible debate claim is that the probability is unknown, not that it is high or low.
Sources
- Top Players by Overall EarningsEsports Earnings (esportsearnings.com)
Divorce probability for a given marriage
Competing interpretation
CDC's provisional 2023 figures give 2.4 divorces and 6.1 marriages per 1,000 population (45 reporting states and D.C.). That is a crude rate per head of population, not the probability that a given marriage ends in divorce, which depends on cohort, age at marriage and how long the marriage has lasted. Arguments for mandatory prenuptial agreements need the base rate for the couples the policy would affect, and that cannot be read off the crude rate.
Sources
- FastStats: Marriage and DivorceU.S. Centers for Disease Control and Prevention, National Center for Health Statistics, 2023

These examples describe real policies and research. The numbers in the interactive below and in the practice questions are hypothetical teaching data unless a question says otherwise.

EInteractive Visualization

Set prevalence, sensitivity and false positive rate, and see natural-frequency counts and the chance a flagged person really has the condition.

Illustrative example — hypothetical data.Adjustable teaching numbers for a hypothetical screening test applied to 10,000 people.

Base rate (prevalence)1.0%

Share of people who actually have the condition.

Sensitivity90%

Of people with the condition, the share the test flags.

False positive rate5.0%

Of people without the condition, the share wrongly flagged.

P(\text{condition}\mid +) = \frac{\text{true positives}}{\text{true positives} + \text{false positives}}

In words: the chance someone flagged really has the condition equals true positives divided by everyone flagged.

True positives

False positives

495

False negatives

True negatives

9,405

Chance a flagged person really has the condition

15.4%

Of 585 people flagged, 90 have the condition and 495 do not.

GLimitations