Module 07 of 15

Simpson's Paradox

How a trend in every group can reverse when the groups are combined.

AMathematical Foundation

Simpson's paradox occurs when a comparison that holds within every subgroup reverses (or vanishes) when the subgroups are pooled. It happens because the pooled rate is a weighted average of the group rates, and the weights (group sizes) differ between the things being compared. If one option is mostly applied in a group with a low baseline rate, its pooled rate is dragged down even if it beats the alternative within every group.

\bar p_A=\sum_g w_{A,g}\,p_{A,g},\qquad w_{A,g}=\frac{n_{A,g}}{\sum_h n_{A,h}}

In words: The pooled rate for A is the weighted average of A's group rates, where each weight is the share of A's cases in that group.

$p_{A,g}$: rate for option A in group g
$w_{A,g}$: share of A's cases that fall in group g

Assumptions

The grouping variable is related both to which option is applied and to the outcome (it is a confounder).
Which comparison is “correct” depends on the causal question, not on the arithmetic.

BIntuitive Explanation

Department A gives more top grades than Department B in introductory courses and in advanced courses. But A teaches mostly advanced courses, where top grades are rarer, and B teaches mostly introductory ones. Pooled, A looks stingier. Nothing is wrong with the arithmetic; the pooled number answers a different question from the grouped numbers.

CConnection to Debate

Any statistic aggregated across groups that differ in composition can mislead: grading across departments with different course mixes, outcomes across schools serving different students, or success rates of policies applied mainly to harder cases. In the grade-caps motion, comparing the share of top grades across universities or departments without accounting for course mix can support the wrong conclusion about grade inflation or strictness.

DReal-World Application

Grade distributions across departments
Competing interpretation
Harvard's Faculty of Arts and Sciences voted in 2026 to limit A grades to 20% of enrolment per course (with a small allowance) from fall 2027, after a faculty report found more than 60% of 2024–25 undergraduate grades were A's. When Princeton ended its A-grade targets in 2014, some faculty warned about disparities between departments. Comparisons of grading across departments are vulnerable to Simpson's paradox: if departments teach different mixes of introductory and advanced courses, one department can award more A's than another at every course level and still fewer overall. Compare within course levels before aggregating.
Sources
- FAS Passes a Grade CapAbigail S. Gerstein, Amann S. Mahajan (The Harvard Crimson), 2026-05-20
- Grading, UnboundPrinceton Alumni Weekly, 2014-11-12

These examples describe real policies and research. The numbers in the interactive below and in the practice questions are hypothetical teaching data unless a question says otherwise.

EInteractive Visualization

Edit counts in a grouped table and watch the within-group comparison and the combined comparison agree or reverse.

Illustrative example — hypothetical data.Hypothetical counts of students awarded a top grade by two departments (A and B), split by course level.

Dataset

Top grades awarded by department and course level
Group	Dept A: top / students	A rate	Dept B: top / students	B rate	Higher
Introductory courses	/	90.0%	/	80.0%	A
Advanced courses	/	55.0%	/	45.0%	A
All courses combined	128 / 220	58.2%	169 / 220	76.8%	B

Reversal under aggregation?

Yes

Combined gap (A − B)

-18.6 pts

Department A awards a higher share of top grades in every course level, yet department B looks more generous overall. The reason is composition: the two departments teach very different mixes of easy and hard courses, and the combined rate is a weighted average dominated by each department’s largest group.

Which comparison is right depends on the causal question. If course level affects grades but is not caused by the department, compare within levels.

FDebate Practice

GLimitations

Simpson's paradox does not mean the grouped comparison is always right. If the grouping variable is itself caused by the option (a mediator), stratifying on it can mislead. Deciding which comparison answers the question requires a causal model of how groups, options and outcomes are related.

AMathematical Foundation

\bar p_A=\sum_g w_{A,g}\,p_{A,g},\qquad w_{A,g}=\frac{n_{A,g}}{\sum_h n_{A,h}}

In words: The pooled rate for A is the weighted average of A's group rates, where each weight is the share of A's cases in that group.

$p_{A,g}$: rate for option A in group g
$w_{A,g}$: share of A's cases that fall in group g

Assumptions

The grouping variable is related both to which option is applied and to the outcome (it is a confounder).
Which comparison is “correct” depends on the causal question, not on the arithmetic.

BIntuitive Explanation

CConnection to Debate

DReal-World Application

Grade distributions across departments
Competing interpretation
Harvard's Faculty of Arts and Sciences voted in 2026 to limit A grades to 20% of enrolment per course (with a small allowance) from fall 2027, after a faculty report found more than 60% of 2024–25 undergraduate grades were A's. When Princeton ended its A-grade targets in 2014, some faculty warned about disparities between departments. Comparisons of grading across departments are vulnerable to Simpson's paradox: if departments teach different mixes of introductory and advanced courses, one department can award more A's than another at every course level and still fewer overall. Compare within course levels before aggregating.
Sources
- FAS Passes a Grade CapAbigail S. Gerstein, Amann S. Mahajan (The Harvard Crimson), 2026-05-20
- Grading, UnboundPrinceton Alumni Weekly, 2014-11-12

These examples describe real policies and research. The numbers in the interactive below and in the practice questions are hypothetical teaching data unless a question says otherwise.

EInteractive Visualization

Edit counts in a grouped table and watch the within-group comparison and the combined comparison agree or reverse.

Illustrative example — hypothetical data.Hypothetical counts of students awarded a top grade by two departments (A and B), split by course level.

Dataset

Top grades awarded by department and course level
Group	Dept A: top / students	A rate	Dept B: top / students	B rate	Higher
Introductory courses	/	90.0%	/	80.0%	A
Advanced courses	/	55.0%	/	45.0%	A
All courses combined	128 / 220	58.2%	169 / 220	76.8%	B

Reversal under aggregation?

Yes

Combined gap (A − B)

-18.6 pts

Which comparison is right depends on the causal question. If course level affects grades but is not caused by the department, compare within levels.

FDebate Practice