2  Session: Introduction to Causal Inference

2.1 Lecture Slides

Open slides in new tab | Download as PDF

2.2 Overview

This session establishes the foundational concepts of causal inference that underpin the entire course. We begin by distinguishing causal questions from prediction and association tasks, using real-world examples to show how confusing correlation with causation leads to costly mistakes. Simpson’s Paradox demonstrates that the same data can support opposite conclusions depending on causal assumptions. The potential outcomes framework (Neyman-Rubin) then provides the formal language to define treatment effects. We examine the identification problem — under what assumptions can causal effects be estimated from data? — and close by introducing the key estimands (ATE, ATT, ATC, CATE) that recur throughout the course.

2.3 Lecture Notes

2.3.1 Why Causality Matters

Most business decisions are fundamentally causal: “Should we launch this ad campaign?” asks what would happen to revenue if we did, not whether past ad spending correlates with past revenue. Yet the standard data science toolkit — regression, classification, clustering — is designed for association and prediction. These are valuable tasks, but they answer different questions than the ones decision-makers actually face.

The distinction rests on a hierarchy of statistical tasks:

  1. Description: What happened? (Summary statistics, visualizations.)
  2. Prediction: Given new inputs, what outcome do we expect? (ML models, forecasting.)
  3. Causal inference: What would happen to \(Y\) if we intervened on \(X\)? (Counterfactual reasoning.)

A predictive model can be highly accurate without capturing any causal relationships. A spam filter does not need to know why certain words appear in spam — it just needs to predict the label. But a marketing manager who wants to know whether a promotional email causes higher conversion needs a fundamentally different analysis.

2.3.1.1 The practitioner gap

Hunermund, Kaminski, and Schmitt (2022) surveyed 234 business practitioners who use machine learning in their organizations. They found that the vast majority deploy ML for decision problems that are causal in nature — pricing, resource allocation, intervention targeting — yet almost none use causal inference methods. The models are optimized for prediction accuracy on historical data, but the decisions they inform require knowledge of counterfactual outcomes under different actions.

Feuerriegel, Shrestha, and von Krogh (2025) provide a framework for integrating causal machine learning into organizations, distinguishing between prediction tasks (where standard ML excels) and decision tasks (where causal reasoning is required). They document several industry applications, including ABB Hitachi Power Grids’ use of heterogeneous treatment effect estimation to target quality interventions in manufacturing, reducing yield loss by half compared to a uniform treatment rule.

2.3.1.2 The eBay search advertising example

Blake, Nosko, and Tadelis (2015) provide one of the most cited examples of causal reasoning in business analytics. eBay was spending millions on branded search engine marketing (SEM) — ads that appeared when users searched for “eBay” on Google. Observational data showed a strong positive correlation between SEM exposure and purchases: users who clicked on eBay’s ads bought more than users who did not.

But eBay ran a large-scale randomized experiment, turning off branded search ads in some geographic markets while keeping them in others. The result: sales barely changed. The “effect” of SEM was almost entirely driven by a confounder — purchase intent. Consumers who searched for “eBay” already intended to buy from eBay; the ad was not causing purchases, it was merely intercepting users who would have navigated to the site organically.

The causal structure is:

  • Purchase Intent \(\rightarrow\) SEM Click
  • Purchase Intent \(\rightarrow\) Sales
  • SEM Click \(\rightarrow\) Sales (very small true causal effect)

Without the experiment, eBay would never have discovered that the observational correlation vastly overstated the causal effect. This example motivates the entire course: formal tools are needed to separate genuine causal effects from confounded associations.

2.3.2 Simpson’s Paradox

Simpson’s Paradox occurs when a trend that appears in several subgroups reverses or disappears when the groups are combined. It is not a statistical anomaly or an edge case — it arises naturally whenever there is confounding and compositional imbalance across groups.

2.3.2.1 The Google gender pay gap

In 2017, the U.S. Department of Labor accused Google of systematically underpaying its female employees. Two years later, Google’s own internal pay equity study concluded the opposite: the company was underpaying men and subsequently increased compensation for thousands of male employees. Both claims were based on real data from the same company.

The slides illustrate this with a simplified dataset of 100 women and 100 men:

  • Unadjusted comparison (ignoring job level): Women earn $3,479 on average vs. men’s $3,960 — a gap of about $481 in favor of men.
  • Within-position comparison: In Non-Management, women earn $148 more; in Management, women earn $273 more — a consistent advantage for women at every level.

The reversal happens because men are disproportionately represented in the higher-paying Management category (41% of men vs. 13% of women). When you aggregate across job levels, the compositional difference swamps the within-group effect.

The critical question is: which comparison is correct? The answer depends entirely on the causal structure:

  • If gender causes job level (e.g., discriminatory promotion practices keep women out of management), then job level is a mediator on the causal pathway from gender to salary. Adjusting for it blocks part of the total causal effect we want to measure. The unadjusted comparison is more informative for assessing total discrimination.
  • If job level is a common cause of both the group variable and salary (e.g., in a different scenario where management status drives both lifestyle choices and pay), then it is a confounder that should be adjusted for.

The same joint distribution of variables supports both stories. Statistics alone cannot resolve which analysis is correct — causal assumptions are required.

2.3.2.2 The Berkeley admissions paradox

The Berkeley admissions case (Bickel, Hammel, and O’Connell, 1975) is the most famous historical example of Simpson’s Paradox. In 1973, UC Berkeley’s graduate admissions data showed that 44% of male applicants were admitted compared to 35% of female applicants — an apparent gender bias in favor of men.

However, when researchers examined admission rates within individual departments, they found that most departments admitted women at equal or higher rates than men. A few departments even showed a statistically significant bias in favor of women.

The resolution: women applied disproportionately to highly competitive departments (such as English and History) with low overall acceptance rates, while men applied disproportionately to less competitive departments (such as Engineering and Chemistry) with high acceptance rates. The aggregate difference was driven by the composition of the applicant pool, not by within-department discrimination.

Here the causal structure matters: if gender influences department choice (which it did, through socialization and field preferences), then department is a mediator, and the aggregate comparison reflects a real consequence of the broader system even if no individual department discriminates. Whether you should “adjust for department” depends on whether you are asking “do departments discriminate?” (adjust) or “does the system as a whole produce unequal outcomes?” (do not adjust).

Simpson’s Paradox in HR analytics

2.3.3 The Potential Outcomes Framework

The potential outcomes framework provides the mathematical language for defining causal effects. It was first developed by Jerzy Neyman (1923) in the context of randomized agricultural experiments, extended to observational studies by Donald Rubin (1974), and given its modern philosophical articulation by Paul Holland (1986), who coined the phrase “the fundamental problem of causal inference.”

2.3.3.1 Core definitions

For each unit \(i\) in a population of \(n\) units:

  • \(D_i \in \{0, 1\}\): binary treatment indicator (treated vs. control)
  • \(Y_i(1)\): the outcome unit \(i\) would experience under treatment
  • \(Y_i(0)\): the outcome unit \(i\) would experience under control

These are potential outcomes — they exist conceptually regardless of which treatment the unit actually receives. The individual treatment effect (ITE) is:

\[\tau_i = Y_i(1) - Y_i(0)\]

The ITE is defined for every unit, but we can never observe it for any unit, because we can only observe one of the two potential outcomes.

NoteThe Switching Equation

The observed outcome is determined by the treatment actually received:

\[Y_i = D_i \cdot Y_i(1) + (1 - D_i) \cdot Y_i(0)\]

When \(D_i = 1\), we observe \(Y_i(1)\) and \(Y_i(0)\) is missing. When \(D_i = 0\), we observe \(Y_i(0)\) and \(Y_i(1)\) is missing. The missing potential outcome is called the counterfactual.

This is the fundamental problem of causal inference (Holland, 1986): every individual causal effect requires a quantity — the counterfactual outcome — that we can never measure. A person cannot simultaneously be treated and untreated. A customer cannot simultaneously receive and not receive a promotional offer. The missing data is not due to poor study design; it is a logical impossibility.

2.3.3.2 SUTVA: making potential outcomes well-defined

For the notation \(Y_i(0)\) and \(Y_i(1)\) to be meaningful, two sub-assumptions must hold:

No interference. Unit \(i\)’s potential outcomes depend only on its own treatment assignment, not on anyone else’s:

\[Y_i(d_1, d_2, \ldots, d_n) = Y_i(d_i)\]

This is violated in many business settings:

  • Network effects: A user joining a social platform changes the experience for existing users.
  • Market interactions: A firm’s pricing decision affects competitors’ outcomes.
  • Vaccination: Vaccinating one person reduces transmission risk for neighbors.

In online A/B testing (Session 05), interference is a first-order concern — treated and control users may interact, contaminating the experiment.

Consistency (no hidden treatment versions). If unit \(i\) receives treatment \(d\), the observed outcome equals the potential outcome under that treatment:

\[D_i = d \implies Y_i = Y_i(d)\]

This requires the treatment to be well-defined. “Taking aspirin” must mean the same thing for everyone — same dosage, same brand, same timing. If different units receive different versions of “treatment,” the potential outcomes are not comparable. Together, no interference and consistency constitute the Stable Unit Treatment Value Assumption (SUTVA).

2.3.3.3 The manipulability debate

The potential outcomes framework requires that treatments be at least conceivably manipulable. A randomized trial of a drug is the clearest case. Advertising spend is continuous but still manipulable. The resume audit study by Bertrand and Mullainathan (2004), where researchers sent identical resumes with names signaling different racial groups, pushes the boundary: race itself is not manipulable, but perceived race on a resume is.

Holland (1986) argued that causes must be things that can, at least in principle, be different for the same unit. This excludes attributes like “being female” as direct causes (we cannot randomize someone’s gender), though we can study the causal effects of signals or perceptions of gender. This perspective differs from the structural causal model (SCM) tradition associated with Judea Pearl, which allows any variable in a structural equation to be a cause. The two frameworks largely agree on practical questions but diverge philosophically on what constitutes a legitimate causal question. We will encounter the DAG-based perspective in Session 02.

2.3.4 Identification

The identification problem asks: under what assumptions can a causal quantity (defined in terms of unobservable potential outcomes) be expressed as a function of observable data?

TipIdentification vs. Estimation

Identification is a conceptual step: can the causal effect be written in terms of observable quantities, given our assumptions? Estimation is a statistical step: given a finite sample, how do we compute it? A quantity that is not identified cannot be estimated no matter how much data we collect. A quantity that is identified can still be estimated poorly with a bad estimator or too little data.

2.3.4.1 Unconditional exchangeability

An estimand is the quantity we aim to estimate — the target of inference. In causal inference, we distinguish causal estimands (defined in terms of potential outcomes) from statistical estimands (defined in terms of observable data). Identification bridges the two.

The simplest identification result relies on exchangeability (also called ignorability). In plain language, exchangeability means that the treated and control groups are comparable — as if treatment were assigned by coin flip. Formally:

\[(Y(1), Y(0)) \perp\!\!\!\perp D\]

This says that treatment assignment is independent of potential outcomes — as if treatment were assigned by a coin flip. Under exchangeability plus consistency:

\[\mathbb{E}[Y_i(1)] - \mathbb{E}[Y_i(0)] = \mathbb{E}[Y_i \mid D_i = 1] - \mathbb{E}[Y_i \mid D_i = 0]\]

The left side is the causal estimand (ATE, defined in potential outcomes). The right side is an associational quantity computable from data (a difference in conditional means). Exchangeability ensures that the group of treated units is a representative sample from the distribution of \(Y(1)\), and similarly for controls.

In a well-conducted randomized experiment, unconditional exchangeability holds by design. In observational studies, it almost never holds because units self-select into treatment.

2.3.4.2 The selection bias decomposition

To understand why the naive comparison fails without exchangeability, consider the following decomposition:

\[\underbrace{\mathbb{E}[Y \mid D=1] - \mathbb{E}[Y \mid D=0]}_{\text{Naive estimator}} = \underbrace{ATT}_{\text{Causal effect on treated}} + \underbrace{\mathbb{E}[Y(0) \mid D=1] - \mathbb{E}[Y(0) \mid D=0]}_{\text{Selection bias}}\]

The naive difference in group means equals the causal effect on the treated (ATT) plus a selection bias term. The selection bias captures the baseline difference between treated and control groups — the difference in outcomes that would exist even without treatment. Under exchangeability, the selection bias term is zero because \(\mathbb{E}[Y(0) \mid D=1] = \mathbb{E}[Y(0) \mid D=0]\) — the groups are comparable at baseline. Without exchangeability, this term can be large and can have any sign, making the naive comparison unreliable.

NoteSelection Bias in Practice

Example: Students who enroll in SAT prep courses tend to score higher. But students who enroll are often already high-achieving or from families that invest heavily in education. The selection bias term \(\mathbb{E}[Y(0) \mid D=1] - \mathbb{E}[Y(0) \mid D=0]\) captures the fact that these students would have scored higher even without the prep course. The naive comparison overstates the causal effect of prep courses.

2.3.4.3 Conditional exchangeability

The more realistic assumption is conditional exchangeability (also called unconfoundedness or selection on observables):

\[(Y(1), Y(0)) \perp\!\!\!\perp D \mid X\]

This says: within strata defined by observed covariates \(X\), treatment assignment is independent of potential outcomes. There may be confounders, but we have measured all of them.

Positivity (also called overlap or common support) requires that every subgroup defined by \(X\) contains both treated and untreated units. Without positivity, we cannot estimate the counterfactual for one of the groups in some strata. Formally:

\[0 < P(D = 1 \mid X = x) < 1 \quad \text{for all } x \text{ with } P(X = x) > 0\]

and consistency plus no interference, the ATE is identified:

\[\tau = \mathbb{E}_X\!\Big[\mathbb{E}[Y \mid D = 1, X] - \mathbb{E}[Y \mid D = 0, X]\Big]\]

This is the adjustment formula. It says: compute the conditional mean difference within each stratum of \(X\), then average over the distribution of \(X\) in the population.

The proof proceeds in four steps, each using exactly one assumption:

\[ \begin{align*} \tau &= \mathbb{E}[Y(1)] - \mathbb{E}[Y(0)] && \text{(linearity of expectation)} \\ &= \mathbb{E}_X[\mathbb{E}[Y(1) \mid X]] - \mathbb{E}_X[\mathbb{E}[Y(0) \mid X]] && \text{(law of iterated expectations)} \\ &= \mathbb{E}_X[\mathbb{E}[Y(1) \mid D=1, X]] - \mathbb{E}_X[\mathbb{E}[Y(0) \mid D=0, X]] && \text{(unconfoundedness + positivity)} \\ &= \mathbb{E}_X[\mathbb{E}[Y \mid D=1, X]] - \mathbb{E}_X[\mathbb{E}[Y \mid D=0, X]] && \text{(consistency)} \end{align*} \]

The final expression contains only observable quantities. This identification result is the foundation for virtually every causal estimation method in the course: regression adjustment estimates the conditional expectations parametrically, matching does it nonparametrically by finding similar units, inverse probability weighting reweights the sample to simulate exchangeability, and double machine learning uses flexible ML models.

2.3.4.4 When do these assumptions fail?

The assumptions can fail in several ways:

  • Unconfoundedness fails when there are unmeasured confounders — variables that affect both treatment and outcome but are not in \(X\). This is the most common and most serious threat. Sessions 06-07 address this with instrumental variables and regression discontinuity designs that do not require unconfoundedness.
  • Positivity fails when certain covariate strata contain only treated (or only control) units. This is common with high-dimensional \(X\) or when treatment is highly selective. Extrapolation into these regions is unreliable.
  • Consistency fails when the treatment is poorly defined or has hidden versions. This is a study design issue, not a statistical one.
  • No interference fails in network settings, marketplace experiments, and public health interventions.

The identification flowchart from the slides summarizes the workflow: define the causal estimand in potential outcomes, apply identification assumptions to translate it into a statistical estimand, then use an estimation method to compute it from data.

2.3.5 Beyond the ATE

The ATE is the most commonly reported estimand, but it is not always the most decision-relevant quantity.

2.3.5.1 Average Treatment Effect on the Treated (ATT)

\[ATT = \mathbb{E}[Y(1) \mid D = 1] - \mathbb{E}[Y(0) \mid D = 1]\]

The ATT asks: for the units that actually received treatment, what was the average causal effect? The key challenge is that we observe \(\mathbb{E}[Y(1) \mid D=1]\) directly (it is just the average outcome among the treated), but \(\mathbb{E}[Y(0) \mid D=1]\) — the counterfactual outcome for treated units had they not been treated — is unobservable.

When is ATT more relevant than ATE?

  • Evaluating existing programs: A government wants to know whether a job training program benefited participants, not what it would do for the general population. The ATT is the right estimand.
  • Opt-in interventions: When treatment is voluntary, the ATT captures the effect for the self-selected population that chose treatment. Scaling the program to non-participants (who may differ systematically) would require the ATE.
  • Difference-in-differences: The canonical DiD estimand is the ATT, not the ATE (Sessions 10-11).

2.3.5.2 Average Treatment Effect on the Control (ATC)

\[ATC = \mathbb{E}[Y(1) \mid D = 0] - \mathbb{E}[Y(0) \mid D = 0]\]

The ATC asks: what would happen to untreated units if they were given treatment? This is relevant when considering program expansion. Note: some authors call this the ATU (Average Treatment Effect on the Untreated); ATC and ATU are synonyms.

2.3.5.3 Decomposition

The ATE is a weighted average of ATT and ATC:

\[ATE = P(D=1) \cdot ATT + P(D=0) \cdot ATC\]

When \(ATT = ATC\), treatment effects do not vary with treatment propensity. When they differ, the choice of estimand matters substantively. As Andrew Heiss explains in his blog post on ATE/ATT/ATU: these are genuinely different quantities that answer different questions, and conflating them is a common source of confusion in applied work.

2.3.5.4 Conditional Average Treatment Effect (CATE)

\[CATE(x) = \mathbb{E}[Y(1) \mid X = x] - \mathbb{E}[Y(0) \mid X = x]\]

The CATE is the treatment effect for the subpopulation with covariates \(X = x\). It captures treatment effect heterogeneity — the fact that treatments do not work equally well for everyone.

CATE estimation is central to personalized decision-making:

  • Precision medicine: Which patients benefit from a new drug?
  • Targeted marketing: Which customers respond to a promotion?
  • Manufacturing quality: Which production units benefit from an intervention? (The ABB Hitachi example from Feuerriegel et al., 2025.)

Estimating CATEs requires more data and stronger modeling assumptions than estimating the ATE. Sessions 08-09 cover the main methods: Double Machine Learning, Causal Forests (Wager and Athey, 2018), and meta-learners (S-learner, T-learner, X-learner).

2.4 Coding Lab

2.4.1 Exercise 1: Spot the Confounder

Difficulty: Basic | Estimated time: 25 minutes

Below are four real-world claims that suggest a causal relationship. For each claim:

  1. Identify the claimed causal relationship (\(X \rightarrow Y\)).
  2. Propose a plausible confounder \(Z\) that could explain the association without a direct causal link.
  3. Draw a DAG using ggdag (R) or graphviz/networkx (Python) showing the confounding structure.

Claims:

  1. “Ice cream sales cause drownings.”
  2. “Hospital patients who receive more treatment have higher mortality.”
  3. “Countries with more Nobel laureates consume more chocolate.”
  4. “Students who use tutoring services get lower grades.”
# Your code here: for each claim, draw a DAG with ggdag
library(ggdag)
library(ggplot2)

# Example structure for Claim 1:
# coord_dag <- list(
#   x = c(IceCream = 0, ??? = 1, Drownings = 2),
#   y = c(IceCream = 0, ??? = 1, Drownings = 0)
# )
# dag <- dagify(
#   IceCream ~ ???,
#   Drownings ~ ???,
#   coords = coord_dag
# )
# Your code here: for each claim, draw a DAG
# Option A: graphviz
# from graphviz import Digraph
# dot = Digraph()
# dot.edge("Confounder", "X")
# dot.edge("Confounder", "Y")
# dot.render(view=True)

# Option B: networkx
# import networkx as nx
# import matplotlib.pyplot as plt
# G = nx.DiGraph()
# G.add_edges_from([("Confounder", "X"), ("Confounder", "Y")])
Show solution 
library(ggdag)

Attaching package: 'ggdag'
The following object is masked from 'package:stats':

    filter
Show solution 
library(ggplot2)
library(cowplot)

# Claim 1: Ice cream -> Drownings (confounder: Temperature/Season)
coord1 <- list(
  x = c(IceCream = 0, Temperature = 1, Drownings = 2),
  y = c(IceCream = 0, Temperature = 1, Drownings = 0)
)
dag1 <- dagify(
  IceCream ~ Temperature,
  Drownings ~ Temperature,
  coords = coord1
)
p1 <- dag1 |>
  ggplot(aes(x = x, y = y, xend = xend, yend = yend)) +
  geom_dag_point(colour = "grey") +
  geom_dag_edges() +
  geom_dag_text(colour = "black", size = 4) +
  theme_dag() +
  labs(title = "Claim 1: Ice cream and drownings")

# Claim 2: Treatment -> Mortality (confounder: Disease severity)
coord2 <- list(
  x = c(Treatment = 0, Severity = 1, Mortality = 2),
  y = c(Treatment = 0, Severity = 1, Mortality = 0)
)
dag2 <- dagify(
  Treatment ~ Severity,
  Mortality ~ Severity,
  Mortality ~ Treatment,
  coords = coord2
)
p2 <- dag2 |>
  ggplot(aes(x = x, y = y, xend = xend, yend = yend)) +
  geom_dag_point(colour = "grey") +
  geom_dag_edges() +
  geom_dag_text(colour = "black", size = 4) +
  theme_dag() +
  labs(title = "Claim 2: Treatment and mortality")

# Claim 3: Chocolate -> Nobel (confounder: National wealth/GDP)
coord3 <- list(
  x = c(Chocolate = 0, Wealth = 1, Nobels = 2),
  y = c(Chocolate = 0, Wealth = 1, Nobels = 0)
)
dag3 <- dagify(
  Chocolate ~ Wealth,
  Nobels ~ Wealth,
  coords = coord3
)
p3 <- dag3 |>
  ggplot(aes(x = x, y = y, xend = xend, yend = yend)) +
  geom_dag_point(colour = "grey") +
  geom_dag_edges() +
  geom_dag_text(colour = "black", size = 4) +
  theme_dag() +
  labs(title = "Claim 3: Chocolate and Nobel prizes")

# Claim 4: Tutoring -> Lower grades (confounder: Academic difficulty)
coord4 <- list(
  x = c(Tutoring = 0, Difficulty = 1, Grades = 2),
  y = c(Tutoring = 0, Difficulty = 1, Grades = 0)
)
dag4 <- dagify(
  Tutoring ~ Difficulty,
  Grades ~ Difficulty,
  Grades ~ Tutoring,
  coords = coord4
)
p4 <- dag4 |>
  ggplot(aes(x = x, y = y, xend = xend, yend = yend)) +
  geom_dag_point(colour = "grey") +
  geom_dag_edges() +
  geom_dag_text(colour = "black", size = 4) +
  theme_dag() +
  labs(title = "Claim 4: Tutoring and grades")

plot_grid(p1, p2, p3, p4, ncol = 2)

Discussion:

  1. Ice cream and drownings: Temperature (or season) is the confounder. Hot weather increases both ice cream consumption and swimming activity, leading to more drownings.
  2. Treatment and mortality: Disease severity is the confounder. Sicker patients receive more aggressive treatment and have higher baseline mortality. The treatment may actually reduce mortality, but the confounded association makes it look harmful.
  3. Chocolate and Nobel prizes: National wealth (GDP per capita) confounds the relationship. Wealthier countries invest more in research infrastructure (producing Nobel laureates) and also consume more luxury goods like chocolate.
  4. Tutoring and grades: Academic difficulty or poor prior performance is the confounder. Students who struggle seek tutoring because they are struggling — the same difficulty that drives them to tutoring also produces lower grades.
Show solution 
from graphviz import Digraph

# Claim 1: Ice cream and drownings
dot1 = Digraph("Claim1")
dot1.attr(label="Claim 1: Ice cream and drownings", labelloc="t")
dot1.node("Temperature")
dot1.node("IceCream", "Ice Cream")
dot1.node("Drownings")
dot1.edge("Temperature", "IceCream")
dot1.edge("Temperature", "Drownings")
dot1.render("claim1_dag", format="png", cleanup=True)

# Claim 2: Treatment and mortality
dot2 = Digraph("Claim2")
dot2.attr(label="Claim 2: Treatment and mortality", labelloc="t")
dot2.node("Severity", "Disease\nSeverity")
dot2.node("Treatment")
dot2.node("Mortality")
dot2.edge("Severity", "Treatment")
dot2.edge("Severity", "Mortality")
dot2.edge("Treatment", "Mortality")
dot2.render("claim2_dag", format="png", cleanup=True)

# Claim 3: Chocolate and Nobel prizes
dot3 = Digraph("Claim3")
dot3.attr(label="Claim 3: Chocolate and Nobel prizes", labelloc="t")
dot3.node("Wealth", "National\nWealth")
dot3.node("Chocolate")
dot3.node("Nobels", "Nobel\nLaureates")
dot3.edge("Wealth", "Chocolate")
dot3.edge("Wealth", "Nobels")
dot3.render("claim3_dag", format="png", cleanup=True)

# Claim 4: Tutoring and grades
dot4 = Digraph("Claim4")
dot4.attr(label="Claim 4: Tutoring and grades", labelloc="t")
dot4.node("Difficulty", "Academic\nDifficulty")
dot4.node("Tutoring")
dot4.node("Grades")
dot4.edge("Difficulty", "Tutoring")
dot4.edge("Difficulty", "Grades")
dot4.edge("Tutoring", "Grades")
dot4.render("claim4_dag", format="png", cleanup=True)

2.4.2 Exercise 2: Simpson’s Paradox in R

Difficulty: Applied | Estimated time: 45 minutes

The dataset below exhibits Simpson’s Paradox. A company has two groups of employees (Group A and Group B). The variable position indicates whether an employee is in management or non-management. We want to understand whether there is a salary difference between groups.

Tasks:

  1. Compute the unadjusted mean salary difference between Group A and Group B.
  2. Compute the adjusted mean salary difference, conditioning on position.
  3. Reproduce the crossing-lines plot from the lecture: plot average salary by group, with separate lines for each position level and for “All Positions.”
  4. Discuss: under which causal story is the adjusted estimate the correct one? Under which is the unadjusted estimate correct?
# Generate the dataset
set.seed(42)

n <- 200
group <- rep(c("A", "B"), each = 100)

# Group A: 85% non-management, 15% management
# Group B: 40% non-management, 60% management
position <- c(
  sample(c("Management", "Non-Management"), 100, replace = TRUE,
         prob = c(0.15, 0.85)),
  sample(c("Management", "Non-Management"), 100, replace = TRUE,
         prob = c(0.60, 0.40))
)

salary <- ifelse(
  position == "Management",
  ifelse(group == "A",
         rnorm(sum(position == "Management" & group == "A"), 5600, 400),
         rnorm(sum(position == "Management" & group == "B"), 5300, 400)),
  ifelse(group == "A",
         rnorm(sum(position == "Non-Management" & group == "A"), 3200, 300),
         rnorm(sum(position == "Non-Management" & group == "B"), 3000, 300))
)

df <- data.frame(group, position, salary)

# (a) Unadjusted mean difference
# Your code here

# (b) Adjusted mean difference (within position)
# Your code here

# (c) Crossing-lines plot
# Your code here

# (d) Discussion: when is adjusted vs. unadjusted correct?
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Generate the dataset
np.random.seed(42)

n = 200
group = ["A"] * 100 + ["B"] * 100

position_A = np.random.choice(
    ["Management", "Non-Management"], size=100,
    p=[0.15, 0.85]
)
position_B = np.random.choice(
    ["Management", "Non-Management"], size=100,
    p=[0.60, 0.40]
)
position = np.concatenate([position_A, position_B])

salary = np.where(
    position == "Management",
    np.where(
        np.array(group) == "A",
        np.random.normal(5600, 400, n),
        np.random.normal(5300, 400, n)
    ),
    np.where(
        np.array(group) == "A",
        np.random.normal(3200, 300, n),
        np.random.normal(3000, 300, n)
    )
)

df = pd.DataFrame({"group": group, "position": position, "salary": salary})

# (a) Unadjusted mean difference
# Your code here

# (b) Adjusted mean difference (within position)
# Your code here

# (c) Crossing-lines plot
# Your code here

# (d) Discussion
Show solution 
library(dplyr)

Attaching package: 'dplyr'
The following objects are masked from 'package:stats':

    filter, lag
The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union
Show solution 
library(ggplot2)

# Generate the dataset
set.seed(42)

n <- 200
group <- rep(c("A", "B"), each = 100)

position <- c(
  sample(c("Management", "Non-Management"), 100, replace = TRUE,
         prob = c(0.15, 0.85)),
  sample(c("Management", "Non-Management"), 100, replace = TRUE,
         prob = c(0.60, 0.40))
)

salary <- ifelse(
  position == "Management",
  ifelse(group == "A",
         rnorm(sum(position == "Management" & group == "A"), 5600, 400),
         rnorm(sum(position == "Management" & group == "B"), 5300, 400)),
  ifelse(group == "A",
         rnorm(sum(position == "Non-Management" & group == "A"), 3200, 300),
         rnorm(sum(position == "Non-Management" & group == "B"), 3000, 300))
)

df <- data.frame(group, position, salary)

# (a) Unadjusted mean difference
unadjusted <- df |>
  group_by(group) |>
  summarise(mean_salary = mean(salary), .groups = "drop")

cat("Unadjusted means:\n")
Unadjusted means:
Show solution 
print(unadjusted)
# A tibble: 2 × 2
  group mean_salary
  <chr>       <dbl>
1 A           3611.
2 B           4204.
Show solution 
cat("\nUnadjusted difference (A - B):",
    unadjusted$mean_salary[unadjusted$group == "A"] -
      unadjusted$mean_salary[unadjusted$group == "B"],
    "\n")

Unadjusted difference (A - B): -592.9301
Show solution 
# (b) Adjusted mean difference (within position)
adjusted <- df |>
  group_by(position, group) |>
  summarise(mean_salary = mean(salary), .groups = "drop")

cat("\nAdjusted means by position:\n")

Adjusted means by position:
Show solution 
print(adjusted)
# A tibble: 4 × 3
  position       group mean_salary
  <chr>          <chr>       <dbl>
1 Management     A           5602.
2 Management     B           5225.
3 Non-Management A           3204.
4 Non-Management B           3006.
Show solution 
within_diffs <- adjusted |>
  tidyr::pivot_wider(names_from = group, values_from = mean_salary) |>
  mutate(diff_A_minus_B = A - B)

cat("\nWithin-position differences (A - B):\n")

Within-position differences (A - B):
Show solution 
print(within_diffs)
# A tibble: 2 × 4
  position           A     B diff_A_minus_B
  <chr>          <dbl> <dbl>          <dbl>
1 Management     5602. 5225.           376.
2 Non-Management 3204. 3006.           198.
Show solution 
# Weighted adjusted difference
position_weights <- df |>
  count(position) |>
  mutate(weight = n / sum(n))

adjusted_diff <- sum(within_diffs$diff_A_minus_B *
                       position_weights$weight)
cat("\nWeighted adjusted difference (A - B):", adjusted_diff, "\n")

Weighted adjusted difference (A - B): 261.193
Show solution 
# (c) Crossing-lines plot
plot_data <- bind_rows(
  adjusted |> rename(Salary = mean_salary, Position = position, Group = group),
  unadjusted |> mutate(Position = "All Positions") |>
    rename(Salary = mean_salary, Group = group)
)

ggplot(plot_data, aes(x = Group, y = Salary,
                      group = Position, colour = Position)) +
  geom_line(linewidth = 1.2) +
  geom_point(size = 3) +
  labs(x = NULL, y = "Average Salary ($)", colour = "Position",
       title = "Simpson's Paradox: Group salary comparison") +
  scale_colour_manual(values = c("All Positions" = "#333333",
                                 "Management" = "#FF7E15",
                                 "Non-Management" = "#1B9E77"))
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_stringMetric, as.graphicsAnnot(x$label)): no font could
be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
Unable to calculate text width/height (using zero)
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"
Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : no
font could be found for family "PublicaSans"

(d) Discussion:

  • Unadjusted is correct if group causally determines position (e.g., Group A members are systematically steered into non-management roles). Then position is a mediator, and we should not condition on it.
  • Adjusted is correct if position is a common cause of group composition and salary (e.g., both variables are determined by the department structure). Then position is a confounder, and we should condition on it.

In this simulated data, Group A earns more within each position level, but earns less overall because Group A members are disproportionately in the lower-paying Non-Management category. The aggregate comparison is confounded by the unequal distribution of positions across groups.

Show solution 
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

np.random.seed(42)

n = 200
group = ["A"] * 100 + ["B"] * 100
position_A = np.random.choice(
    ["Management", "Non-Management"], size=100, p=[0.15, 0.85]
)
position_B = np.random.choice(
    ["Management", "Non-Management"], size=100, p=[0.60, 0.40]
)
position = np.concatenate([position_A, position_B])

salary = np.where(
    position == "Management",
    np.where(
        np.array(group) == "A",
        np.random.normal(5600, 400, n),
        np.random.normal(5300, 400, n)
    ),
    np.where(
        np.array(group) == "A",
        np.random.normal(3200, 300, n),
        np.random.normal(3000, 300, n)
    )
)

df = pd.DataFrame({"group": group, "position": position, "salary": salary})

# (a) Unadjusted mean difference
unadjusted = df.groupby("group")["salary"].mean()
print("Unadjusted means:")
print(unadjusted)
print(f"\nUnadjusted difference (A - B): {unadjusted['A'] - unadjusted['B']:.2f}")

# (b) Adjusted mean difference (within position)
adjusted = df.groupby(["position", "group"])["salary"].mean().unstack()
print("\nAdjusted means by position:")
print(adjusted)
adjusted["diff"] = adjusted["A"] - adjusted["B"]
print("\nWithin-position differences (A - B):")
print(adjusted["diff"])

weights = df["position"].value_counts(normalize=True)
adj_diff = sum(adjusted.loc[p, "diff"] * weights[p] for p in weights.index)
print(f"\nWeighted adjusted difference (A - B): {adj_diff:.2f}")

# (c) Crossing-lines plot
fig, ax = plt.subplots(figsize=(8, 5))

for pos, color in [("Management", "#FF7E15"), ("Non-Management", "#1B9E77")]:
    means = df[df["position"] == pos].groupby("group")["salary"].mean()
    ax.plot(["A", "B"], [means["A"], means["B"]], "o-",
            color=color, linewidth=1.5, markersize=8, label=pos)

overall = df.groupby("group")["salary"].mean()
ax.plot(["A", "B"], [overall["A"], overall["B"]], "o-",
        color="#333333", linewidth=1.5, markersize=8, label="All Positions")

ax.set_xlabel("")
ax.set_ylabel("Average Salary ($)")
ax.set_title("Simpson's Paradox: Group salary comparison")
ax.legend()
plt.tight_layout()
plt.show()

2.4.3 Exercise 3: Potential Outcomes Table

Difficulty: Challenge | Estimated time: 40 minutes

The table below shows 10 units with treatment assignment \(D\), a covariate \(X\), and partially revealed potential outcomes. Some potential outcomes are known (imagine an oracle told us); the rest follow the usual missing-data pattern.

\(i\) \(D_i\) \(X_i\) \(Y_i(0)\) \(Y_i(1)\)
1 0 0 4 ?
2 1 0 ? 6
3 0 0 3 ?
4 1 0 ? 8
5 0 1 7 ?
6 1 1 ? 10
7 0 1 6 ?
8 1 1 ? 12
9 0 1 8 ?
10 1 1 ? 9

Tasks:

  1. Use the switching equation to fill in the observed outcome \(Y_i\) for each unit.
  2. Compute the naive ATE: difference in mean \(Y\) between treated (\(D=1\)) and control (\(D=0\)) groups.
  3. Assess ignorability: compare \(\mathbb{E}[Y(0) \mid D=1]\) vs. \(\mathbb{E}[Y(0) \mid D=0]\) using the known values. Is there evidence of selection bias?
  4. Compute the CATE separately for \(X=0\) and \(X=1\).
  5. Compute the ATT using the known potential outcomes.
# Set up the potential outcomes table
po_table <- data.frame(
  i    = 1:10,
  D    = c(0, 1, 0, 1, 0, 1, 0, 1, 0, 1),
  X    = c(0, 0, 0, 0, 1, 1, 1, 1, 1, 1),
  Y0   = c(4, NA, 3, NA, 7, NA, 6, NA, 8, NA),
  Y1   = c(NA, 6, NA, 8, NA, 10, NA, 12, NA, 9)
)

# (a) Compute observed Y using the switching equation
# Your code here

# (b) Naive ATE
# Your code here

# (c) Check ignorability: E[Y(0)|D=1] vs E[Y(0)|D=0]
# Your code here

# (d) CATE for X=0 and X=1
# Your code here

# (e) ATT
# Your code here
import numpy as np
import pandas as pd

po_table = pd.DataFrame({
    "i":  range(1, 11),
    "D":  [0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
    "X":  [0, 0, 0, 0, 1, 1, 1, 1, 1, 1],
    "Y0": [4, np.nan, 3, np.nan, 7, np.nan, 6, np.nan, 8, np.nan],
    "Y1": [np.nan, 6, np.nan, 8, np.nan, 10, np.nan, 12, np.nan, 9],
})

# (a) Compute observed Y using the switching equation
# Your code here

# (b) Naive ATE
# Your code here

# (c) Check ignorability
# Your code here

# (d) CATE for X=0 and X=1
# Your code here

# (e) ATT
# Your code here
Show solution 
library(dplyr)

po_table <- data.frame(
  i    = 1:10,
  D    = c(0, 1, 0, 1, 0, 1, 0, 1, 0, 1),
  X    = c(0, 0, 0, 0, 1, 1, 1, 1, 1, 1),
  Y0   = c(4, NA, 3, NA, 7, NA, 6, NA, 8, NA),
  Y1   = c(NA, 6, NA, 8, NA, 10, NA, 12, NA, 9)
)

# (a) Observed Y via switching equation: Y = D * Y(1) + (1 - D) * Y(0)
po_table <- po_table |>
  mutate(Y_obs = ifelse(D == 1, Y1, Y0))

cat("(a) Observed outcomes:\n")
(a) Observed outcomes:
Show solution 
print(po_table |> select(i, D, X, Y0, Y1, Y_obs))
    i D X Y0 Y1 Y_obs
1   1 0 0  4 NA     4
2   2 1 0 NA  6     6
3   3 0 0  3 NA     3
4   4 1 0 NA  8     8
5   5 0 1  7 NA     7
6   6 1 1 NA 10    10
7   7 0 1  6 NA     6
8   8 1 1 NA 12    12
9   9 0 1  8 NA     8
10 10 1 1 NA  9     9
Show solution 
# (b) Naive ATE: E[Y|D=1] - E[Y|D=0]
mean_treated   <- mean(po_table$Y_obs[po_table$D == 1])
mean_control   <- mean(po_table$Y_obs[po_table$D == 0])
naive_ate      <- mean_treated - mean_control

cat("\n(b) Naive ATE:\n")

(b) Naive ATE:
Show solution 
cat("  E[Y|D=1] =", mean_treated, "\n")
  E[Y|D=1] = 9
Show solution 
cat("  E[Y|D=0] =", mean_control, "\n")
  E[Y|D=0] = 5.6
Show solution 
cat("  Naive ATE =", naive_ate, "\n")
  Naive ATE = 3.4
Show solution 
# (c) Check ignorability: E[Y(0)|D=1] vs E[Y(0)|D=0]
# E[Y(0)|D=0] is directly observed:
E_Y0_D0 <- mean(po_table$Y0[po_table$D == 0], na.rm = TRUE)

# E[Y(0)|D=1] is not directly observed, but we can reason about it.
# With known Y(0) for D=0 units and known Y(1) for D=1 units,
# we cannot directly observe E[Y(0)|D=1].
# However, we CAN check conditional on X:
# E[Y(0)|D=0, X=0] vs E[Y(0)|D=0, X=1]

cat("\n(c) Checking for selection bias:\n")

(c) Checking for selection bias:
Show solution 
cat("  E[Y(0)|D=0] =", E_Y0_D0, "(directly observed)\n")
  E[Y(0)|D=0] = 5.6 (directly observed)
Show solution 
cat("  E[Y(0)|D=1] is unobserved (the counterfactual for treated units).\n")
  E[Y(0)|D=1] is unobserved (the counterfactual for treated units).
Show solution 
cat("  We cannot directly test unconditional ignorability.\n")
  We cannot directly test unconditional ignorability.
Show solution 
cat("  But we can check whether treated units have higher X values:\n")
  But we can check whether treated units have higher X values:
Show solution 
cat("  E[X|D=1] =", mean(po_table$X[po_table$D == 1]), "\n")
  E[X|D=1] = 0.6
Show solution 
cat("  E[X|D=0] =", mean(po_table$X[po_table$D == 0]), "\n")
  E[X|D=0] = 0.6
Show solution 
cat("  Treatment is correlated with X, suggesting potential selection.\n")
  Treatment is correlated with X, suggesting potential selection.
Show solution 
cat("  Since higher X is associated with higher Y(0),\n")
  Since higher X is associated with higher Y(0),
Show solution 
cat("  E[Y(0)|D=1] > E[Y(0)|D=0] => positive selection bias.\n")
  E[Y(0)|D=1] > E[Y(0)|D=0] => positive selection bias.
Show solution 
cat("  The naive ATE likely overstates the true causal effect.\n")
  The naive ATE likely overstates the true causal effect.
Show solution 
# (d) CATE for X=0 and X=1
# CATE(X=0): compare treated vs control within X=0
cate_x0_treated <- mean(po_table$Y_obs[po_table$D == 1 & po_table$X == 0])
cate_x0_control <- mean(po_table$Y_obs[po_table$D == 0 & po_table$X == 0])
cate_x0 <- cate_x0_treated - cate_x0_control

cate_x1_treated <- mean(po_table$Y_obs[po_table$D == 1 & po_table$X == 1])
cate_x1_control <- mean(po_table$Y_obs[po_table$D == 0 & po_table$X == 1])
cate_x1 <- cate_x1_treated - cate_x1_control

cat("\n(d) Conditional ATE:\n")

(d) Conditional ATE:
Show solution 
cat("  CATE(X=0):", cate_x0_treated, "-", cate_x0_control, "=", cate_x0, "\n")
  CATE(X=0): 7 - 3.5 = 3.5
Show solution 
cat("  CATE(X=1):", cate_x1_treated, "-", cate_x1_control, "=", cate_x1, "\n")
  CATE(X=1): 10.33333 - 7 = 3.333333
Show solution 
# Adjusted ATE (weighted by X distribution)
p_x0 <- mean(po_table$X == 0)
p_x1 <- mean(po_table$X == 1)
adjusted_ate <- p_x0 * cate_x0 + p_x1 * cate_x1
cat("  Adjusted ATE:", adjusted_ate, "\n")
  Adjusted ATE: 3.4
Show solution 
cat("  Compare to naive ATE:", naive_ate, "\n")
  Compare to naive ATE: 3.4
Show solution 
# (e) ATT: E[Y(1)|D=1] - E[Y(0)|D=1]
# E[Y(1)|D=1] is directly observed
E_Y1_D1 <- mean(po_table$Y1[po_table$D == 1], na.rm = TRUE)
# E[Y(0)|D=1] is the counterfactual; we approximate using conditional means
# Under conditional exchangeability given X:
# E[Y(0)|D=1] ≈ weighted average of E[Y(0)|D=0, X=x] over X|D=1

E_Y0_D0_X0 <- mean(po_table$Y0[po_table$D == 0 & po_table$X == 0], na.rm = TRUE)
E_Y0_D0_X1 <- mean(po_table$Y0[po_table$D == 0 & po_table$X == 1], na.rm = TRUE)

# Distribution of X among treated
n_treated_x0 <- sum(po_table$D == 1 & po_table$X == 0)
n_treated_x1 <- sum(po_table$D == 1 & po_table$X == 1)
n_treated    <- sum(po_table$D == 1)

E_Y0_D1_approx <- (n_treated_x0 / n_treated) * E_Y0_D0_X0 +
                   (n_treated_x1 / n_treated) * E_Y0_D0_X1

att <- E_Y1_D1 - E_Y0_D1_approx

cat("\n(e) ATT (under conditional exchangeability given X):\n")

(e) ATT (under conditional exchangeability given X):
Show solution 
cat("  E[Y(1)|D=1] =", E_Y1_D1, "\n")
  E[Y(1)|D=1] = 9
Show solution 
cat("  E[Y(0)|D=1] (approx) =", E_Y0_D1_approx, "\n")
  E[Y(0)|D=1] (approx) = 5.6
Show solution 
cat("  ATT =", att, "\n")
  ATT = 3.4
Show solution 
import numpy as np
import pandas as pd

po_table = pd.DataFrame({
    "i":  range(1, 11),
    "D":  [0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
    "X":  [0, 0, 0, 0, 1, 1, 1, 1, 1, 1],
    "Y0": [4, np.nan, 3, np.nan, 7, np.nan, 6, np.nan, 8, np.nan],
    "Y1": [np.nan, 6, np.nan, 8, np.nan, 10, np.nan, 12, np.nan, 9],
})

# (a) Observed Y via switching equation
po_table["Y_obs"] = np.where(po_table["D"] == 1, po_table["Y1"], po_table["Y0"])
print("(a) Observed outcomes:")
print(po_table[["i", "D", "X", "Y0", "Y1", "Y_obs"]])

# (b) Naive ATE
treated = po_table[po_table["D"] == 1]
control = po_table[po_table["D"] == 0]
naive_ate = treated["Y_obs"].mean() - control["Y_obs"].mean()
print(f"\n(b) Naive ATE:")
print(f"  E[Y|D=1] = {treated['Y_obs'].mean():.2f}")
print(f"  E[Y|D=0] = {control['Y_obs'].mean():.2f}")
print(f"  Naive ATE = {naive_ate:.2f}")

# (c) Selection bias check
print(f"\n(c) Checking for selection bias:")
print(f"  E[X|D=1] = {treated['X'].mean():.2f}")
print(f"  E[X|D=0] = {control['X'].mean():.2f}")
print("  Treatment is correlated with X => potential selection bias.")

# (d) CATE
for x_val in [0, 1]:
    sub_t = po_table[(po_table["D"] == 1) & (po_table["X"] == x_val)]
    sub_c = po_table[(po_table["D"] == 0) & (po_table["X"] == x_val)]
    cate = sub_t["Y_obs"].mean() - sub_c["Y_obs"].mean()
    print(f"\n(d) CATE(X={x_val}) = {sub_t['Y_obs'].mean():.2f}"
          f" - {sub_c['Y_obs'].mean():.2f} = {cate:.2f}")

p_x0 = (po_table["X"] == 0).mean()
p_x1 = (po_table["X"] == 1).mean()
cate_0 = (po_table[(po_table["D"]==1) & (po_table["X"]==0)]["Y_obs"].mean() -
          po_table[(po_table["D"]==0) & (po_table["X"]==0)]["Y_obs"].mean())
cate_1 = (po_table[(po_table["D"]==1) & (po_table["X"]==1)]["Y_obs"].mean() -
          po_table[(po_table["D"]==0) & (po_table["X"]==1)]["Y_obs"].mean())
adjusted_ate = p_x0 * cate_0 + p_x1 * cate_1
print(f"  Adjusted ATE = {adjusted_ate:.2f}")
print(f"  Naive ATE = {naive_ate:.2f}")

# (e) ATT under conditional exchangeability given X
E_Y1_D1 = treated["Y1"].mean()
E_Y0_D0_X0 = control[control["X"] == 0]["Y0"].mean()
E_Y0_D0_X1 = control[control["X"] == 1]["Y0"].mean()

n_t_x0 = ((po_table["D"] == 1) & (po_table["X"] == 0)).sum()
n_t_x1 = ((po_table["D"] == 1) & (po_table["X"] == 1)).sum()
n_t = (po_table["D"] == 1).sum()

E_Y0_D1_approx = (n_t_x0 / n_t) * E_Y0_D0_X0 + (n_t_x1 / n_t) * E_Y0_D0_X1
att = E_Y1_D1 - E_Y0_D1_approx
print(f"\n(e) ATT (under conditional exchangeability given X):")
print(f"  E[Y(1)|D=1] = {E_Y1_D1:.2f}")
print(f"  E[Y(0)|D=1] (approx) = {E_Y0_D1_approx:.2f}")
print(f"  ATT = {att:.2f}")

2.5 References & Resources

2.5.1 Academic Literature

  • Hernán, M. A. & Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC. Chapters 1–2. Free online
  • Neal, B. (2020). Introduction to Causal Inference from a Machine Learning Perspective. Chapters 1–2. Free online
  • Imbens, G. W. (2020). “Potential Outcome and Directed Acyclic Graph Approaches to Causality.” Journal of Economic Literature, 58(4), 1129–1179. DOI
  • Cunningham, S. (2021). Causal Inference: The Mixtape. Yale University Press. Chapter 4: Potential Outcomes. Free online
  • Hünermund, P., Kaminski, J. & Schmitt, C. (2022). “Causal Machine Learning and Business Decision Making.” Working Paper. DOI
  • Feuerriegel, S., Shrestha, Y. R. & von Krogh, G. (2025). “A New Machine Learning Approach Answers What-If Questions.” MIT Sloan Management Review, 66(3), 65–69. Link
  • Bickel, P. J., Hammel, E. A. & O’Connell, J. W. (1975). “Sex Bias in Graduate Admissions: Data from Berkeley.” Science, 187(4175), 398–404. DOI
  • Blake, T., Nosko, C. & Tadelis, S. (2015). “Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment.” Econometrica, 83(1), 155–174. DOI
  • Holland, P. W. (1986). “Statistics and Causal Inference.” Journal of the American Statistical Association, 81(396), 945–960. DOI
  • Rubin, D. B. (1974). “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies.” Journal of Educational Psychology, 66(5), 688–701. DOI
  • Bertrand, M. & Mullainathan, S. (2004). “Are Emily and Greg More Employable Than Lakisha and Jamal?” American Economic Review, 94(4), 991–1013. DOI
  • Luca, M. (2021). “Leaders: Stop Confusing Correlation with Causation.” Harvard Business Review. Link

2.5.2 Practitioner Resources

  • Heiss, A. (2024). “Demystifying Causal Inference Estimands: ATE, ATT, and ATU.” Blog post. Link
  • Wang, L., Richardson, T. & Robins, J. (2025). “Causal Inference: A Tale of Three Frameworks.” arXiv preprint. arXiv
  • Tyler Vigen, Spurious Correlations. Link