Assignment V — Rolling Out Self-Checkout

Overview

You are a data scientist at FreshField, a regional grocery chain of about 200 stores across 40 metropolitan markets. To lift weekly footfall (customer visits), the operations team is rolling out self-checkout / scan-and-go store by store. The rollout was not simultaneous or random. One flagship market went live alone — a deliberate showcase, with no comparable cohort launching alongside it. Several waves of markets converted on a staggered schedule across the year — some launching before the flagship’s solo debut, some after. The rest of the estate has not converted yet.

The board wants two numbers and a defense of each:

  1. The flagship effect — how much did self-checkout lift footfall at the single flagship market, and how confident can we be given there is no cohort to compare it against?
  2. The portfolio effect — across the staggered pilot markets, what is the average footfall lift, net of the confounds that make a naive before/after comparison misleading?

This is the Part-V capstone. It synthesises Session 10 (Difference-in-Differences) — to estimate the portfolio effect from the staggered pilots — Session 11 (Synthetic Control) — to estimate the single-flagship effect with no natural comparison group — and Session 12 (Counterfactual Imputation) — the unifying view that every one of these estimators is imputing the same blanked-out \(Y(0)\) block, differing only in the assumption it is willing to buy. You will take one panel dataset from a set of misleading naive numbers to two defensible, estimand-annotated effects, and write a board memo that justifies which lens each question selects — from the estimand, not the toolbox.

ImportantThe spine — the estimand chooses the lens

There is no single “right method” here. A board question about one unit (the flagship, with no cohort) selects synthetic control; a question about an average over many treated units (the staggered pilots) selects difference-in-differences. Both impute the same missing potential outcome — the footfall the treated markets would have had without self-checkout — they just differ in the assumption they lean on to fill in that blank. Two artefacts run through the whole assignment and are graded explicitly:

  • The assumption ledger. For every estimator you use, name the assumption it buys (parallel trends? the treated unit inside the donor hull? no post-treatment controls?) and argue whether it is plausible in this data — not in general.
  • The estimand-driven choice. For each board question, show which lens the estimand selects (one unit vs. many? does assignment depend on the outcome’s own trend? is a candidate control downstream of treatment?) — the estimand, not the toolbox, makes the call.

A memo that reports one blended number, or that averages a credible estimator with a discredited one, or that picks a method because it is fashionable rather than because the estimand demands it, has missed the point of Part V.

WarningThis assignment IS the Part-V comprehension gate — no quiz

Assignments I–IV each carry a short closed-book quiz. Assignment V has none. A5 is itself the comprehension gate for Part V, which is why it asks for interpretation and judgement, not only estimation: three of its eight parts close on a written judgement essay, and the estimators are marshalled into bake-off tables so your effort concentrates on reading them, not just running them. The parts that carry the most weight are the ones where you explain why the numbers disagree and which you would stake a decision on.

Dataset & Codebook

The data are simulated from a calibrated panel model so that a ground-truth answer exists; focus your effort on the methodology, not on hunting for the “real” numbers. You are given two views of the same rollout — use whichever the estimand calls for:

Download grocery_panel.csv (≈ 2.4 MB, 22,048 rows) — the store-week panel (212 stores × 104 weeks). Use it for the staggered difference-in-differences and the counterfactual-imputation / factor estimators.

Download grocery_markets.csv (≈ 430 KB, 4,160 rows) — the market-week aggregate (40 markets × 104 weeks). Use it for the single-flagship synthetic control.

Save both into the same folder as your source file and read them with plain relative paths.

grocery_panel.csv — store-week

Column Type Description
store_id id (chr) Store identifier (S0001..S0212).
market_id id (chr) Parent market identifier (M01..M40).
week integer Week index 1–104.
date date Calendar date (Monday) of the week; runs from 2024-03-25.
footfall integer Weekly customer visits — the OUTCOME \(Y\). Use log(footfall) so effects read as percent.
transactions integer Weekly transactions (secondary outcome, not the target).
revenue numeric Weekly revenue (secondary outcome, not the target).
sco_live binary 0/1 1 if self-checkout is live in this store-week — the TREATMENT \(D\). Derive each market’s go-live week as the first week sco_live == 1.
market_size numeric Time-invariant market scale.
urban_share numeric (0.1–0.9) Time-invariant urban catchment share.
region integer 1–4 Market region.
has_parking binary 0/1 Store attribute (time-invariant).
store_sqm integer Store floor area in m² (time-invariant).
competitor_density numeric Nearby-competitor density (time-invariant store attribute).
staffed_lanes numeric Number of staffed checkout lanes open that week.
temp_c numeric Weekly average temperature (a weather control).
confidence_index numeric National weekly consumer-confidence index (same series for all markets).

grocery_markets.csv — market-week

Column Type Description
market_id id (chr) Market identifier (M01..M40).
week, date integer, date Week index 1–104 and its calendar date.
footfall integer Market-total weekly footfall (sum over the market’s stores).
footfall_per_store numeric Mean weekly footfall per store. Use log(footfall_per_store) as the SC outcome for the flagship.
transactions, revenue integer, numeric Market-total weekly transactions and revenue.
staffed_lanes numeric Market-total staffed checkout lanes.
sco_live binary 0/1 1 if any store in the market is live that week.
confidence_index numeric National weekly consumer-confidence index (same series for all markets).
temp_c numeric Market mean weekly temperature.
n_stores integer Number of stores in the market.
market_size, urban_share, region numeric, numeric, integer Time-invariant market scale, urban catchment share, region.
NoteSome columns are candidate proxies — you decide what for

Beyond the outcome and the treatment, the panel carries observable series you may find useful: a national confidence_index, weekly temp_c, and the time-invariant region and urban_share. Whether any of them explains the movements you will uncover in Parts D–F — and which explains which — is for you to establish from the data, not something the codebook tells you.

WarningThere is no “go-live week”, “cohort”, or “flagship” column — you derive the rollout structure yourself

The data ships with the observed treatment status sco_live only. It does not tell you when each market went live, which markets share a launch wave, which never converted, or which market is the flagship. Reconstructing the rollout — each market’s go-live week, the cohort structure, the never-treated pool, and the lone flagship — is Part A, and everything downstream depends on getting it right. One warning up front: the flagship is not simply the market that goes live earliest.

TipStarter code 
library(tidyverse); library(data.table)
panel   <- fread("grocery_panel.csv")
markets <- fread("grocery_markets.csv")
# outcome Y = footfall (use log(footfall) so effects read as %) ; treatment D = sco_live
# DERIVE go-live: per market, the first week sco_live == 1 (never-treated: none)
import pandas as pd, numpy as np
panel   = pd.read_csv("grocery_panel.csv")
markets = pd.read_csv("grocery_markets.csv")
# outcome Y = footfall (use log(footfall) so effects read as %) ; treatment D = sco_live
# DERIVE go-live: per market, the first week sco_live == 1 (never-treated: none)

You may submit in R or Python — both earn full marks (see the language-parity note below). Report your results in a rendered Quarto document (.qmd → HTML/PDF) or Jupyter notebook. Show your code, your numbers, and your reasoning. Throughout: work on log(footfall) so coefficients read as percentages, and cluster standard errors at the market level (the assignment level).

The assignment is organised into eight parts A–H with the weights below (summing to 100%). Parts D, F, and H each close on a marked judgement essay — that is where the comprehension gate lives.

Part Focus Weight
A Reconstruct the rollout + the 2×2 10%
B Static/dynamic TWFE · Goodman–Bacon · the bad control 12%
C The heterogeneity-robust family (CS · SA · BJS · Gardner) 13%
D Parallel-trends test I: event study · HonestDiD · Essay 1 15%
E The single-flagship synthetic-control family 15%
F Factor imputation (IFEct & matrix completion) · naming the factors · Essay 2 20%
G Parallel-trends test II: placebo · leave-one-out · sensitivity 8%
H Heterogeneity · bake-off · board memo · Essay 3 7%

Part A — Reconstruct the rollout + the 2×2 (10%)

Before estimating anything, recover the design from the raw panel.

  • Derive each market’s go-live week as the first week sco_live == 1, and label the never-treated markets. Report the cohort structure: how many markets go live in each week, and how many never convert. Tabulate cohort sizes — you will need them in the next bullet.
  • Identify the flagship correctly. Exactly one market goes live alone — a go-live week with a single market in it. That lone market is the flagship. The earliest go-live week is not necessarily the flagship: check its cohort size. Explain why cohort size = 1 — one treated unit with no comparable launch group — is what forces a synthetic-control approach for its effect, while the multi-market waves can be pooled with difference-in-differences.
  • Sanity-check the panel. Is it balanced (every store observed in every week)? Any missingness? How many treated vs. never-treated markets? Does the never-treated pool look like a usable comparison/donor set — does its spread of market_size and urban_share bracket the treated markets (and the flagship in particular)?
  • The 2×2 by hand, and its identity with TWFE. Take the earliest multi-market wave vs. the never-treated pool, collapse each to one pre- and one post-launch period, and compute the double difference from the four cell means by hand. Then show that a two-period two-way-fixed-effects regression returns the identical number. State plainly which quantity this 2×2 is (a building block for one wave) and which it is not (the portfolio ATT).
  • The naive treated-only before/after. Compute the treated-only post-minus-pre difference on log(footfall) for the wave markets. Is it biased? In which direction, and why — is it “selection on winners”, or is something more mundane at work? (Think about what a common secular trend and seasonality do to a treated-only pre/post gap.)

State, up front, the two estimands you will target — the flagship ATT (effect on the one flagship market) and the portfolio ATT (average effect across the staggered pilot markets) — and note that they are different questions with different answers. Keep them apart for the rest of the report.

Part B — Static/dynamic TWFE, Goodman–Bacon, and the bad control (12%)

Estimate the portfolio effect on the staggered pilot waves (set the flagship aside — feeding a one-market cohort into the staggered machinery would hand the estimator a single-cluster, invalid, standard error). Build the classic DiD engine and interrogate it.

  • Static two-way fixed effects (TWFE). Regress log(footfall) on sco_live with store and week fixed effects, clustered on market. Report it. Compare to the naive number: what did the week fixed effects strip out? Compare to what you will later believe is the truth — is TWFE above or below it, and does the sign of that gap match the staggering pathology you saw in S10, or point elsewhere?
  • Dynamic TWFE event study, with a plot. Estimate effects by event time (weeks relative to each market’s own go-live), binning never-treated markets to a sentinel reference and the endpoints to a sensible window (e.g. \([-8, +12]\)). Plot the leads and lags. Are the leads flat? Note what you see — you will return to it with a sharper diagnostic in Part D.
  • Goodman–Bacon decomposition. Decompose the static TWFE estimate into its 2×2 building blocks. Report the weight on the forbidden “later-vs-earlier-treated” comparisons and the weight on the clean “treated-vs-never-treated” comparisons, with each block’s average estimate. Then answer the diagnostic question this buys you: is staggering the whole story here, or is the inflation coming from somewhere else? (If the clean block is itself far from where the robust estimators will land, staggering is not the disease.)
  • The staffed_lanes question (the bad control). A colleague reviewing your DiD says: “You should control for staffed_lanes — busier stores keep more lanes open, so it’s a confounder you’re ignoring.” Evaluate the claim rigorously before accepting it. (i) Add staffed_lanes to the TWFE specification and report what happens to the estimate. (ii) Regress staffed_lanes on sco_live (same fixed effects): what does self-checkout do to the number of staffed lanes? (iii) State the rule and apply it — is staffed_lanes a pre-treatment confounder or a post-treatment consequence of the treatment you are studying? Name the bias you would introduce by conditioning on it, and add it to your assumption ledger as the thing a good specification must not condition on.

Part C — The heterogeneity-robust family (13%)

Static TWFE is contaminated by forbidden comparisons under staggered adoption. Build the modern heterogeneity-robust menu, which forms only clean comparisons. Work on the wave markets + never-treated pool, log(footfall), market-clustered. Produce a bake-off table of these estimators:

  • Callaway–Sant’Anna (did::att_gt) — the doubly-robust ATT(g,t) grid against a never-treated control group, aggregated to an overall portfolio ATT and reported per cohort. Use a universal base period (you will need it for Part D’s event study and HonestDiD).
  • Sun–Abraham (fixest::sunab) — the same idea in one saturated regression. Report the overall ATT.
  • Borusyak–Jaravel–Spiess imputation (didimputation) — fit two-way fixed effects on the untreated cells only, impute each treated cell’s \(\hat Y(0)\), and average the treated-cell residuals (“fit white, predict grey”).
  • Gardner (did2s) — the two-stage imputation estimator (a model-answer bonus; CS/SA and BJS are the required floor).

Two things to read off the results and state explicitly:

  1. The per-cohort ATTs are a clue. Do the cohorts show the same effect, or does it trend with launch order? If earlier cohorts show larger effects, what might that be a footprint of? (Hold the interpretation for Parts D and F — but flag the pattern here.)
  2. The family will (roughly) agree — do not over-read the agreement. If four independently-implemented estimators cluster together but all sit well above the naive week-fixed-effects intuition, is that corroboration, or could it be a shared bias? Convergence across methods certifies robustness only against the failure modes those methods differ on; it says nothing about a failure mode they all share. State which shared assumption they all rest on. Do not average the ladder — the disagreement between families is a clue you chase in Parts D–F, not noise to average away.

Part E — The single-flagship synthetic control (15%)

Now the flagship — one treated market, no cohort. Work on the market-week panel (grocery_markets.csv), outcome log(footfall_per_store), with the never-treated markets as the donor pool and the pre-period being the weeks before the flagship’s go-live. Produce a small bake-off table of the synthetic-control family:

  • Classic (Abadie–Diamond–Hainmueller) synthetic control. Build a convex weighting of donors that tracks the flagship’s pre-period trajectory. Report the pre-fit RMSPE, name the donor markets carrying the most weight (is it a genuine blend, or a single-donor collapse?), and read off the post-period gap (the SC ATT).
  • Is the gap more than noise? At \(N = 1\) you cannot cluster or run a \(t\)-test. Use placebo-in-space (re-run the SC pretending each donor was treated; rank the flagship’s post/pre MSPE ratio among them for a Fisher-style \(p\)) and a modern prediction interval (scpi) for an honest effect band. Report the placebo rank as a rank, not as “significant at 5%” — with a small donor pool the Fisher \(p\) has a floor. What does each device tell you that a point estimate cannot?
  • Refine and stress-test. Fit an augmented (ridge) synthetic control and compare — does correcting the pre-fit move the answer, and should it, given how good the classic pre-fit already is? Optionally add synthetic difference-in-differences and read the DiD / SC / synthetic-DiD triple as three corners of one weighted two-way regression. Note honestly where estimates agree to the digit (a deterministic quadratic program) versus where they differ by an engine/predictor choice (synthetic-control weights are not unique). Land on a defensible flagship effect with a band, and say plainly how confident a single-unit estimate lets you be.

Part F — Factor imputation: recover and name the hidden structure (20%)

This is the heart of the assignment. Read the whole panel — waves and flagship, all 212 stores — through the counterfactual-imputation / interactive-fixed-effects lens of S12. The additive two-way model \(Y_{it}(0) = \alpha_i + \lambda_t + \varepsilon_{it}\) is one hypothesis about the missing \(Y(0)\); a factor model \(Y_{it}(0) = \alpha_i + \lambda_t + \sum_{r}\ell_{ri}F_{rt} + \varepsilon_{it}\) is a richer one. Fit both and let cross-validation choose.

  • Race IFEct against matrix completion under CV. Use fect (method "both", CV = TRUE, two-way force) to let cross-validation pick the imputer (hard-impute factor model vs. soft-impute nuclear-norm) and, for the factor model, the number of factors \(r\). Report the CV-selected method, the CV-selected \(r\), and the ATT. Plot the native dynamic-gap (event-study) figures for the factor fit and the matrix-completion fit — does one flatten the pre-period while the other leaves it lifted?
  • Report what CV gave you — do not hunt a rank. Fold draws come from the ambient RNG, so the CV pick is a seed lottery: run the CV under several seeds and measure how often each method and each \(r\) wins, then report that distribution. Do not fish for the seed that lands the rank you want — that is truth-peeking. State the disciplined pick (e.g. “CV prefers the factor model at \(r = k\) on \(m/n\) seeds here”). Note that fect’s default CV rule keeps the simplest model within one standard error of the best, not the raw argmin.
  • Extract the factors and NAME them — from data you can see. At the CV-selected rank, pull out the estimated factors \(\hat F_t\) and loadings \(\hat\ell_i\). A factor model identifies only the product \(\ell_i F_t\), so the raw signs are arbitrary — pin each factor’s sign to an observable series before interpreting. Then, using only observable columns (candidate time series: confidence_index, temp_c, a seasonal cosine; candidate loading axes: region, urban_share), work out what each latent factor is. Beware one trap: some fect factor plots label the two-way time fixed effect as a “factor” — it is not; a real factor loads in opposite directions across units (a fan), a time effect lifts everyone the same way.

Judgement Essay 2 (part of the 20%)

Prompt. Name each latent factor you recovered, citing the observable series it correlates with (and the loading axis that explains which markets feel it). Then determine which factor, if any, broke parallel trends — by relating each factor’s loading to the rollout order (correlate the loading with each market’s go-live week). Convict the guilty factor and acquit the innocent one, each with its number. Translate the verdict into a business story: what real operations decision does the guilty factor correspond to, and why does that decision — however sensible it sounds — corrupt the naive and DiD estimates while leaving the factor model standing? Address the sign/scale indeterminacy explicitly (how did you pin the signs?).

Write this as a short essay (≈ 350–550 words). The load-bearing evidence is the loading-vs-go-live correlation, not an assertion — cite it for both factors.

WarningRuntime honesty — read before you fit

Several fits here are slow, and it is not only the fect machinery. Budget for a full uncached run of roughly 75–90 minutes, dominated by one fit: the leave-one-period-out pre-trend refit in Part G takes roughly 60 minutes at a bootstrap count large enough to be trustworthy. It refits the whole model once per pre-period, and its joint pre-trend tests invert a pre-period-by-pre-period bootstrap covariance over ~60 pre-periods — which needs nboots well above that (≈200) to be stable; smaller values return NA (below ~60) or unstable p-values (~100). A much faster, deterministic alternative is se = FALSE, which uses the jackknife variance instead of the bootstrap (the approach the S12 lecture takes) and sidesteps the conditioning problem entirely — use it as a fallback if the 60-minute bootstrap run is impractical on your machine. The rest is quick: the cross-validated method = "both" fit takes a few minutes; the fect_sens relative-magnitudes sweep in Part G adds several more minutes; and the HonestDiD sensitivity in Part D takes a few minutes of its own. Cache every one of these fits (save the fitted objects to disk and reload them) so a re-render is near-instant — the instructor must be able to re-render your report without re-running the slow fits. And remember the CV point above: the fold draws are seeded by the ambient RNG, so set your seed before the call and report the CV pick you actually got, rather than searching for a particular rank.

Part H — Heterogeneity, the bake-off, and the board memo (7%)

Bring everything together for FreshField’s board.

  • Heterogeneity from the imputation estimator. The factor fit returns an estimated effect for every treated store-week, not just a headline. Split the treated markets by urban_share (e.g. tertiles) and report the effect by group. Is the lift uniform, or graded? Guard against two artefacts: a common-horizon check (do groups launched at different times get averaged over ramps of different lengths?) and a per-group placebo audit (is any group’s counterfactual contaminated?).
  • The estimand-annotated bake-off table. One row per estimator: the naive number, static TWFE, the heterogeneity-robust family, the factor/MC imputers, and the flagship synthetic-control family — each with its estimand (portfolio vs. flagship), its estimate, and the assumption it buys. Rank by assumption credibility, not by number. This table is your assumption ledger.
  • The estimand-driven method choice, stated explicitly. Which lens does each of the board’s two questions select, and why the estimand — not the tool — chose it? Name the two traps you defused (a rollout selected on the outcome’s own trend; a post-treatment “control”).
  • A one-page board memo. The flagship effect and the portfolio effect, each as a number plus a counterfactual and a caveat, an honest verdict on how much to trust a single-unit estimate, a clear recommendation on the rollout, and the heterogeneity finding if you trust it. Do not collapse the two effects into one headline.

Judgement Essay 3 (part of the 7%)

Prompt. Is the urban heterogeneity you measured a genuine causal effect-modification finding, or an artefact of the factor model / the rollout schedule? Separate the possible artefact channels (timing, the counterfactual model, the choice of rank), rule out what you can and bound what you cannot, and state concretely what you would need to see to believe it fully.

Write this as a short essay (≈ 300–500 words). The audit, not the anecdote, decides — do not transplant a conclusion from lecture; read this data.

Rubric

Each part is graded on the four-column scale below. Parts D, F, and H bundle their judgement essay into the stated weight.

Part Weight Excellent Good Needs Improvement Missing / Inadequate
A — Rollout + 2×2 10% Go-live derived from sco_live; cohort structure correct; the flagship identified as the lone cohort (with the explicit note it is not the earliest wave, and that cohort-size-1 forces SC over DiD); both estimands stated as distinct; panel confirmed balanced and never-treated pool shown to bracket the treated covariates; a hand 2×2 and its identity with two-period TWFE; the naive number explained as upward bias from common time movement, not selection-on-winners. Structure and estimands right, but the flagship rationale is thin or the earliest wave is mistaken for the flagship; 2×2 computed but the TWFE-equivalence not shown. Partial structure; estimands not distinct; or the flagship folded into the DiD waves. No derivation.
B — TWFE, Bacon, bad control 12% Static + dynamic TWFE on log(footfall), market-clustered, with an event-study plot; Goodman–Bacon reported with the forbidden-comparison weight, used to argue whether staggering is the disease here; staffed_lanes shown to be caused by treatment, diagnosed as a post-treatment mediator, correctly excluded, and the over-control bias named. Ladder + Bacon built, but the “is staggering the whole story” reading is thin, or the bad-control mechanism/naming is loose. TWFE reported as if unbiased; or staffed_lanes added as a “control” with no diagnosis. Not addressed.
C — Heterogeneity-robust family 13% CS (never-treated + DR + universal base + market clustering, overall and per-cohort) and BJS, plus SA/Gardner, in a bake-off; states the family roughly agrees yet all sit high, and warns the agreement certifies the staggering fix, not parallel trends; reads the per-cohort pattern as a selection signature; does not average the ladder. Required menu built and clustered, but one estimator or the “shared-bias, not corroboration” reading is under-argued. Two estimators only; or the robust number reported as the portfolio effect with no caveat that PT is untested. No heterogeneity-robust estimator.
D — PT test I + Essay 1 15% Universal-base event study that surfaces the drifting pre-trend (deviation + drift correlation quantified); a power read on the pre-test; HonestDiD breakdown \(\bar M^{*}\) read as fragility, not a pass; Essay 1 nails the level-vs-trend contrast (unit FE absorbs the level, not the slope) and connects the drift to a plausible ops selection story, grounded in the measured numbers. Event study + violation found and a breakdown value, but the why (Essay 1) is qualitative or the power read is missing. Pooled/differenced pre-trend only; violation missed or unexplained; HonestDiD read as a pass. No PT diagnostic.
E — Single-flagship SC 15% Classic SC with pre-RMSPE + donor weights; placebo-in-space (as a rank) and an scpi band for honest inference; augmented-SC (and optionally synthetic-DiD) contrast; a defensible flagship effect with a band, plus an honest note on weight non-uniqueness / to-the-digit agreement. SC + one inference device + a point effect, but the honest band or the ASCM/robustness contrast is thin. Point SC with no inference; or the donor pool mis-specified (treated or flagship markets left in). No synthetic control.
F — Factor imputation + Essay 2 20% IFEct vs. MC raced under CV with the pick measured across seeds (not fished); dynamic-gap plots; factors extracted, signs pinned to a proxy, and named from observable series; the time-FE “factor” trap avoided; Essay 2 convicts the guilty factor and acquits the innocent one using the loading-vs-go-live correlation for both, with a business translation of the rollout selection. Factors named and the guilty one identified, but the naming leans on assertion over correlation, or the seed-stability / sign-pinning discipline is skipped. Factors named from the DGP rather than the data; wrong factor convicted; the time FE read as a real factor; or the CV rank hunted rather than reported. No factor/imputation estimator.
G — PT test II 8% Placebo on both \(r=0\) and the factor rank (fires additively, passes with factors); relative-magnitudes \(\bar M^{*}\) reported as a pair with its ruler and not compared like-for-like to Part D; the pinned-ceiling / stale-LOO-object traps avoided; treats every \(p\)-value as a price, not a gate. Two of the three tests run and read correctly, but a bare \(\bar M^{*}\), a like-for-like comparison to Part D, or the stale LOO number slips in. One test only, or any \(p\)-value read as a pass/fail gate. No second PT diagnostic.
H — Heterogeneity, bake-off, memo + Essay 3 7% Group heterogeneity with a common-horizon check and a per-group placebo audit; an estimand-annotated bake-off ranked by assumption credibility that is a real ledger; the estimand-driven lens choice made explicit; a one-page memo with both effects (number + counterfactual + caveat), an honest single-unit verdict, and an actionable recommendation; Essay 3 separates the artefact channels and states what would convince. Memo + reconciliation present, but the ledger or the method-choice logic is thin, the two effects blur, or Essay 3 misses an artefact channel. Numbers restated with no estimand interpretation or recommendation. No memo.

In addition to part-specific scoring, the general evaluation criteria apply (correctness, code quality, communication clarity, reproducibility, and AI-use disclosure per course policy — see the syllabus).

NoteLanguage parity — R or Python, both earn full marks

Either language earns full credit. The following are verified equivalent and accepted interchangeably: staggered DiD — R did / Python differences; event study & TWFE — R fixest / Python pyfixest; imputation — R didimputation / a hand-rolled pyfixest “fit-white, predict-grey”; doubly-robust staggered DiD — Python DoubleMLDIDMulti; synthetic control — R tidysynth / Python pysyncon (Synth); augmented SC — R augsynth / Python pysyncon.AugSynth; honest SC bands — R scpi / Python scpi_pkg.

A few tools are R-first with no verified Python equivalent: bacondecomp (Goodman–Bacon), HonestDiD, synthdid, fect / fect_sens (IFEct, matrix completion, and the leave-one-out and relative-magnitudes diagnostics of Parts F–G), and gsynth. Where one of these diagnostics is only available in R, Python submissions may describe it or use the nearest available substitute with no penalty — you will not lose marks for reasoning about a diagnostic you cannot run in Python. Do not attempt a multi-treated-unit “multisynth” — it is out of scope here.

Submission

Content

  • A written report of 10–12 pages (selectively displayed code, figures, references), structured per the eight parts above, including the three judgement essays and the board memo.
  • Annotated, reproducible code (R or Python). The instructor must be able to open your source file, install the listed packages, and re-render your report end-to-end on a clean machine. Cache the slow fect fits so a re-render does not take 20 minutes.

Format

  • Submit one source file plus the rendered PDF. The source file may be a Quarto Markdown (.qmd) or a Jupyter notebook (.ipynb) — both accepted, in either R or Python.
  • File names: [YourLastName]_Assignment_5.{qmd|ipynb} and [YourLastName]_Assignment_5.pdf.
  • Bundle the two files into a single zip: [YourLastName]_Assignment_5.zip.
  • Do not include the CSVs in the zip. When grading, the instructor places the two provided CSVs in the same folder as your source file before re-rendering; read them with plain relative paths (fread("grocery_panel.csv")), not parent-directory paths.
  • If your code reads from any additional external file you created (e.g. a cached fit or a saved weights table), bundle it alongside the source file — but never the provided datasets.

Deadline

  • Sunday, 2 August 2026, 23:59 (Hamburg local time).
  • Upload to the StudIP folder (link in the course shell).
WarningIf something breaks

If you cannot complete a part due to a technical problem, submit anyway. Document what you tried, what failed, and what you would have done. Partial credit is awarded generously when the reasoning is shown — and in this assignment the reasoning is most of the grade.