Modern Panel Data Methods For Causal Inference

Abstract

In the early 21st century, social scientists very often rely on panel data for causal inference. In particular, two‐way fixed effects (TWFE) regressions, where an outcome is regressed on period and location fixed effects and a treatment variable, are ubiquitous. Of the 100 most cited papers published by the American Economic Review from 2015 to 2019, 26 of them use a TWFE regression to measure the effect of a treatment on an outcome. TWFE regressions are also very commonly used by sociologists, political scientists, epidemiologists, and researchers in environmental sciences. The success of TWFE regressions probably originates from the fact that for a long time, researchers have believed that TWFE estimators only rely on a parallel trends assumption, requiring that in the absence of the treatment, the outcome would have followed the same evolution over time in treated and control locations. The parallel trends assumption can be partly tested, by comparing the outcome evolutions in treated and control locations prior to the treatment. Thus, until recently researchers thought that to confirm the validity of a TWFE design, they just had to show that a pre-trends test was not rejected.

However, two recent strands of literature have shattered that consensus. A first literature strand has shown that TWFE regressions do not only rely on a parallel trends assumption, but also rely on the assumption that the treatment’s effect does not vary across time and space, an often implausible assumption. A second literature strand has shown that tests of the parallel trends assumption often lack statistical power, and may fail to detect differential trends between treated and control locations that are often large enough to account for a significant share of the treatment’s estimated effect. These two strands of literature call for more robust methods than TWFE regressions for causal inference using panel data.

Our textbook “Modern Panel Data Methods For Causal Inference” offers a pedagogical but thorough overview of the main modern panel data methods used in the social sciences for causal inference. Unlike TWFE regressions, a one-size-fits-all method which can be used irrespective of the study design, the more-robust methods we describe are tailored to specific designs, so the book’s chapters cover specific study designs, from the simplest to the most complex . After presenting the set-up (Chapter 2) and issues with TWFE regressions (Chapter 3), we review in Chapter 4 the basic difference-in-differences (DID) design with a binary treatment and no variation in treatment timing. In this case only, the TWFE and DID estimator coincide, and treatment effects can be identified under a parallel trends assumption. We conclude Chapter 4 by stressing the recent literature that has shown that tests of parallel trends are often underpowered. This leads us naturally to chapter 5, where we still consider the basic design with a binary treatment and no variation with treatment timing, and consider various estimators that rely on weaker assumptions than parallel trends: combinations of matching and DID estimators relying on a conditional parallel trends assumption (Section 5.1); estimators relying on an approximate parallel trends assumption à la Rambachan and Roth (Section 5.2); synthetic controls and interactive fixed effects estimators (Section 5.3), and estimators with grouped patterns of heterogeneity à la Bonhomme and Manresa (Section 5.4). In chapter 6, we turn to binary and staggered treatments, with variation in treatment timing. We review estimators that have been proposed under parallel trends or conditional parallel trends assumptions in that design (e.g. that of Callaway and Sant’Anna, or that of Borusyak et al.), as well as generalizations of the synthetic control estimator. In chapter 7, we consider heterogeneous adoption designs with two time periods, where all locations are untreated in the first period and receive heterogeneous treatment doses in the second period, and describe estimators more robust than TWFE regressions: in designs with stayers (locations with a treatment dose equal to zero in the second period); in designs with quasi-stayers (locations with a treatment dose close to zero in the second period); and in designs without stayers or quasi-stayers. In chapter 8, we consider general designs, where the treatment and its path are left unrestricted, and describe estimators that build upon the insights from the previous chapters and can be used if the treatment does not have dynamic effects (the contemporaneous outcome only depends on the contemporaneous treatment, not on past treatments). In chapter 9, we consider again general designs, and propose estimators that can be used if the treatment has dynamic effects. Chapters 10, 11, and 12 consider other common designs: designs with several treatments (chapter 10), sequential randomized experiments (chapter 11), and panel Bartik regressions (chapter 12).

This practically‐minded book also introduces the reader to the main software tools computing the reviewed estimators. The methods are illustrated by revisiting several published articles among the most cited articles published in economics over the last two decades. For instance, the seminal American Economic Review (AER, 2013) paper by Autor et al. on the effect of imports from China on US manufacturing employment is revisited in Chapter 12. Six other seminal papers published in the AER and other prestigious journals are revisited in chapters 3, 4, 6, 7, 8, and 9. All the necessary code to reproduce our replications will be made available to the readers.