The Promise and Pitfalls of Differences-in-Differences: Reflections on ‘16 and Pregnant’ and Other Applications

22 Pages Posted: 1 Aug 2018

See all articles by Ariella Kahn-Lang

Ariella Kahn-Lang

Harvard University

Kevin Lang

Boston University - Department of Economics; National Bureau of Economic Research (NBER)

Date Written: July 2018

Abstract

We use the exchange between Kearney/Levine and Jaeger/Joyce/Kaestner on “16 and Pregnant” to reexamine the use of DiD as a response to the failure of nature to properly design an experiment for us. We argue that 1) any DiD paper should address why the original levels of the experimental and control groups differed, and why this would not impact trends, 2) the parallel trends argument requires a justification of the chosen functional form and that the use of the interaction coefficients in probit and logit may be justified in some cases, and 3) parallel trends in the period prior to treatment is suggestive of counterfactual parallel trends, but parallel pre-trends is neither necessary nor sufficient for the parallel counterfactual trends condition to hold. Importantly, the purely statistical approach uses pretesting and thus generates the wrong standard errors. Moreover, we underline the dangers of implicitly or explicitly accepting the null hypothesis when failing to reject the absence of a differential pre-trend.

Suggested Citation

Kahn-Lang, Ariella and Lang, Kevin, The Promise and Pitfalls of Differences-in-Differences: Reflections on ‘16 and Pregnant’ and Other Applications (July 2018). NBER Working Paper No. w24857, Available at SSRN: https://ssrn.com/abstract=3218109

Kevin Lang

Boston University - Department of Economics ( email )

270 Bay State Road
Boston, MA 02215
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
28
Abstract Views
191
PlumX Metrics