Are Difference-in-Differences Estimates Blowing Smoke? A Cautionary Tale From an Oft-Used Natural Experiment

Abstract

We illustrate the sensitivity of two-way fixed effects difference-indifferences estimates to innocuous changes in data structure. Using the staggered rollout of state-level bank branching deregulations, three outcome variables are brought to bear on the interventions: personal income growth (a replication), house prices (new to the literature), and per capita cigarette consumption (a falsification test). Estimates are sensitive to panel length, and we document that the data structure creates the false impression of a causal effect of the interventions on all three outcome variables. We contend that any test using this particular natural experiment is at risk of generating spurious results.

Keywords: banking, deregulation, economic growth, difference-indifferences, two-way fixed effect estimation

JEL Classification: C13, C18, C23, G21, G28, O47