Reusing Natural Experiments
Charles A. Dice Center Working Paper No. 2019-21
56 Pages Posted: 21 Sep 2019
Date Written: September 20, 2019
Natural experiments are used in empirical research to make causal inferences. After a natural experiment is first used, other researchers often reuse the setting, examining different outcomes based on causal chain arguments. Using simulation evidence combined with two extensively studied natural experiments, business combination laws and the Regulation SHO pilot, we show that the repeated use of a natural experiment significantly increases the likelihood of false discoveries. To correct this, we propose multiple testing methods which account for dependence across tests and we show evidence of their efficacy.
Keywords: False Positive, Identification, Multiple Hypothesis Testing, Natural Experiments
JEL Classification: G1, G10
Suggested Citation: Suggested Citation