Reusing Natural Experiments

68 Pages Posted: 21 Sep 2019 Last revised: 1 May 2020

See all articles by Davidson Heath

Davidson Heath

University of Utah - David Eccles School of Business

Matthew Ringgenberg

University of Utah - Department of Finance

Mehrdad Samadi

Southern Methodist University (SMU) - Finance Department

Ingrid M. Werner

The Ohio State University - Fisher College of Business; CEPR

Date Written: April 30, 2020

Abstract

After a natural experiment is first used, other researchers often reuse the setting, examining different outcome variables. We examine the consequences of reusing an experimental setting using two extensively studied natural experiments, business combination laws and the Regulation SHO pilot. We apply multiple hypothesis corrections and our findings suggest many results in the existing literature are false positives. We provide guidelines for inference when an experiment is reused using simulation evidence for several popular empirical settings including difference-in-differences regressions, instrumental variables regressions, and regression discontinuity designs.

Keywords: False Positive, Identification, Multiple Hypothesis Testing, Natural Experiments

JEL Classification: G1, G10

Suggested Citation

Heath, Davidson and Ringgenberg, Matthew C. and Samadi, Mehrdad and Werner, Ingrid M., Reusing Natural Experiments (April 30, 2020). Fisher College of Business Working Paper No. 2019-03-021, Charles A. Dice Center Working Paper No. 2019-21, SMU Cox School of Business Research Paper No. 19-11, Available at SSRN: https://ssrn.com/abstract=3457525 or http://dx.doi.org/10.2139/ssrn.3457525

Davidson Heath

University of Utah - David Eccles School of Business ( email )

1645 E Campus Center Dr
Salt Lake City, UT 84112-9303
United States

Matthew C. Ringgenberg

University of Utah - Department of Finance ( email )

David Eccles School of Business
Salt Lake City, UT 84112
United States

Mehrdad Samadi

Southern Methodist University (SMU) - Finance Department ( email )

United States

Ingrid M. Werner (Contact Author)

The Ohio State University - Fisher College of Business ( email )

2100 Neil Avenue
Columbus, OH 43210-1144
United States
614-292-6460 (Phone)
614-292-2418 (Fax)

CEPR ( email )

London
United Kingdom

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
841
Abstract Views
5,559
rank
31,366
PlumX Metrics