The Trouble with Instruments: The Need for Pre-Treatment Balance in Shock-IV Designs

69 Pages Posted: 12 Apr 2015 Last revised: 18 Apr 2019

Date Written: April 16, 2019

Abstract

Credible causal inference in accounting and finance research often comes from “natural” experiments. These experiments can be exploited using several “shock-based” research designs, including difference-in-differences (DiD), instrumental variables based on the shock (shock-IV), and regression discontinuity (RD). We study here shock-IV designs using panel data. Shock-IVs are potentially more credible than non-shock IVs, but can be problematic unless carefully handled. We identify all shock-IV papers in two broad datasets; and re-examine three of the apparently strongest papers – Duchin, Matsusaka and Ozbas (“DMO,” JFE 2010); Iliev (JF 2010); and Desai and Dharmapala (REStat 2009). We show that the IVs in all three papers are unusably weak – once we enforce covariate balance and common support for treated and control firms, the instruments are no longer significant in the first stage. All three papers also show non-parallel pre-treatment trends on outcomes or core covariates. The problems with these papers generalize to our full sample, and to other papers exploiting the same shocks as DMO. A core conclusion of our reexamination is that “pre-treatment balance” (common support, covariate balance, and parallel pre-treatment trends) is necessary for credible shock-IV designs. We provide a good practice checklist for shock-IV design with panel data. Finding valid, usable shock-IVs is more challenging than prior research suggests.

The Online Appendix is available at: http://ssrn.com/abstract=2859113.

Keywords: Instrumental variables; shock-based research design; exclusion restriction; covariate balance

JEL Classification: C26, G34, G38

Suggested Citation

Atanasov, Vladimir A. and Black, Bernard S., The Trouble with Instruments: The Need for Pre-Treatment Balance in Shock-IV Designs (April 16, 2019). ECGI - Finance Working Paper; Northwestern Law & Econ Research Paper 16-01; Northwestern Public Law Research Paper No. 15-26. Available at SSRN: https://ssrn.com/abstract=2417689 or http://dx.doi.org/10.2139/ssrn.2417689

Vladimir A. Atanasov

William and Mary - Raymond A. Mason School of Business ( email )

P.O. Box 8795
Williamsburg, VA 23187-8795
United States

Bernard S. Black (Contact Author)

Northwestern University - Pritzker School of Law ( email )

375 E. Chicago Ave
Chicago, IL 60611
United States
312-503-2784 (Phone)

Northwestern University - Kellogg School of Management

2001 Sheridan Road
Evanston, IL 60208
United States
847-491-5049 (Phone)

European Corporate Governance Institute (ECGI)

Brussels
Belgium

Register to save articles to
your library

Register

Paper statistics

Downloads
1,036
rank
19,918
Abstract Views
5,726
PlumX Metrics