How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice based on Over 60 Replicated Studies

159 Pages Posted: 16 Aug 2021 Last revised: 27 Dec 2021

See all articles by Apoorva Lal

Apoorva Lal

Stanford University

Mackenzie William Lockhart

affiliation not provided to SSRN

Yiqing Xu

Stanford University

Ziwen Zu

University of California, San Diego

Date Written: December 21, 2021

Abstract

Instrumental variable (IV) strategies are commonly used in political science to establish causal relationships, yet the identifying assumptions required by an IV design are demanding and it remains challenging for researchers to evaluate their plausibility. We replicate 61 papers published in three top journals in political science from the past decade (2010-2020) and document several troubling patterns: (1) researchers often miscalculate the first-stage F statistics, overestimating the strength of their IVs; (2) most researchers rely on classical asymptotic standard errors, which often severely underestimate the uncertainties around the two-stage-least-squares (2SLS) estimates; (3) in the majority of the replicated studies, the 2SLS estimates are much bigger than the ordinary-least-squares estimates, and their ratio is negatively correlated with the strength of the IVs in studies where the IVs are not experimentally generated, suggesting potential violations of the exclusion restriction. To improve practice, we provide a checklist for researchers to avoid these pitfalls and recommend a zero-first-stage test and a local-to-zero procedure to guard against failures of the identifying assumptions.

Keywords: instrumental variables, two-stage-least-squared, replications, weak IV, exclusion restriction, zero-first-stage test

JEL Classification: C26

Suggested Citation

Lal, Apoorva and Lockhart, Mackenzie William and Xu, Yiqing and Zu, Ziwen, How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice based on Over 60 Replicated Studies (December 21, 2021). Available at SSRN: https://ssrn.com/abstract=3905329 or http://dx.doi.org/10.2139/ssrn.3905329

Apoorva Lal

Stanford University ( email )

Stanford, CA 94305
United States

HOME PAGE: http://apoorvalal.github,io

Mackenzie William Lockhart

affiliation not provided to SSRN

Yiqing Xu (Contact Author)

Stanford University ( email )

Stanford, CA 94305
United States

HOME PAGE: http://yiqingxu.org

Ziwen Zu

University of California, San Diego ( email )

CA
United States
3472442058 (Phone)
92092 (Fax)

HOME PAGE: http://www.ziwenzu.com

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
712
Abstract Views
2,137
rank
51,961
PlumX Metrics