The Dangers of Extreme Counterfactuals

Political Analysis, Vol. 14, No. 2, pp. 131-159, 2006

29 Pages Posted: 16 Jun 2008

See all articles by Gary King

Gary King

Harvard University

Langche Zeng

University of California, San Diego

Abstract

We address the problem that occurs when inferences about counterfactuals -- predictions, "what if" questions, and causal effects -- are attempted far from the available data. The danger of these extreme counterfactuals is that substantive conclusions drawn from statistical models that fit the data well turn out to be based largely on speculation hidden in convenient modeling assumptions that few would be willing to defend. Yet existing statistical strategies provide few reliable means of identifying extreme counterfactuals. We offer a proof that inferences farther from the data are more model-dependent, and then develop easy-to-apply methods to evaluate how model-dependent our answers would be to specified counterfactuals. These methods require neither sensitivity testing over specified classes of models nor evaluating any specific modeling assumptions. If an analysis fails the simple tests we offer, then we know that substantive results are sensitive to at least some modeling choices that are not based on empirical evidence.

Suggested Citation

King, Gary and Zeng, Langche, The Dangers of Extreme Counterfactuals. Available at SSRN: https://ssrn.com/abstract=1082033

Gary King (Contact Author)

Harvard University ( email )

1737 Cambridge St.
Institute for Quantitative Social Science
Cambridge, MA 02138
United States
617-500-7570 (Phone)

HOME PAGE: http://gking.harvard.edu

Langche Zeng

University of California, San Diego ( email )

9500 Gilman Drive
Code 0521
La Jolla, CA 92093-0521
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
25
Abstract Views
435
PlumX Metrics