Logistic Regression in Rare Events Data

Political Analysis, Vol. 9, No. 2, pp. 137-163, Spring, 2001

27 Pages Posted: 17 Jan 2008

See all articles by Gary King

Gary King

Harvard University

Langche Zeng

University of California, San Diego

Abstract

We study rare events data, binary dependent variables with dozens to thousands of times fewer ones (events, such as wars, vetoes, cases of political activism, or epidemiological infections) than zeros ("nonevents"). In many literatures, these variables have proven difficult to explain and predict, a problem that seems to have at least two sources. First, popular statistical procedures, such as logistic regression, can sharply underestimate the probability of rare events. We recommend corrections that outperform existing methods and change the estimates of absolute and relative risks by as much as some estimated effects reported in the literature. Second, commonly used data collection strategies are grossly inefficient for rare events data. The fear of collecting data with too few events has led to data collections with huge numbers of observations but relatively few, and poorly measured, explanatory variables, such as in international conflict data with more than a quarter-million dyads, only a few of which are at war. As it turns out, more efficient sampling designs exist for making valid inferences, such as sampling all variable events (e.g., wars) and a tiny fraction of nonevents (peace). This enables scholars to save as much as 99% of their (nonfixed) data collection costs or to collect much more meaningful explanatory variables. We provide methods that link these two results, enabling both types of corrections to work simultaneously, and software that implements the methods developed.

Suggested Citation

King, Gary and Zeng, Langche, Logistic Regression in Rare Events Data. Political Analysis, Vol. 9, No. 2, pp. 137-163, Spring, 2001 , Available at SSRN: https://ssrn.com/abstract=1083726

Gary King (Contact Author)

Harvard University ( email )

1737 Cambridge St.
Institute for Quantitative Social Science
Cambridge, MA 02138
United States
617-500-7570 (Phone)

HOME PAGE: http://gking.harvard.edu

Langche Zeng

University of California, San Diego ( email )

9500 Gilman Drive
Code 0521
La Jolla, CA 92093-0521
United States