Comparing Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment

Posted: 10 Jan 2009

See all articles by Kevin Arceneaux

Kevin Arceneaux

Temple University - Department of Political Science

Alan Gerber

Yale University - Department of Political Science; Yale University - Cowles Foundation

Donald P. Green

Columbia University

Abstract

In the social sciences, randomized experimentation is the optimal research design for establishing causation. However, for a number of practical reasons, researchers are sometimes unable to conduct experiments and must rely on observational data. In an effort to develop estimators that can approximate experimental results using observational data, scholars have given increasing attention to matching. In this article, we test the performance of matching by gauging the success with which matching approximates experimental results. The voter mobilization experiment presented here comprises a large number of observations (60,000 randomly assigned to the treatment group and nearly two million assigned to the control group) and a rich set of covariates. This study is analyzed in two ways. The first method, instrumental variables estimation, takes advantage of random assignment in order to produce consistent estimates. The second method, matching estimation, ignores random assignment and analyzes the data as though they were nonexperimental. Matching is found to produce biased results in this application because even a rich set of covariates is insufficient to control for preexisting differences between the treatment and control group. Matching, in fact, produces estimates that are no more accurate than those generated by ordinary least squares regression. The experimental findings show that brief paid get-out-the-vote phone calls do not increase turnout, while matching and regression show a large and significant effect.

Keywords: Field Experiments, Randomization, Matching, Methodology, GOTV, Voter mobilization, Voter Turnout

JEL Classification: C93

Suggested Citation

Arceneaux, Kevin and Gerber, Alan and Green, Donald P., Comparing Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment. Political Analysis, Vol. 14, pp. 37-62, 2006. Available at SSRN: https://ssrn.com/abstract=1325267

Kevin Arceneaux (Contact Author)

Temple University - Department of Political Science ( email )

Philadelphia, PA 19122
United States

Alan Gerber

Yale University - Department of Political Science ( email )

Box 208269
New Haven, DC 06520-8269
United States
203-432-5232 (Phone)

Yale University - Cowles Foundation

Box 208281
New Haven, CT 06520-8281
United States

Donald P. Green

Columbia University ( email )

7th Floor, International Affairs Bldg.
420 W. 118th Street
New York, NY 10027
United States

Register to save articles to
your library

Register

Paper statistics

Abstract Views
276
PlumX Metrics