A Cautionary Note on the Use of Matching to Estimate Causal Effects: An Empirical Example Comparing Matching Estimates to an Experimental Benchmark

Sociological Methods & Research, Vol. 39, No. 2, pp. 256-282, 2010

Posted: 7 Jan 2011

See all articles by Kevin Arceneaux

Kevin Arceneaux

Temple University - Department of Political Science

Alan Gerber

Yale University - Department of Political Science; Yale University - Cowles Foundation

Donald P. Green

Columbia University

Date Written: 2010

Abstract

In recent years, social scientists have increasingly turned to matching as a method for drawing causal inferences from observational data. Matching compares those who receive a treatment to those with similar background attributes who do not receive a treatment. Researchers who use matching frequently tout its ability to reduce bias, particularly when applied to data sets that contain extensive background information. Drawing on a randomized voter mobilization experiment, we compare estimates generated by matching to an experimental benchmark. The enormous sample size enables us to exactly match each treated subject to forty untreated subjects. Matching greatly exaggerates the effectiveness of pre-election phone calls encouraging voter participation. Moreover, it can produce nonsensical results: matching suggests that another pre-election phone call that encouraged people to wear their seat belts also generated huge increases in voter turnout. This illustration suggests that caution is warranted when applying matching estimators to observational data, particularly when one is uncertain about the potential for biased inference.

Keywords: Causal inference, Matching, Voter mobilization, Bias

JEL Classification: C93, C00

Suggested Citation

Arceneaux, Kevin and Gerber, Alan and Green, Donald P., A Cautionary Note on the Use of Matching to Estimate Causal Effects: An Empirical Example Comparing Matching Estimates to an Experimental Benchmark (2010). Sociological Methods & Research, Vol. 39, No. 2, pp. 256-282, 2010. Available at SSRN: https://ssrn.com/abstract=1734761

Kevin Arceneaux (Contact Author)

Temple University - Department of Political Science ( email )

Philadelphia, PA 19122
United States

Alan Gerber

Yale University - Department of Political Science ( email )

Box 208269
New Haven, DC 06520-8269
United States
203-432-5232 (Phone)

Yale University - Cowles Foundation

Box 208281
New Haven, CT 06520-8281
United States

Donald P. Green

Columbia University ( email )

7th Floor, International Affairs Bldg.
420 W. 118th Street
New York, NY 10027
United States

Register to save articles to
your library

Register

Paper statistics

Abstract Views
331
PlumX Metrics