Does Regression Produce Representative Estimates of Causal Effects?
Peter M. Aronow
Yale University - Department of Political Science
New York University (NYU) - Wilf Family Department of Politics
February 2, 2015
EPSA 2013 Annual General Conference Paper 585
With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The "effective sample'' that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the "multiple regression weights'' that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.
Number of Pages in PDF File: 41
Keywords: causal inference; external validity; multiple regression; observational studies; randomized experiments
Date posted: July 8, 2013 ; Last revised: February 11, 2015