Evaluating School Vouchers: Evidence from a Within-Study Comparison

56 Pages Posted: 14 Apr 2017 Last revised: 17 May 2017

See all articles by Kaitlin Anderson

Kaitlin Anderson

Michigan State University

Patrick Wolf

University of Arkansas - Department of Education Reform

Date Written: April 3, 2017


Randomized Controlled Trials (RCTs) are the “gold-standard” for estimating causal impacts of educational programs. Students subject to lotteries, however, often are not representative of the broader population of students experiencing the educational treatment. With few exceptions, researchers are not able to determine how much selection bias exists when various quasi-experimental approaches are used in place of experimental ones within a school choice context. We are left wondering about the magnitude of the internal-for-external validity tradeoff that education researchers often face.

This study assesses the extent to which methods such as propensity score matching or observational models with control variables can replicate the “benchmark” experimental results of the District of Columbia Opportunity Scholarship (DC OSP) school voucher evaluation. The federal private school voucher program is an exemplar subject for study because self-selection is assumed to be a major influence on whether or not a low-income urban student attends a private school. We treat Instrumental Variables Analysis (IV) estimates of the impact of private schooling on student outcomes, some of which are being presented for the first time in this study, as the causal “benchmark” estimate. While our data are fairly limited, and the results relatively imprecise, we find preliminary evidence that covariate choice matters, and that method choice matters, but perhaps only when comparing to a broader sample that includes students who did not apply to the program.

Interestingly, we find that the direction of the estimation bias that we detect from some of the quasi-experimental approaches is positive when the sample is limited to program applicants, but negative when it is expanded to include non-applicants. This finding suggests that the applicants to means-tested school voucher programs are negatively selected, but the subgroup of applicants who actually use a voucher if offered tend to be positively selected.

Keywords: school vouchers, school choice, within-study comparison, randomized controlled trial, quasi-experimental design, internal validity, external validity, selection bias

Suggested Citation

Anderson, Kaitlin and Wolf, Patrick, Evaluating School Vouchers: Evidence from a Within-Study Comparison (April 3, 2017). EDRE Working Paper No. 2017-10, Available at SSRN: https://ssrn.com/abstract=2952967 or http://dx.doi.org/10.2139/ssrn.2952967

Kaitlin Anderson (Contact Author)

Michigan State University ( email )

201H Erickson Hall
620 Farm Lane
East Lansing, MI 48933
United States

Patrick Wolf

University of Arkansas - Department of Education Reform ( email )

201 Graduate Education Building
Fayetteville, AR 72701
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
PlumX Metrics