Download this Paper Open PDF in Browser

Data vs. Methods: Quasi-Experimental Evaluation of Alternative Sample Selection Corrections for Missing College Entrance Exam Score Data

96 Pages Posted: 11 Jun 2016  

Robert Garlick

Duke University - Department of Economics

Joshua Hyman

University of Connecticut - Department of Public Policy; University of Connecticut - Neag School of Education

Date Written: June 7, 2016

Abstract

In 2007, Michigan began requiring all high school students to take the ACT college entrance exam. This natural experiment allows us to evaluate the performance of several parametric and semiparametric sample selection correction models. We apply each model to the censored, prepolicy test score data and compare the predicted values to the uncensored, post-policy distribution. We vary the set of model predictors to imitate the varying levels of data detail to which a researcher may have access. We find that predictive performance is sensitive to predictor choice but not correction model choice. All models perform poorly using student demographics and school- and district-level characteristics as predictors. However, all models perform well when including students’ prior and contemporaneous scores on other tests. Similarly, correction models using group-level data perform better with more finely disaggregated groups, but produce similar predictions under different functional form assumptions. Our findings are not explained by an absence of selection, the assumptions of the parametric models holding, or the data lacking sufficient variation to permit useful semiparametric estimation. We conclude that “data beat methods” in this setting: gains from using less restrictive econometric methods are small relative to gains from seeking richer or more disaggregated data.

JEL Classification: J01, I20, C10

Suggested Citation

Garlick, Robert and Hyman, Joshua, Data vs. Methods: Quasi-Experimental Evaluation of Alternative Sample Selection Corrections for Missing College Entrance Exam Score Data (June 7, 2016). Economic Research Initiatives at Duke (ERID) Working Paper No. 221. Available at SSRN: https://ssrn.com/abstract=2793486 or http://dx.doi.org/10.2139/ssrn.2793486

Robert Garlick

Duke University - Department of Economics ( email )

213 Social Sciences Building
Box 90097
Durham, NC 27708-0204
United States

Joshua Hyman (Contact Author)

University of Connecticut - Department of Public Policy ( email )

1800 Asylum Ave.
Library Building, 4th Floor
West Hartford, CT 06117
United States

University of Connecticut - Neag School of Education ( email )

United States

Paper statistics

Downloads
29
Abstract Views
222