A Seven-College Experiment Using Algorithms to Track Students: Impacts and Implications for Equity and Fairness

Abstract

Tracking is widespread in education. In U.S. post-secondary education alone, at least 71% of colleges use a test to track students into various courses. However, there are concerns that placement tests lack validity and unnecessarily reduce education opportunities for students from under-represented groups. While research has shown that tracking can improve student learning, inaccurate placement has consequences: students face misaligned curricula and must pay tuition for remedial courses that do not bear credits toward graduation. We develop an alternative system that uses algorithms to predict college readiness and track students into courses. Compared to the most widely-used placement tests in the country, the algorithms are more predictive of future performance. We conduct an experiment across seven colleges to evaluate the effects of algorithmic placement. Placement rates into college-level courses increase substantially without reducing pass rates. Algorithmic placement generally, though not always, narrows differences in college placement rates and remedial course taking across demographic groups. We use the experimental design and variation in placement rates to assess the disparate impact of each system. Test scores exhibit substantially more discrimination than algorithms; a significant share of test-score disparities between Hispanic or Black students and white students is explained by discrimination. We also show that the selective labels problem nearly doubles the prediction error for college English performance but has almost no impact on the prediction error for college math performance. A detailed cost analysis shows that algorithmic placement is socially efficient: it increases college credits earned while saving costs for students and the government.