Rules and Discretion in the Evaluation of Students and Schools: The Case of the New York Regents Examinations

42 Pages Posted: 24 Aug 2011

See all articles by Thomas S. Dee

Thomas S. Dee

Stanford University - School of Education; National Bureau of Economic Research (NBER)

Brian A. Jacob

University of Michigan at Ann Arbor

Jonah E. Rockoff

Columbia Business School - Finance and Economics; National Bureau of Economic Research (NBER)

Justin McCrary

Columbia University - Law School; National Bureau of Economic Research (NBER)

Date Written: February 21, 2011

Abstract

The challenge of designing effective performance measurement and incentives is a general one in economic settings where behavior and outcomes are not easily observable. These issues are particularly prominent in education where, over the last two decades, test-based accountability systems for schools and students have proliferated. In this study, we present evidence that the design and decentralized, school-based scoring of New York’s high-stakes Regents Examinations have led to pervasive manipulation of student test scores that are just below performance thresholds. Specifically, we document statistically significant discontinuities in the distributions of subject-specific Regent scores that align with the cut scores used to determine both student eligibility to graduate and school accountability. Our results suggest that roughly 3 to 5 percent of the exam scores that qualified for a high school diploma actually had performance below the state requirements. Using multiple sources of data, we present evidence that score manipulation is driven by local teachers’ desire to help their students avoid sanctions associated with failure to meet exam standards, not the recent creation of school accountability systems. We also provide some evidence that variation in the extent of manipulation across schools tends to favor traditionally disadvantaged student groups.

Suggested Citation

Dee, Thomas S. and Jacob, Brian A. and Rockoff, Jonah E. and McCrary, Justin, Rules and Discretion in the Evaluation of Students and Schools: The Case of the New York Regents Examinations (February 21, 2011). Columbia Business School Research Paper. Available at SSRN: https://ssrn.com/abstract=1915387 or http://dx.doi.org/10.2139/ssrn.1915387

Thomas S. Dee

Stanford University - School of Education ( email )

Stanford, CA 94305-3096
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Brian A. Jacob

University of Michigan at Ann Arbor ( email )

500 S. State Street
Ann Arbor, MI 48109
United States

Jonah E. Rockoff (Contact Author)

Columbia Business School - Finance and Economics ( email )

3022 Broadway
New York, NY 10027
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Justin McCrary

Columbia University - Law School ( email )

435 West 116th Street
New York, NY 10025
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
148
Abstract Views
1,016
rank
202,038
PlumX Metrics