Rules and Discretion in the Evaluation of Students and Schools: The Case of the New York Regents Examinations

42 Pages Posted: 24 Aug 2011

See all articles by Thomas S. Dee

Thomas S. Dee

Stanford University - School of Education; National Bureau of Economic Research (NBER)

Brian A. Jacob

University of Michigan at Ann Arbor

Jonah E. Rockoff

Columbia University - Columbia Business School, Finance; National Bureau of Economic Research (NBER)

Justin McCrary

Columbia University - Law School; National Bureau of Economic Research (NBER)

Date Written: February 21, 2011

Abstract

The challenge of designing effective performance measurement and incentives is a general one in economic settings where behavior and outcomes are not easily observable. These issues are particularly prominent in education where, over the last two decades, test-based accountability systems for schools and students have proliferated. In this study, we present evidence that the design and decentralized, school-based scoring of New York’s high-stakes Regents Examinations have led to pervasive manipulation of student test scores that are just below performance thresholds. Specifically, we document statistically significant discontinuities in the distributions of subject-specific Regent scores that align with the cut scores used to determine both student eligibility to graduate and school accountability. Our results suggest that roughly 3 to 5 percent of the exam scores that qualified for a high school diploma actually had performance below the state requirements. Using multiple sources of data, we present evidence that score manipulation is driven by local teachers’ desire to help their students avoid sanctions associated with failure to meet exam standards, not the recent creation of school accountability systems. We also provide some evidence that variation in the extent of manipulation across schools tends to favor traditionally disadvantaged student groups.

Suggested Citation

Dee, Thomas S. and Jacob, Brian A. and Rockoff, Jonah E. and McCrary, Justin, Rules and Discretion in the Evaluation of Students and Schools: The Case of the New York Regents Examinations (February 21, 2011). Columbia Business School Research Paper, Available at SSRN: https://ssrn.com/abstract=1915387 or http://dx.doi.org/10.2139/ssrn.1915387

Thomas S. Dee

Stanford University - School of Education ( email )

Stanford, CA 94305-3096
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Brian A. Jacob

University of Michigan at Ann Arbor ( email )

500 S. State Street
Ann Arbor, MI 48109
United States

Jonah E. Rockoff (Contact Author)

Columbia University - Columbia Business School, Finance ( email )

3022 Broadway
New York, NY 10027
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Justin McCrary

Columbia University - Law School ( email )

435 West 116th Street
New York, NY 10025
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
254
Abstract Views
1,914
Rank
218,031
PlumX Metrics