Achievement Gap Estimates and Deviations from Cardinal Comparability

FEDS Working Paper No. 2015-040

54 Pages Posted: 28 Jun 2015

See all articles by Eric Reed Nielsen

Eric Reed Nielsen

Board of Governors of the Federal Reserve System

Multiple version iconThere are 2 versions of this paper

Date Written: May 12, 2015


This paper assesses the sensitivity of standard empirical methods for measuring group differences in achievement to violations in the cardinal comparability of achievement test scores. The paper defines a distance measure over possible weighting functions (scalings) of test scores. It then constructs worst-case bounds for the bias in the estimated achievement gap (or achievement gap change) that could result from using the observed rather than the true test scale, given that the true and observed scales are no more than a fixed distance from each other. The worst-case weighting functions have simple, closed-form expressions consisting of achievement thresholds, flat regions in which test scores are uninformative, and regions in which the observed test scores are actually cardinally comparable. The paper next estimates these worst-case weighting functions for black/white and high-/low-income achievement gaps and gap changes using data from several commonly employed surveys. The results of this empirical exercise suggest that cross-sectional achievement gap estimates tend to be quite robust to scale misspecification. In contrast, achievement gap change estimates seem to be quite sensitive to the choice of test scale. Standard empirical methods may not robustly identify the sign of the trend in achievement inequality between students from different racial groups and income classes. Furthermore, ordinal methods may be more powerful and will continue to have the correct size when the test scale has been misspecified.

Keywords: Achievement gaps, Econometrics, Health, education, welfare, Inequalty, Measurement, Robustness

JEL Classification: C18, I24, I26

Suggested Citation

Nielsen, Eric Reed, Achievement Gap Estimates and Deviations from Cardinal Comparability (May 12, 2015). Available at SSRN: or

Eric Reed Nielsen (Contact Author)

Board of Governors of the Federal Reserve System ( email )

20th Street and Constitution Avenue NW
Washington, DC 20551
United States

Register to save articles to
your library


Paper statistics

Abstract Views
PlumX Metrics