Measuring Racial Discrimination in Algorithms

36 Pages Posted: 23 Dec 2020 Last revised: 10 Feb 2025

See all articles by David Arnold

David Arnold

University of California, San Diego (UCSD)

Will Dobbie

Harvard University - Harvard Kennedy School (HKS)

Peter Hull

University of Chicago - Becker Friedman Institute for Economics

Date Written: December 2020

Abstract

There is growing concern that the rise of algorithmic decision-making can lead to discrimination against legally protected groups, but measuring such algorithmic discrimination is often hampered by a fundamental selection challenge. We develop new quasi-experimental tools to overcome this challenge and measure algorithmic discrimination in the setting of pretrial bail decisions. We first show that the selection challenge reduces to the challenge of measuring four moments: the mean latent qualification of white and Black individuals and the race-specific covariance between qualification and the algorithm’s treatment recommendation. We then show how these four moments can be estimated by extrapolating quasi-experimental variation across as-good-as-randomly assigned decision-makers. Estimates from New York City show that a sophisticated machine learning algorithm discriminates against Black defendants, even though defendant race and ethnicity are not included in the training data. The algorithm recommends releasing white defendants before trial at an 8 percentage point (11 percent) higher rate than Black defendants with identical potential for pretrial misconduct, with this unwarranted disparity explaining 77 percent of the observed racial disparity in algorithmic recommendations. We find a similar level of algorithmic discrimination with regression-based recommendations, using a model inspired by a widely used pretrial risk assessment tool.

Suggested Citation

Arnold, David and Dobbie, Will and Hull, Peter, Measuring Racial Discrimination in Algorithms (December 2020). NBER Working Paper No. w28222, Available at SSRN: https://ssrn.com/abstract=3753132

David Arnold (Contact Author)

University of California, San Diego (UCSD) ( email )

9500 Gilman Drive
Mail Code 0502
La Jolla, CA 92093-0112
United States

Will Dobbie

Harvard University - Harvard Kennedy School (HKS) ( email )

79 John F. Kennedy Street
Cambridge, MA 02138
United States

Peter Hull

University of Chicago - Becker Friedman Institute for Economics ( email )

Chicago, IL 60637
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
26
Abstract Views
575
PlumX Metrics