The Problem of Infra-Marginality in Outcome Tests for Discrimination

21 Pages Posted: 21 Jul 2016 Last revised: 18 May 2017

See all articles by Camelia Simoiu

Camelia Simoiu

Stanford University

Sam Corbett-Davies

Stanford University

Sharad Goel

Stanford University

Date Written: July 18, 2016

Abstract

Outcome tests are a popular method for detecting bias in lending, hiring, and policing decisions. These tests operate by comparing the success rate of decisions across groups. For example, if loans made to minority applicants are observed to be repaid more often than loans made to whites, it suggests that only exceptionally qualified minorities are granted loans, indicating discrimination.

Outcome tests, however, are known to suffer from the problem of infra-marginality: even absent discrimination, the repayment rates for minority and white loan recipients might differ if the two groups have different risk distributions. Thus, at least in theory, outcome tests can fail to accurately detect discrimination. We develop a new statistical test of discrimination --- the threshold test --- that mitigates the problem of infra-marginality by jointly estimating decision thresholds and risk distributions via a hierarchical Bayesian latent variable model. Applying our test to a dataset of 4.5 million police stops in North Carolina, we find that the problem of infra-marginality is more than a theoretical possibility, and can cause the outcome test to yield misleading results in practice.

Keywords: bayesian, threshold test, racial, discrimination, police search, traffic stops

Suggested Citation

Simoiu, Camelia and Corbett-Davies, Sam and Goel, Sharad, The Problem of Infra-Marginality in Outcome Tests for Discrimination (July 18, 2016). Available at SSRN: https://ssrn.com/abstract=2811449 or http://dx.doi.org/10.2139/ssrn.2811449

Camelia Simoiu (Contact Author)

Stanford University ( email )

Stanford, CA 94305
United States

Sam Corbett-Davies

Stanford University ( email )

Gates Computer Science Building
353 Serra Mall
Stanford, CA 94305-9025
United States

Sharad Goel

Stanford University ( email )

475 Via Ortega
Stanford, CA 94305
United States

HOME PAGE: http://5harad.com

Register to save articles to
your library

Register

Paper statistics

Downloads
145
rank
191,160
Abstract Views
755
PlumX Metrics