Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability

48 Pages Posted: 22 May 2019 Last revised: 5 Jun 2019

See all articles by Jon Kleinberg

Jon Kleinberg

Cornell University - Department of Computer Science

Sendhil Mullainathan

University of Chicago

Multiple version iconThere are 2 versions of this paper

Date Written: May 1, 2019

Abstract

Algorithms are increasingly used to aid, or in some cases supplant, human decision-making, particularly for decisions that hinge on predictions. As a result, two additional features in addition to prediction quality have generated interest: (i) to facilitate human interaction and understanding with these algorithms, we desire prediction functions that are in some fashion simple or interpretable; and (ii) because they influence consequential decisions, we also want them to produce equitable allocations. We develop a formal model to explore the relationship between the demands of simplicity and equity. Although the two concepts appear to be motivated by qualitatively distinct goals, we show a fundamental inconsistency between them. Specifically, we formalize a general framework for producing simple prediction functions, and in this framework we establish two basic results. First, every simple prediction function is strictly improvable: there exists a more complex prediction function that is both strictly more efficient and also strictly more equitable. Put another way, using a simple prediction function both reduces utility for disadvantaged groups and reduces overall welfare relative to other options. Second, we show that simple prediction functions necessarily create incentives to use information about individuals' membership in a disadvantaged group—incentives that weren't present before simplification, and that work against these individuals. Thus, simplicity transforms disadvantage into bias against the disadvantaged group. Our results are not only about algorithms but about any process that produces simple models, and as such they connect to the psychology of stereotypes and to an earlier economics literature on statistical discrimination.

JEL Classification: C54, C55, D8, I30, J7, K00

Suggested Citation

Kleinberg, Jon and Mullainathan, Sendhil, Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability (May 1, 2019). University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2019-77. Available at SSRN: https://ssrn.com/abstract=3391770 or http://dx.doi.org/10.2139/ssrn.3391770

Jon Kleinberg

Cornell University - Department of Computer Science ( email )

4130 Upson Hall
Ithaca, NY 14853-7501
United States

Sendhil Mullainathan (Contact Author)

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
14
Abstract Views
122
PlumX Metrics