Abstract

http://ssrn.com/abstract=2477899
 


 



Big Data's Disparate Impact


Solon Barocas


Microsoft Research

Andrew D. Selbst


Yale Information Society Project; Georgetown University Law Center

2016

104 California Law Review 671 (2016)

Abstract:     
Advocates of algorithmic techniques like data mining argue that these techniques eliminate human biases from the decision-making process. But an algorithm is only as good as the data it works with. Data is frequently imperfect in ways that allow these algorithms to inherit the prejudices of prior decision makers. In other cases, data may simply reflect the widespread biases that persist in society at large. In still others, data mining can discover surprisingly useful regularities that are really just preexisting patterns of exclusion and inequality. Unthinking reliance on data mining can deny historically disadvantaged and vulnerable groups full participation in society. Worse still, because the resulting discrimination is almost always an unintentional emergent property of the algorithm’s use rather than a conscious choice by its programmers, it can be unusually hard to identify the source of the problem or to explain it to a court.

This Essay examines these concerns through the lens of American antidiscrimination law — more particularly, through Title VII’s prohibition of discrimination in employment. In the absence of a demonstrable intent to discriminate, the best doctrinal hope for data mining’s victims would seem to lie in disparate impact doctrine. Case law and the Equal Employment Opportunity Commission’s Uniform Guidelines, though, hold that a practice can be justified as a business necessity when its outcomes are predictive of future employment outcomes, and data mining is specifically designed to find such statistical correlations. Unless there is a reasonably practical way to demonstrate that these discoveries are spurious, Title VII would appear to bless its use, even though the correlations it discovers will often reflect historic patterns of prejudice, others’ discrimination against members of protected groups, or flaws in the underlying data

Addressing the sources of this unintentional discrimination and remedying the corresponding deficiencies in the law will be difficult technically, difficult legally, and difficult politically. There are a number of practical limits to what can be accomplished computationally. For example, when discrimination occurs because the data being mined is itself a result of past intentional discrimination, there is frequently no obvious method to adjust historical data to rid it of this taint. Corrective measures that alter the results of the data mining after it is complete would tread on legally and politically disputed terrain. These challenges for reform throw into stark relief the tension between the two major theories underlying antidiscrimination law: anticlassification and antisubordination. Finding a solution to big data’s disparate impact will require more than best efforts to stamp out prejudice and bias; it will require a wholesale reexamination of the meanings of “discrimination” and “fairness.”

Number of Pages in PDF File: 62

Keywords: big data, data mining, algorithms, discrimination, employment discrimination, Title VII, civil rights, disparate treatment, disparate impact, procedural fairness, substantive fairness, inequality

JEL Classification: D63, J71, J78, M51, O33


Open PDF in Browser Download This Paper

Date posted: August 11, 2014 ; Last revised: July 15, 2016

Suggested Citation

Barocas, Solon and Selbst, Andrew D., Big Data's Disparate Impact (2016). 104 California Law Review 671 (2016). Available at SSRN: http://ssrn.com/abstract=2477899

Contact Information

Solon Barocas (Contact Author)
Microsoft Research
641 Avenue of Americas
New York, NY 10011
United States
Andrew D. Selbst
Yale Information Society Project ( email )
127 Wall Street
New Haven, CT 06511
United States
Georgetown University Law Center ( email )
600 New Jersey Avenue, NW
Washington, DC 20001
United States
Feedback to SSRN


Paper statistics
Abstract Views: 22,978
Downloads: 4,659
Download Rank: 1,091

© 2016 Social Science Electronic Publishing, Inc. All Rights Reserved.  FAQ   Terms of Use   Privacy Policy   Copyright   Contact Us
This page was processed by apollobot1 in 0.188 seconds