Discriminating Tastes: Customer Ratings as Vehicles for Bias

21 Pages Posted: 27 Oct 2016 Last revised: 1 Nov 2016

Alex Rosenblat

Data & Society Research Institute

Karen EC Levy

Cornell University

Solon Barocas

Microsoft Research

Tim Hwang

Data & Society Research Institute

Date Written: October 19, 2016

Abstract

Consumer-sourced rating systems are a dominant method of worker evaluation in platform-based work. These systems facilitate the semi-automated management of large, disaggregated workforces, and the rapid growth of service platforms — but may also represent a potential backdoor to employment discrimination. Our paper analyzes the Uber platform as a case study to explore how bias may creep into evaluations of drivers through consumer-sourced rating systems. A good deal of social science research suggests that aggregated consumer ratings are likely to be inflected with biases against members of legally protected groups. While companies are legally prohibited from making employment decisions based on protected characteristics of workers, their reliance on potentially biased consumer ratings to make material determinations may nonetheless lead to disparate impact in employment outcomes. Hence, the mediating role of the rating system opens the door to employment discrimination.

We analyze the limitations of current civil rights law to address this issue, and outline a number of operational, legal, and design-based interventions that might assist in so doing. The analysis highlights how innovative work structures challenge traditional legal frameworks, and require creative design, development, operation, and regulation to ensure that they do not facilitate discriminatory outcomes against historically disadvantaged groups.

Keywords: platforms, data, discrimination, bias, inequality, ratings, sharing economy, algorithm

Suggested Citation

Rosenblat, Alex and Levy, Karen EC and Barocas, Solon and Hwang, Tim, Discriminating Tastes: Customer Ratings as Vehicles for Bias (October 19, 2016). Available at SSRN: https://ssrn.com/abstract=2858946

Alex Rosenblat

Data & Society Research Institute ( email )

36 West 20th Street
New York,, NY
United States

HOME PAGE: http://www.datasociety.net

Karen EC Levy

Cornell University ( email )

Ithaca, NY 14853
United States

Solon Barocas (Contact Author)

Microsoft Research

641 Avenue of Americas
New York, NY 10011
United States

Tim Hwang

Data & Society Research Institute ( email )

36 West 20th Street
New York,, NY
United States

Paper statistics

Downloads
139
Rank
171,889
Abstract Views
707