Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics

In Proceedings of the 21st ACM Conference on Economics and Computation (pp. 679-681).

Columbia Business School Research Paper Forthcoming

44 Pages Posted: 24 Jun 2020 Last revised: 14 Dec 2020

See all articles by Bo Cowgill

Bo Cowgill

Columbia University - Columbia Business School

Fabrizio Dell'Acqua

Columbia University - Columbia Business School

Sam Deng

Columbia University - School of Engineering

Daniel Hsu

Columbia University - School of Engineering

Nakul Verma

Columbia University - School of Engineering

Augustin Chaintreau

Columbia University

Date Written: June 1, 2020

Abstract

Why do biased predictions arise about human capital? What interventions can prevent them? We evaluate 8.2 million algorithmic predictions of math skill from ~400 AI engineers, each of whom developed an algorithm under a randomly assigned experimental condition. Our treatment arms modified programmers' incentives, training data, awareness, and/or technical knowledge of AI ethics. We then assess out-of-sample predictions from their algorithms using randomized audit manipulations of algorithm inputs and ground-truth math performance for 20K subjects. We find that biased predictions are mostly caused by biased training data. However, one-third of the benefit of better training data comes through a novel economic mechanism: Engineers exert greater effort and are more responsive to incentives when given better training data. We also assess how performance varies with programmers' demographic characteristics, and their performance on a psychological test of implicit bias (IAT) concerning gender and careers. We find no evidence that female, minority and low-IAT engineers exhibit lower bias or discrimination in their code. However we do find that prediction errors are correlated within demographic groups, which creates performance improvements through cross-demographic averaging. Finally, we quantify the benefits and tradeoffs of practical managerial or policy interventions such as technical advice, simple reminders and improved incentives for decreasing algorithmic bias.

Suggested Citation

Cowgill, Bo and Dell'Acqua, Fabrizio and Deng, Sam and Hsu, Daniel and Verma, Nakul and Chaintreau, Augustin, Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics (June 1, 2020). In Proceedings of the 21st ACM Conference on Economics and Computation (pp. 679-681)., Columbia Business School Research Paper Forthcoming, Available at SSRN: https://ssrn.com/abstract=3615404 or http://dx.doi.org/10.2139/ssrn.3615404

Bo Cowgill (Contact Author)

Columbia University - Columbia Business School ( email )

3022 Broadway
New York, NY 10027
United States

Fabrizio Dell'Acqua

Columbia University - Columbia Business School ( email )

3022 Broadway
New York, NY 10027
United States

Sam Deng

Columbia University - School of Engineering ( email )

New York, NY 10027
United States

Daniel Hsu

Columbia University - School of Engineering ( email )

New York, NY 10027
United States

Nakul Verma

Columbia University - School of Engineering ( email )

New York, NY 10027
United States

Augustin Chaintreau

Columbia University ( email )

3022 Broadway
New York, NY 10027
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
1,199
Abstract Views
4,628
rank
21,196
PlumX Metrics