Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics

In Proceedings of the 21st ACM Conference on Economics and Computation (pp. 679-681).

Posted: 24 Jun 2020

See all articles by Bo Cowgill

Bo Cowgill

Columbia University - Columbia Business School

Fabrizio Dell'Acqua

Columbia University - Columbia Business School

Sam Deng

Columbia University - School of Engineering

Daniel Hsu

Columbia University - School of Engineering

Nakul Verma

Columbia University - School of Engineering

Augustin Chaintreau

Columbia University

Date Written: June 1, 2020

Abstract

Why do biased beliefs arise, and what interventions can prevent them? We study this topic in a field experiment about using machine learning to predict human capital. We randomly assign ~400 AI engineers to predict standardized test scores of OECD residents under different experimental conditions. We then assess the resulting predictive algorithms using the realized test performance and through randomized audit-like manipulations of algorithmic inputs. We find that biased beliefs are mostly caused by biased training data. However, simple reminders about bias are almost half as effective at fully de-biasing training data. We find mixed results on technical education and programmer demographics. Programmers who understand technical guidance successfully reduce bias. However, many do not follow the advice, resulting in algorithms that are worse than programmers given a simple reminder. Predictions by female and minority AI programmers do not exhibit less bias or discrimination. However prediction errors are correlated within engineering demographics, creating bias reductions from cross-demographic averaging. We also find no effects of our incentive treatments and no evidence that programmers' implicit associations between gender and math (measured through an IAT) are correlated with bias in code.

Suggested Citation

Cowgill, Bo and Dell'Acqua, Fabrizio and Deng, Sam and Hsu, Daniel and Verma, Nakul and Chaintreau, Augustin, Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics (June 1, 2020). In Proceedings of the 21st ACM Conference on Economics and Computation (pp. 679-681)., Available at SSRN: https://ssrn.com/abstract=3615404 or http://dx.doi.org/10.2139/ssrn.3615404

Bo Cowgill (Contact Author)

Columbia University - Columbia Business School ( email )

3022 Broadway
New York, NY 10027
United States

Fabrizio Dell'Acqua

Columbia University - Columbia Business School ( email )

3022 Broadway
New York, NY 10027
United States

Sam Deng

Columbia University - School of Engineering ( email )

New York, NY 10027
United States

Daniel Hsu

Columbia University - School of Engineering ( email )

New York, NY 10027
United States

Nakul Verma

Columbia University - School of Engineering ( email )

New York, NY 10027
United States

Augustin Chaintreau

Columbia University ( email )

3022 Broadway
New York, NY 10027
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
112
Abstract Views
415
PlumX Metrics