The Ethics of Online Controlled Experiments (A/B testing)

31 Pages Posted: 21 Nov 2022

See all articles by Andrea Polonioli

Andrea Polonioli

Coveo Labs, Coveo Solutions Inc; Department of Legal Studies, University of Bologna

Riccardo Ghioni

University of Florence - Department of Statistics, Computer Science, Applications; Department of Legal Studies, University of Bologna

Ciro Greco

South Park Commons

Prathm Juneja

University of Oxford - Oxford Internet Institute

Jacopo Tagliabue

South Park Commons

David Watson

King's College London

Luciano Floridi

Yale University - Digital Ethics Center; University of Bologna- Department of Legal Studies

Date Written: November 2022

Abstract

Online controlled experiments, also known as A/B tests, have become ubiquitous. While many practical challenges in running experiments at scale have been thoroughly discussed, the ethical dimension of A/B testing has been neglected. This article fills this gap in the literature by introducing a new, soft ethics and governance framework that explicitly recognizes how the rise of an experimentation culture in industry settings brings not only unprecedented opportunities to businesses but also significant responsibilities. More precisely, the article (a) introduces a set of principles to encourage ethical and responsible experimentation to protect users, customers, and society; (b) argues that ensuring compliance with the proposed principles is a complex challenge unlikely to be addressed by resorting to a one-solution response; (c) discusses the relevance and effectiveness of several mechanisms and policies in educating, governing, and incentivizing companies conducting online controlled experiments; and (d) offers a list of prompting questions specifically designed to help and empower practitioners by stimulating specific ethical deliberations and facilitating coordination among different groups of stakeholders.

Keywords: A/B tests, Research Ethics, Digital Ethics, Governance, Applied Ethics

Suggested Citation

Polonioli, Andrea and Ghioni, Riccardo and Greco, Ciro and Juneja, Prathm and Tagliabue, Jacopo and Watson, David and Floridi, Luciano, The Ethics of Online Controlled Experiments (A/B testing) (November 2022). Available at SSRN: https://ssrn.com/abstract=4282827 or http://dx.doi.org/10.2139/ssrn.4282827

Andrea Polonioli (Contact Author)

Coveo Labs, Coveo Solutions Inc ( email )

3175 des Quatre-Bourgeois Suite 200
Quebec City
Canada

Department of Legal Studies, University of Bologna ( email )

, Via Zamboni, 27, 40126
Bologna
Italy

Riccardo Ghioni

University of Florence - Department of Statistics, Computer Science, Applications ( email )

Florence
Italy

Department of Legal Studies, University of Bologna ( email )

, Via Zamboni, 27, 40126
Bologna
Italy

Ciro Greco

South Park Commons ( email )

United States

Prathm Juneja

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford OX1 3PG Oxfordshire, Oxfordshire OX1 3JS
United Kingdom

Jacopo Tagliabue

South Park Commons ( email )

United States

David Watson

King's College London ( email )

Strand
London, England WC2R 2LS
United Kingdom

Luciano Floridi

Yale University - Digital Ethics Center ( email )

85 Trumbull Street
New Haven, CT CT 06511
United States
2034326473 (Phone)

University of Bologna- Department of Legal Studies ( email )

Via Zamboni 22
Bologna, Bo 40100
Italy

HOME PAGE: http://www.unibo.it/sitoweb/luciano.floridi/en

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
502
Abstract Views
2,579
Rank
108,109
PlumX Metrics