Ethical Testing in the Real World: Evaluating Physical Testing of Adversarial Machine Learning

Workshop on Dataset Curation and Security // Workshop on Navigating the Broader Impacts of AI Research -- Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

8 Pages Posted:

See all articles by Kendra Albert

Kendra Albert

Harvard Law School

Maggie Delano

Engineering Department, Swarthmore College

Jon Penney

Harvard University - Berkman Klein Center for Internet & Society; Osgoode Hall Law School; Harvard Law School; Citizen Lab, University of Toronto

Afsaneh Rigot

ARTICLE 19

Ram Shankar Siva Kumar

Microsoft Corporation; Harvard University - Berkman Klein Center for Internet & Society

Date Written: December 17, 2020

Abstract

This paper critically assesses the adequacy and representativeness of physical domain testing for various adversarial machine learning (ML) attacks against computer vision systems involving human subjects. Many papers that deploy such attacks characterize themselves as “real world.” Despite this framing, however, we found the physical or real-world testing conducted was minimal, provided few details about testing subjects and was often conducted as an afterthought or demonstration. Adversarial ML research without representative trials or testing is an ethical, scientific, and health/safety issue that can cause real harms. We introduce the problem and our methodology, and then critique the physical domain testing methodologies employed by papers in the field. We then explore various barriers to more inclusive physical testing in adversarial ML and offer recommendations to improve such testing notwithstanding these challenges

Keywords: Artificial Intelligence, AI, Machine Learning, ML Adversarial Machine Learning, Ethics, Physical Testing, Physical Domain, Methodology, ML attacks, FRT, Facial Recognition Technology, Invisibility, Algorithms, Research Ethics, Design, Security, Privacy, Human Rights

JEL Classification: K1, K23, K42, O32, 031,

Suggested Citation

Albert, Kendra and Delano, Maggie and Penney, Jonathon and Rigot, Afsaneh and Siva Kumar, Ram Shankar, Ethical Testing in the Real World: Evaluating Physical Testing of Adversarial Machine Learning (December 17, 2020). Workshop on Dataset Curation and Security // Workshop on Navigating the Broader Impacts of AI Research -- Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Available at SSRN: https://ssrn.com/abstract=

Kendra Albert

Harvard Law School ( email )

1563 Massachusetts Ave
Cambridge, MA 02138
United States

Maggie Delano

Engineering Department, Swarthmore College ( email )

500 College Ave
Swarthmore, PA 19081
United States

Jonathon Penney (Contact Author)

Harvard University - Berkman Klein Center for Internet & Society ( email )

Harvard Law School
23 Everett, 2nd Floor
Cambridge, MA Nova Scotia 02138
Canada

Osgoode Hall Law School ( email )

4700 Keele Street
Toronto, Ontario M3J 1P3
Canada

Harvard Law School ( email )

1575 Massachusetts
Hauser 406
Cambridge, MA 02138
United States

Citizen Lab, University of Toronto ( email )

Munk School of Global Affairs
University of Toronto
Toronto, Ontario M5S 3K7
Canada

Afsaneh Rigot

ARTICLE 19 ( email )

60 Farringdon Road
London, EC1R 1UQ
United Kingdom

Ram Shankar Siva Kumar

Microsoft Corporation ( email )

One Microsoft Way
Redmond, WA 98052
United States

Harvard University - Berkman Klein Center for Internet & Society ( email )

Harvard Law School
23 Everett, 2nd Floor
Cambridge, MA 02138
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
2
Abstract Views
19
PlumX Metrics