Is Tricking a Robot Hacking?

19 Pages Posted: 28 Mar 2018  

Ryan Calo

University of Washington - School of Law; Stanford University - Law School; Yale Law School

Ivan Evtimov

University of Washington - Paul G. Allen School of Computer Science & Engineering

Earlence Fernandes

University of Washington - Paul G. Allen School of Computer Science & Engineering

Tadayoshi Kohno

University of Washington - Paul G. Allen School of Computer Science & Engineering

David O'Hair

University of Washington School of Law

Date Written: March 27, 2018

Abstract

The term “hacking” has come to signify breaking into a computer system. A number of local, national, and international laws seek to hold hackers accountable for breaking into computer systems to steal information or disrupt their operation. Other laws and standards incentivize private firms to use best practices in securing computers against attack.

A new set of techniques, aimed not at breaking into computers but at manipulating the increasingly intelligent machine learning models that control them, may force law and legal institutions to reevaluate the very nature of hacking. Three of the authors have shown, for example, that it is possible to use one’s knowledge of a system to fool a driverless car into perceiving a stop sign as a speed limit. Other techniques build secret blind spots into machine learning systems or seek to reconstruct the private data that went into their training.

The unfolding renaissance in artificial intelligence (AI), coupled with an almost parallel discovery of its vulnerabilities, requires a reexamination of what it means to “hack,” i.e., to compromise a computer system. The stakes are significant. Unless legal and societal frameworks adjust, the consequences of misalignment between law and practice include inadequate coverage of crime, missing or skewed security incentives, and the prospect of chilling critical security research. This last one is particularly dangerous in light of the important role researchers can play in revealing the biases, safety limitations, and opportunities for mischief that the mainstreaming of artificial intelligence appears to present.

The authors of this essay represent an interdisciplinary team of experts in machine learning, computer security, and law. Our aim is to introduce the law and policy community within and beyond academia to the ways adversarial machine learning (ML) alter the nature of hacking and with it the cybersecurity landscape. Using the Computer Fraud and Abuse Act of 1986 — the paradigmatic federal anti-hacking law — as a case study, we mean to evidence the burgeoning disconnect between law and technical practice. And we hope to explain what is at stake should we fail to address the uncertainty that flows from the prospect that hacking now includes tricking.

Keywords: computers, artificial intelligence (AI), machine learning (ML), model training, deep learning, hacking, robots, robotics, algorithms, image recognition, Computer Fraud and Abuse Act (CFAA)

Suggested Citation

Calo, Ryan and Evtimov, Ivan and Fernandes, Earlence and Kohno, Tadayoshi and O'Hair, David, Is Tricking a Robot Hacking? (March 27, 2018). University of Washington School of Law Research Paper No. 2018-05. Available at SSRN: https://ssrn.com/abstract=3150530 or http://dx.doi.org/10.2139/ssrn.3150530

Ryan Calo (Contact Author)

University of Washington - School of Law ( email )

William H. Gates Hall
Box 353020
Seattle, WA 98105-3020
United States

Stanford University - Law School ( email )

559 Nathan Abbott Way
Stanford, CA 94305-8610
United States

Yale Law School ( email )

127 Wall Street
New Haven, CT 06511
United States

Ivan Evtimov

University of Washington - Paul G. Allen School of Computer Science & Engineering ( email )

Seattle, WA 98195
United States

HOME PAGE: http://https://www.cs.washington.edu/

Earlence Fernandes

University of Washington - Paul G. Allen School of Computer Science & Engineering ( email )

Seattle, WA 98195
United States

HOME PAGE: http://https://www.cs.washington.edu/

Tadayoshi Kohno

University of Washington - Paul G. Allen School of Computer Science & Engineering ( email )

Seattle, WA 98195
United States

David O'Hair

University of Washington School of Law ( email )

William H. Gates Hall
Box 353020
Seattle, WA 98105-3020
United States

HOME PAGE: http://www.law.uw.edu

Register to save articles to
your library

Register

Paper statistics

Downloads
443
rank
57,843
Abstract Views
3,868
PlumX