Algorithms that Remember: Model Inversion Attacks and Data Protection Law

376 Philosophical Transactions of the Royal Society A 20180083, 2018, DOI: 10.1098/rsta.2018.0083

15 Pages Posted: 6 Aug 2018 Last revised: 19 Oct 2018

See all articles by Michael Veale

Michael Veale

University College London, Department of Science, Technology, Engineering and Public Policy (STEaPP)

Reuben Binns

University of Oxford

Lilian Edwards

Date Written: July 12, 2018

Abstract

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around 'model inversion' and 'membership inference' attacks, which indicate that the process of turning training data into machine learned systems is not one-way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.

Keywords: data protection, gdpr, machine learning, model inversion, membership inference, algorithmic accountability

Suggested Citation

Veale, Michael and Binns, Reuben and Edwards, Lilian, Algorithms that Remember: Model Inversion Attacks and Data Protection Law (July 12, 2018). 376 Philosophical Transactions of the Royal Society A 20180083, 2018, DOI: 10.1098/rsta.2018.0083. Available at SSRN: https://ssrn.com/abstract=3212755

Michael Veale (Contact Author)

University College London, Department of Science, Technology, Engineering and Public Policy (STEaPP) ( email )

Boston House
36–38 Fitzroy Square
London, W1t 3EY
United Kingdom

Reuben Binns

University of Oxford ( email )

Mansfield Road
Oxford, Oxfordshire OX1 4AU
United Kingdom

No contact information is available for Lilian Edwards

Register to save articles to
your library

Register

Paper statistics

Downloads
390
rank
70,225
Abstract Views
2,126
PlumX Metrics