Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR

52 Pages Posted: 2 Nov 2017 Last revised: 24 Mar 2018

Sandra Wachter

University of Oxford - Oxford Internet Institute

Brent Mittelstadt

University of Oxford - Oxford Internet Institute

Chris Russell

University of Surrey

Date Written: October 6, 2017

Abstract

There has been much discussion of the “right to explanation” in the EU General Data Protection Regulation, and its existence, merits, and disadvantages. Implementing a right to explanation that opens the ‘black box’ of algorithmic decision-making faces major legal and technical barriers. Explaining the functionality of complex algorithmic decision-making systems and their rationale in specific cases is a technically challenging problem. Some explanations may offer little meaningful information to data subjects, raising questions around their value. Data controllers have an interest to not disclose information about their algorithms that contains trade secrets, violates the rights and freedoms of others (e.g. privacy), or allows data subjects to game or manipulate decision-making.

Explanations of automated decisions need not hinge on the general public understanding how algorithmic systems function. Even though such interpretability is of great importance and should be pursued, explanations can, in principle, be offered without opening the black box. Looking at explanations as a means to help a data subject act rather than merely understand, one could gauge the scope and content of explanations according to the specific goal or action they are intended to support.

From the perspective of individuals affected by automated decision-making, we propose three aims for explanations:

(1) to inform and help the individual understand why a particular decision was reached,

(2) to provide grounds to contest the decision if the outcome is undesired, and

(3) to understand what would need to change in order to receive a desired result in the future, based on the current decision-making model.

We assess how each of these goals finds support in the GDPR, and the extent to which they hinge on opening the ‘black box’. We suggest data controllers should offer a particular type of explanation, ‘unconditional counterfactual explanations’, to support these three aims. These counterfactual explanations describe the smallest change to the world that can be made to obtain a desirable outcome, or to arrive at the “closest possible world.” As multiple variables or sets of variables can lead to one or more desirable outcomes, multiple counterfactual explanations can be provided, corresponding to different choices of nearby possible worlds for which the counterfactual holds. Counterfactuals describe a dependency on the external facts that lead to that decision without the need to convey the internal state or logic of an algorithm. As a result, counterfactuals serve as a minimal solution that bypasses the current technical limitations of interpretability, while striking a balance between transparency and the rights and freedoms of others (e.g. privacy, trade secrets).

Keywords: accountability, algorithms, artificial intelligence, auditing, automated decision-making, counterfactuals, data ethics, data protection, explainability, GDPR, interpretability, machine learning, right to explanation, transparency

Suggested Citation

Wachter, Sandra and Mittelstadt, Brent and Russell, Chris, Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR (October 6, 2017). Harvard Journal of Law & Technology, Forthcoming. Available at SSRN: https://ssrn.com/abstract=3063289 or http://dx.doi.org/10.2139/ssrn.3063289

Sandra Wachter (Contact Author)

University of Oxford - Oxford Internet Institute ( email )

41 St. Giles
Oxford, OX1 3JS
United Kingdom
@SandraWachter5 (Phone)
@SandraWachter5 (Fax)

Brent Mittelstadt

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford OX1 3PG Oxfordshire, Oxfordshire OX1 3JS
United Kingdom

Chris Russell

University of Surrey ( email )

Guildford
Guildford, Surrey GU2 5XH
United Kingdom

Register to save articles to
your library

Register

Paper statistics

Downloads
1,388
rank
11,498
Abstract Views
6,741
PlumX