Good Explanation for Algorithmic Transparency

31 Pages Posted: 7 Jan 2020

See all articles by Joy Lu

Joy Lu

Carnegie Mellon University, Tepper School of Business

Dokyun (DK) Lee

Carnegie Mellon University - David A. Tepper School of Business

Tae Wan Kim

Carnegie Mellon University - David A. Tepper School of Business

David Danks

Carnegie Mellon University

Date Written: November 11, 2019

Abstract

Machine learning algorithms have gained widespread usage across a variety of domains, both in providing predictions to expert users and recommending decisions to everyday users. However, these AI systems are often black boxes, and end-users are rarely provided with an explanation of the algorithmic output, which could lead to a significant loss of trust and willingness to use. The critical need for explanation and justification by AI systems has led to calls for algorithmic transparency, including the "right to explanation" in the EU General Data Protection Regulation (GDPR), which requires many companies to provide a meaningful explanation to involved parties. These initiatives all presuppose that we know what constitutes a meaningful or good explanation, but there has been limited research on this question in the context of AI systems. In this paper, we (1) develop a generalizable framework grounded in philosophy, psychology, and interpretable machine learning to investigate and define characteristics of good explanation, and (2) conduct a large-scale lab experiment to measure the impact of different factors on perceptions of understanding, fairness, and trust within a loan application context. The framework and study together form a concrete guide for managers to present algorithmic prediction rationales to end-users to foster trust and adoption. They also highlight elements of explanation to be considered by AI researchers and engineers in designing, developing, and deploying explainable machine learning algorithms.

Keywords: explainable AI, interpretable AI, lab experiments

Suggested Citation

Lu, Joy and Lee, Dokyun (DK) and Kim, Tae Wan and Danks, David, Good Explanation for Algorithmic Transparency (November 11, 2019). Available at SSRN: https://ssrn.com/abstract=3503603 or http://dx.doi.org/10.2139/ssrn.3503603

Joy Lu (Contact Author)

Carnegie Mellon University, Tepper School of Business ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213-3890
United States

Dokyun (DK) Lee

Carnegie Mellon University - David A. Tepper School of Business ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213-3890
United States

Tae Wan Kim

Carnegie Mellon University - David A. Tepper School of Business ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213-3890
United States

David Danks

Carnegie Mellon University ( email )

Pittsburgh, PA 15213-3890
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
199
Abstract Views
1,348
rank
166,389
PlumX Metrics