How Explainability Contributes to Trust in AI

2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22)

10 Pages Posted: 30 Jan 2022 Last revised: 9 May 2022

See all articles by Andrea Ferrario

Andrea Ferrario

Dep. Management, Technology, and Economics ETH Zurich; Mobiliar Lab for Analytics at ETH

Michele Loi

Department of Mathematics, Politecnico di Milano

Date Written: January 28, 2022

Abstract

We provide a philosophical explanation of the relation between artificial intelligence (AI) explainability and trust in AI, providing a case for expressions, such as “explainability fosters trust in AI,” that commonly appear in the literature. This explanation relates the justification of the trustworthiness of an AI with the need to monitor it during its use. We discuss the latter by referencing an account of trust, called “trust as anti-monitoring,” that different authors contributed developing. We focus our analysis on the case of medical AI systems, noting that our proposal is compatible with internalist and externalist justifications of trustworthiness of medical AI and recent accounts of warranted contractual trust. We propose that “explainability fosters trust in AI” if and only if it fosters justified and warranted paradigmatic trust in AI, i.e., trust in the presence of the justified belief that the AI is trustworthy, which, in turn, causally contributes to rely on the AI in the absence of monitoring. We argue that our proposed approach can intercept the complexity of the interactions between physicians and medical AI systems in clinical practice, as it can distinguish between cases where humans hold different beliefs on the trustworthiness of the medical AI and exercise varying degrees of monitoring on them. Finally, we apply our account to user’s trust in AI, where, we argue, explainability does not contribute to trust. By contrast, when considering public trust in AI as used by a human, we argue, it is possible for explainability to contribute to trust. Our account can explain the apparent paradox that in order to trust AI, we must trust AI users not to trust AI completely. Summing up, we can explain how explainability contributes to justified trust in AI, without leaving a reliabilist framework, but only by redefining the trusted entity as an AI-user dyad.

Keywords: Artificial Intelligence, Explainable Artificial Intelligence, Trust, Healthcare, Trustworthiness, Ethics of Artificial Intelligence

Suggested Citation

Ferrario, Andrea and Loi, Michele, How Explainability Contributes to Trust in AI (January 28, 2022). 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), Available at SSRN: https://ssrn.com/abstract=4020557 or http://dx.doi.org/10.2139/ssrn.4020557

Andrea Ferrario (Contact Author)

Dep. Management, Technology, and Economics ETH Zurich ( email )

Zurich
Switzerland

Mobiliar Lab for Analytics at ETH ( email )

Zürich, 8092
Switzerland

Michele Loi

Department of Mathematics, Politecnico di Milano ( email )

Milano, CH-8006
Italy

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
269
Abstract Views
1,056
rank
158,446
PlumX Metrics