The Meaning of 'Explainability Fosters Trust in AI'

Posted: 4 Sep 2021 Last revised: 31 Jan 2022

See all articles by Andrea Ferrario

Andrea Ferrario

Dep. Management, Technology, and Economics ETH Zurich; Mobiliar Lab for Analytics at ETH

Michele Loi

Department of Mathematics, Politecnico di Milano

Date Written: September 2, 2021

Abstract

We provide a philosophical explanation of the relation between artificial intelligence (AI) explainability and trust in AI, providing a case for expressions, such as “explainability fosters trust in AI,” that commonly appear in the literature. This explanation considers the justification of the trustworthiness of an AI with the need to monitor it during its use. We discuss the latter by referencing an account of trust, called “trust as anti-monitoring,” that different authors contributed developing. We focus our analysis on the case of medical AI systems, noting that our proposal is compatible with internalist and externalist justifications of trustworthiness of medical AI and recent accounts of warranted contractual trust. We propose that “explainability fosters trust in AI” if and only if it fosters warranted paradigmatic trust in AI, i.e., trust in the presence of the justified belief that the AI is trustworthy, which, in turn, causally contributes to rely on the AI in the absence of monitoring. We argue that our proposed approach can intercept the complexity of the interactions between physicians and medical AI systems in clinical practice, as it can distinguish between cases where humans hold different beliefs on the trustworthiness of the medical AI and exercise varying degrees of monitoring on them. Finally, we discuss the main limitations of explainable AI methods and we argue that the case of “explainability fosters trust in AI” may be feasible only in a limited number of physician-medical AI interactions in clinical practice.

Keywords: Artificial Intelligence, Explainable Artificial Intelligence, Trust, Healthcare, Trustworthiness, Ethics of Artificial Intelligence

Suggested Citation

Ferrario, Andrea and Loi, Michele, The Meaning of 'Explainability Fosters Trust in AI' (September 2, 2021). Available at SSRN: https://ssrn.com/abstract=3916396 or http://dx.doi.org/10.2139/ssrn.3916396

Andrea Ferrario (Contact Author)

Dep. Management, Technology, and Economics ETH Zurich ( email )

Zurich
Switzerland

Mobiliar Lab for Analytics at ETH ( email )

Zürich, 8092
Switzerland

Michele Loi

Department of Mathematics, Politecnico di Milano ( email )

Milano, CH-8006
Italy

Do you want regular updates from SSRN on Twitter?

Paper statistics

Abstract Views
777
PlumX Metrics