Explanation Seeking and Recommendation Adherence in Human-to-Human versus Human-to-Artificial Intelligence Interactions

Abstract

We conduct a lab experiment to examine when and why do explanation seeking and recommendation adherence behaviours differ in human-to-human versus human-to-AI settings and across variances in task certainty and deviation from prior belief. We find that individuals seek explanations more often from humans than AI models in unfavourable and abnormal conditions, and that recommendation adherence also differs between humans versus AI, tied to the salience of factors that influence the recommendation outcome, with higher levels of confidence associated with AI in unfavourable conditions. When interacting with humans, a favourable highly salient factor results in greater adherence, especially when accompanied by an explanation. However, when interacting with AI, greater adherence occurs despite the highly salient factor being unfavourable. We find that explanations matter for recommendation adherence, but only when the recommendation is favourable, unexpected, and from a human. Lastly, we find that trust is more frequently associated with AI over humans, particularly in unfavourable and abnormal conditions. We support our experimental findings with a posthoc analysis of participant comments using machine learning topic modelling, emotion classification, predictive modelling, and qualitative analysis.

Keywords: artificial intelligence, explanations, algorithm aversion, recommendation adherence, certainty, behavioural experiment, topic modelling