Ai Nudging and Decision Quality: Evidence from Randomized Experiments in Online Recommendation Setting

Abstract

This study explores the impacts of AI nudging on customer purchase decisions. Digital nudging is a well-established technique used to alter people’s behaviors in a predictable way. With the rapid development of Artificial Intelligence/Machine Learning (AI/ML) and the widespread integration of the “black box” algorithm in the digital choice architecture, personalized targeting nudges can vastly influence individual and collective behaviors and lead to undesired consequences. AI nudge refers to the situation when human outsources developing and implementing nudges to AI/ML systems. Drawing upon the literature on nudge and recommendation agents/systems in IS, this study investigated the impact of two types of recommendation badges on user decision quality: AI nudge (e.g., Amazon’s Choice) and non-AI nudge (e.g., Best Seller). We found that these two badges can lead to different user perceptions of transparency and thus affect the choice confidence of product selection. In addition, the effect of perceived transparency on choice confidence is contingent upon the mismatch/match between the recommendation and users’ preferences, with perceived transparency exerting significantly higher influence on choice confidence in the preference match condition. We tested our research model using a randomized experiment and post-task survey data collected from 837 US-based college students with online shopping experience. This is the first empirical study examining the impact of AI nudging on user decision-making on e-commerce platforms and will contribute to the nudge literature and biased recommendation research in IS. The study also brings ethical implications to the use of AI/ML models and calls for careful oversight on delegating the power of nudging to AI in guiding online user behavior.a Department of Computer Information Systems, Walker College of Business, Appalachian State University, Boone, NC, United States.

Keywords: AI nudging, recommendation badge, transparency, recommendation dissonance, decision quality, randomized experiment.