Evaluating the Impact of Humanization and Emotion in Social Justice Chatbots: Evidence from a Randomized Field Experiment

Abstract

In response to the significant Social Justice movements of 2020, many organizations are seeking sustainable methods of engagement amid dwindling resources. This study investigates the efficacy of Artificial Intelligence (AI) chatbots designed for nonprofits, diverging from traditional business-centric AI-based conversational agents. By collaborating with a Minneapolis-based social justice organization, the research examines the role of emotional versus informational appeals and anthropomorphism in chatbot interactions within a randomized field experiment involving 276 participants. Notably, we find that the number of emotional appeals in the interaction and increased anthropomorphismboth negatively impact conversion. For anthropomorphism, we surmise conversational expectations of a real human drive the participants to convert less. We also find that increasing emotional appeals evoke a highly emotional response, leading to reactance which also leads to lower conversion. These findings suggest that traditional means of persuasion, social cues, and emotional appeals might not fit social justice contexts and may even harm donor interactions. Finally, our findings indicate that the order of engagement solicitations matters:soliciting donations at the end of a sequence of asks results in higher conversion. The study provides valuable insights for chatbot design tailored to social justice and charitable giving organizations and emphasizes crafting appropriate AI-driven messaging for effective outreach.