Llm-Based Conversational Agents for Behaviour Change Support: A Randomized Controlled Trial Examining Efficacy, Safety, and the Role of User Behaviour

Abstract

This study examines the use of Motivational Interviewing (MI) principles in a GPT-4-based chatbot, MIcha, to promote behaviour change. We conducted a pre-registered randomised controlled trial to assess the integration of MI techniques in conversational agents, aiming to support users' behaviour change through guided self-reflection and identify how users interact with large language model (LLM)-based systems in this context. Results indicate that short conversations with LLM-based chatbots are successful at increasing users' readiness to change and usage of MI principles during text generation can effectively mitigate potential harms. Additionally, we identified distinct user behaviour types—cooperative, reflective, and pre-informed—that significantly influenced the outcomes of interactions. These findings demonstrate the potential of MI principles in enhancing the efficacy of conversational agents for behaviour change and highlight the importance of user behaviour in shaping interaction dynamics.

Keywords: Human-Computer-Interaction, Behaviour Change, Large Language Models, Conversational Agents