Voice-based AI in Call Center Customer Service: A Natural Field Experiment

Abstract

Voice-based artificial intelligence (AI) systems have been recently deployed to replace traditional interactive voice response (IVR) systems in call center customer service. However, there is little evidence that sheds light on how the implementation of AI systems impacts customer behavior, as well as AI systems’ effects on call center customer service performance. By leveraging the proprietary data obtained from a natural field experiment in a large telecommunication company, we examine how the introduction of a voice-based AI system affects call length, customers’ demand for human service, and customer complaints in call center customer service. We find that the implementation of the AI system temporarily increases the duration of machine service and customers’ demand for human service; however, it persistently reduces customer complaints. Furthermore, our results reveal interesting heterogeneity in the effectiveness of the voice-based AI system. For relatively simple service requests, the AI system reduces customer complaints for both experienced and inexperienced customers. However, for complex requests, customers appear to learn from the prior experience of interacting with the AI system, which leads to fewer complaints. Moreover, the AI-based system has a significantly larger effect on reducing customer complaints for older and female customers as well as for customers who have had extensive experience using the IVR system. Finally, we find that the speech-recognition failures in customer-AI interactions lead to increases in customers’ demand for human service and customer complaints. The results from this study provide implications for the implementation of an AI system in call center operations.

Keywords: Artificial intelligence; customer service; service flexibility; natural field experiment; difference-in-differences