Nowcasting in Chatbot Design: Leveraging Service Journey Patterns to Improve User Satisfaction

52 Pages Posted: 21 May 2020

See all articles by Yang Wang

Yang Wang

affiliation not provided to SSRN

Yuran Wang

Zhejiang University - School of Management

Xueming Luo

Temple University

xiao yi wang

affiliation not provided to SSRN

Date Written: April 15, 2020

Abstract

The rise of intelligent conversation agents, or chatbots, are responsible for the dramatic decrease in remote customer service agent jobs. However, chatbots in their current form, are far from infallible. We theorize that there is an inherent trade-off between a chatbot's response relevance and conversational efficiency in the standard knowledge-bank architecture. Knowledge bank size increases the relevance of successfully queried results, but also increases the difficulty of disambiguating user intents. This inherent trade-off leads to the development of unintelligent fail-safe artifacts such as user confirmations. We argue that, in order to improve user experience and satisfaction, we must decouple knowledge bank size from conversational efficiency. To achieve this, we first design a new artifact that we dub the sequential FAQ (sFAQ) and then evaluate its causal impact on user satisfaction. An sFAQ uses machine learning techniques to first discover common user service journey patterns, then leverage these learned patterns to predict likely subsequent inputs given any focal sequence of inputs. We show that by proactively suggesting potential questions to the user, we can reduce the need for natural language input and thus reduce the need to disambiguate user intent. We then use a novel application of regression discontinuity design (RDD) to study the causal impact of the eliminated reconfirmation dialogues on user satisfaction. Combined, we are able to demonstrate that by eliminating the unintelligent fail-safe artifacts such as user confirmations, the sFAQ will increase satisfaction. Our approach of combining predictive machine learning and causal econometric analysis enables us to open the black box for the underlying causal mechanism linking sFAQ and user satisfaction. This kind of mechanism identification would not be possible even with experimental testing in the field. Our methods and results have useful implications for chatbot applications and user interface design science.

Keywords: design science, chat-bot, RDD, computer human interaction, machine learning

Suggested Citation

Wang, Yang and Wang, Yuran and Luo, Xueming and wang, xiao yi, Nowcasting in Chatbot Design: Leveraging Service Journey Patterns to Improve User Satisfaction (April 15, 2020). Available at SSRN: https://ssrn.com/abstract=3576988 or http://dx.doi.org/10.2139/ssrn.3576988

Yang Wang (Contact Author)

affiliation not provided to SSRN

Yuran Wang

Zhejiang University - School of Management ( email )

Hangzhou, Zhejiang Province 310058
China

Xueming Luo

Temple University ( email )

1810 N. 13th Street
Floor 2
Philadelphia, PA 19128
United States

HOME PAGE: http://www.fox.temple.edu/mcm_people/xueming-luo/

Xiao yi Wang

affiliation not provided to SSRN

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
370
Abstract Views
1,346
Rank
147,579
PlumX Metrics