When Systems Fail: Training Data Accuracy and Operational Transparency

Abstract

There is an increasing dependence on chatbots to achieve high-quality automated customer service. However, these systems rely on accurate training data created by workers interacting with a remote system that can be unreliable. Our primary research questions are 1) to what extent can system failures impact worker performance after a system fails and is restored, and 2) what remedies exist that can reduce the impact of these failures on worker performance. To answer these questions, we conducted eight experiments (four at a large US university and four on Amazon Mechanical Turk) in which subjects were asked to perform tasks commonly used to train data used for an artificial intelligence (AI) model. In one set of experiments, subjects answer questions based on text which would be used to train a chatbot. In another set of experiments, subjects classify images, which is the most used classification tool in AI. Consistently, our results show that a system failure leads to a decrease in task accuracy after the system recovers from failure and comes back online. Providing a neural network with more accurately labeled training data results in around a 5% improvement in accuracy on out-of-sample predictions. Furthermore, providing employees with operational transparency about the failure restoration status brings accuracy back to pre-failure levels, performing better than performance-based pay, a common tool to motivate high-accuracy work. Finally, we use mediation analysis to test for four plausible mechanisms behind our main effect and find that worker confidence is an important mediating factor.

Keywords: operational transparency, service operations, lab experiment, artificial intelligence, future of work