When Systems Fail: Remote Worker Accuracy and Operational Transparency
26 Pages Posted: 23 Apr 2021
Date Written: April 22, 2021
There is an increasing dependence on remote work systems in many industries, including service operators---a trend that is likely to accelerate in the coming years due to demands for higher flexibility in our workforce. However, workers are unlikely to be successful working remotely when the systems they rely on are unreliable. Our primary research questions are 1) to what extent can system failures impact worker performance after a system fails and is restored, and 2) what remedies exist that can reduce the impact of these failures on worker performance. To answer these questions, we conduct eight experiments (four at a large US university and four on Amazon Mechanical Turk) in which subjects are asked to perform tasks commonly used to train data used for an artificial intelligence (AI) model. In one set of experiments, subjects classify images, which is the most used classification tool in AI. In another set of experiments, subjects train a chatbot, which is a tool expected to make a significant impact among service operators. Consistently, our results show that a system failure leads to a decrease in task accuracy after the system recovers from failure and comes back online. Furthermore, providing employees with operational transparency about the failure restoration status brings accuracy back to pre-failure levels, performing just as well as performance-based pay, a common tool to motivate high-accuracy work. Finally, we use mediation analysis to test for four plausible mechanisms behind our main effect and find that worker confidence is an important mediating factor.
Keywords: operational transparency, service operations, lab experiment, artificial intelligence, future of work
Suggested Citation: Suggested Citation