Real-Effort Incentives in Online Labor Markets: Punishments and Rewards for Individuals and Groups

46 Pages Posted: 9 Jul 2014 Last revised: 6 May 2022

See all articles by Matthew J. Hashim

Matthew J. Hashim

University of Arizona

Jesse Bockstedt

University of Arizona - Department of Management Information Systems

Date Written: August 12, 2021

Abstract

Online labor markets and the humans that power them serve a critical role in the advancement of artificial intelligence and supervised machine learning via the creation of high-quality training datasets. The use of human effort in online labor markets is not enough, however, as a key factor is understanding the possible interventions that market operators can leverage to incentivize high-quality human effort among their labor force. We propose that platforms could implement mechanisms such as rewards or punishments at individual or group levels to incentivize real-effort and high-quality output. We apply our interventions using a collaborative image tagging experiment and the results provide interesting insights and non-obvious consequences. On average, interventions applied at the group level outperformed interventions applied at the individual level. Punishing the group provides the most controversial incentive strategy and provides a non-obvious significant improvement in effort. Rewarding or sanctioning an individual had similar effects on average, with both treatments leading to significant increases in effort post-intervention. In contrast to predictions, sanctioning appears to significantly motivate those that were punished. Overall, interventions applied in our real-effort collaborative image tagging experiment had a significant impact on behavior and provides guidance for online labor market operators.

Keywords: online labor market, free-riding, incentive mechanism, economic experiment, economics of IS

Suggested Citation

Hashim, Matthew J. and Bockstedt, Jesse, Real-Effort Incentives in Online Labor Markets: Punishments and Rewards for Individuals and Groups (August 12, 2021). Available at SSRN: https://ssrn.com/abstract=2463453 or http://dx.doi.org/10.2139/ssrn.2463453

Matthew J. Hashim (Contact Author)

University of Arizona ( email )

Department of Management Information Systems
Eller College of Management
Tucson, AZ 85721
United States

Jesse Bockstedt

University of Arizona - Department of Management Information Systems ( email )

AZ
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
168
Abstract Views
1,328
rank
245,970
PlumX Metrics