The Validity of Amazon’s Mechanical Turk in Assessing Risk Preferences - A Research Note

Abstract

In this study, we investigate the validity of Amazon’s Mechanical Turk (MTurk) for assessing risk preferences via the widely-used lottery-choice task of Holt and Laury (2002). More specifically, we study the share of participants who show a consistent answer-pattern and, thus, consistent risk preferences. Our results show that in a sample of 1202 participants on MTurk, only 43 percent managed to answer the task consistently. By comparing this percentage with accounting studies which use this task in the laboratory, we observe that the share of consistent answers is considerably lower when using MTurk. Our results also reveal that participants who answered inconsistently spent less time on the task and showed different risk preference values. Furthermore, several demographic variables seem to influence the likelihood of answering consistently. Our findings indicate that researchers should be careful when considering MTurk as a platform to recruit participants and should expect to need larger samples.

Keywords: Risk Preferences, MTurk, Risk Aversion, Consistency, Validity, Data Quality