MTurk and Online Panel Research: The Impact of COVID-19, Bots, TikTok, and other Contemporary Developments
The Cambridge Handbook of Consumer Psychology, Cait Lamberton; Derek Rucker; Stephen A. Spiller (2nd Edition).
34 Pages Posted: 25 Mar 2022
Date Written: March 24, 2022
Abstract
Researchers are increasingly leveraging crowdsourcing platforms like Amazon’s Mechanical Turk (MTurk), CloudResearch, and Prolific to access large, diverse human subject pools. These online panels have become a common source of data for behavioral researchers and consumer psychologists alike (Goodman & Paolacci, 2017; Wright & Goodman, 2019). In this chapter we review contemporary issues associated with online panel research. One of our main findings is that the COVID-19 pandemic impacted the extent to which researchers use online panels and the workers participating on certain online panels. We also find that factors, like a TikTok video, can impact who uses these online panels and why. We also examine the data quality practices of researchers using longitudinal data and find that many practices do not align with what is currently recommended in the literature. We provide several recommendations for researchers to conduct high quality behavioral research online. These include the use of appropriate prescreens before data collection, data analysis pre-registration practices, and avoiding post-screens after data collection that are not pre-registered. We also recommend researchers thoroughly report details on recruitment, restrictions, completion rates, and any differences in dropout rates across conditions.
Keywords: Mechanical Turk, MTurk, crowdsourcing, data collection, sampling, COVID, COVID-19, online research, TikTok, Social Media
Suggested Citation: Suggested Citation