Time-of-Day and Day-of-Week Variations in Amazon Mechanical Turk Survey Responses

40 Pages Posted: 29 Jul 2021

See all articles by Carola Binder

Carola Binder

Haverford College - Department of Economics

Date Written: July 6, 2021


Social science research studies are frequently conducted on Amazon Mechanical Turk (MTurk). I use data from four previous economics studies conducted on Amazon Mechanical Turk, with a total of 2780 observations, to study how participant characteristics and behaviors depend on the day of the week and time of day of participation. Most notably, Saturday participants are older, less educated, and more likely to have low income compared to other participants. Controlling for demographics, Saturday participants are more likely to answer objective knowledge questions correctly and to provide reasonable inflation forecasts, less likely to provide "don't know" responses, and less likely to use social media as a primary source of economic news. Night participants are less likely to get economic news from print sources such as newspapers. Standard data cleaning procedures typically neither reduce nor exacerbate these patterns. Implications of these findings are especially important for researchers designing high-frequency surveys of macroeconomic expectations intended to enable identification of the effects of monetary policy announcements or other events of interest via a high frequency approach. Systematic day-of-week variation in respondents' knowledge and reported expectations, if not properly accounted for, could threaten such an identification scheme.

Keywords: Amazon Mechanical Turk, survey experiments, intertemporal effects, social sciences, research methods and design, data collection, economics, psychology, political science, crowdsourcing

JEL Classification: D80, D83, D84, E31

Suggested Citation

Binder, Carola, Time-of-Day and Day-of-Week Variations in Amazon Mechanical Turk Survey Responses (July 6, 2021). Available at SSRN: https://ssrn.com/abstract=3880632 or http://dx.doi.org/10.2139/ssrn.3880632

Carola Binder (Contact Author)

Haverford College - Department of Economics ( email )

Haverford, PA 19041
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Abstract Views
PlumX Metrics