Notes from a Day on the Forums: Recommendations for Maintaining a Good Reputation as an Amazon Mechanical Turk Requester

3 Pages Posted: 18 May 2015 Last revised: 28 Oct 2015

Antonio Arechar

Massachusetts Institute of Technology (MIT)

Gordon T. Kraft-Todd

Yale University, Faculty of Arts & Sciences, Department of Psychology, Students

David G. Rand

MIT

Date Written: May 2, 2015

Abstract

Social scientists are increasingly collecting data online, and Amazon’s Mechanical Turk (mTurk) service has become a ubiquitous platform for recruiting online subjects. While a growing number of academic journal articles investigate the merits of this methodology, its increasing popularity and the rapidly changing nature of the online community demand frequent monitoring of workers’ feedback to ensure best practices. Thus, we decided to spend “a day at the forums” to learn more about workers' thoughts and feedback, and try to establish some best practices. We found that (i) Turkers consistently preferred short studies with only a handful of questions per page, (ii) Accepting HITs (and paying bonuses) on time is as important as the amount paid, (iii) it is important to keep a close eye on what turkers post on websites such as redditt and turkopticon, (iv) accessing the HIT should be as seamless as possible, (v) and that having transparent procedures for payment and disqualification is important. We also provide a set of links to useful MTurk resources.

Suggested Citation

Arechar, Antonio and Kraft-Todd, Gordon T. and Rand, David G., Notes from a Day on the Forums: Recommendations for Maintaining a Good Reputation as an Amazon Mechanical Turk Requester (May 2, 2015). Available at SSRN: https://ssrn.com/abstract=2602075

Antonio Arechar

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

Gordon T. Kraft-Todd

Yale University, Faculty of Arts & Sciences, Department of Psychology, Students ( email )

P.O. Box 208205
New Haven, CT 06520-8205
United States

David G. Rand (Contact Author)

MIT ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

HOME PAGE: http://www.daverand.org

Register to save articles to
your library

Register

Paper statistics

Downloads
141
Abstract Views
1,085
PlumX