MTurk Workers' Use of Low-Cost 'Virtual Private Servers' to Circumvent Screening Methods: A Research Note

21 Pages Posted: 23 Aug 2018 Last revised: 14 Sep 2018

See all articles by Sean A. Dennis

Sean A. Dennis

University of Kentucky

Brian M. Goodson

Clemson University

Chris Pearson

University of Kentucky

Date Written: August 17, 2018

Abstract

Emerging literature examines the integrity of MTurk-based samples in social science research and develops techniques to ensure this data is reliable. We identify a pervasive, yet previously undocumented threat to the integrity of MTurk data and propose techniques to mitigate this threat. MTurk samples for two independent studies included alarming proportions of participants who circumvented several accepted sample screening methods. These ‘imposters’ provided responses through virtual private servers (VPS) that concealed their physical locations, thereby rendering conventional location screening methods useless. VPSs also allow individuals to submit multiple responses from the same device without detection. We identified this issue by investigating large sets of unique IP addresses with identical GPS coordinates; each such IP address traces back to a server farm and VPS provider. VPS participants provide disturbingly low-quality open responses. This highlights the importance of considering VPS use, in conjunction with established screening procedures, when working with MTurk samples.

Keywords: Amazon Mechanical Turk; online experiments; participant screening; online labor markets; TurkPrime; virtual private servers

Suggested Citation

Dennis, Sean A. and Goodson, Brian Matthew and Pearson, Chris, MTurk Workers' Use of Low-Cost 'Virtual Private Servers' to Circumvent Screening Methods: A Research Note (August 17, 2018). Available at SSRN: https://ssrn.com/abstract=3233954 or http://dx.doi.org/10.2139/ssrn.3233954

Sean A. Dennis (Contact Author)

University of Kentucky ( email )

Lexington, KY 40506
United States

Brian Matthew Goodson

Clemson University ( email )

Clemson, SC 29631
United States

Chris Pearson

University of Kentucky ( email )

Lexington, KY 40506
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
1,162
rank
16,131
Abstract Views
6,226
PlumX Metrics