| ||||||||
Table of Contents | ||||||||
POLITICAL METHODS: COMPUTATIONAL eJOURNAL"Methods for Collecting Large-Scale Non-Expert Text Coding"
DREW CONWAY, New York University (NYU) - Department of Politics The task of coding text for discrete categories or quantifiable scales is a classic problem in political science. Traditionally, this task is executed by qualified “experts.� While productive, this method is time consuming, resource intensive, and introduces bias. In the following paper I present the findings from a series of experiments developed to assess the viability of using crowd-sourcing platforms for political text coding, and how variations in the collection mechanism affects the quality of output. To do this, the labor pool available on Amazon’s Mechanical Turk platform were asked to identify policy statements and positions from a text corpus of party manifestos. To evaluate the quality of the the non-expert codings, this text corpus is also coded by multiple experts for comparison. The evidence from these experiments show that crowd-sourcing is an effective alternative means to generating quantitative categorization from text. The presence of a filter on workers increases the quality of output, but variation on that filter have little affect. The primary weakness of the non-experts participating in these experiments is their systematic inability to identify texts that contain no policy statement. | ||||||||
^top
About this eJournal
This eJournal distributes working and accepted paper abstracts. Papers in this area study the formation, structure and function of networks or apply computational methods or algorithmic game theory to understanding politics.
SubmissionsTo submit your research to SSRN, sign in to the SSRN User HeadQuarters, click the My Papers link on left menu and then the Start New Submission button at top of page. Distribution ServicesIf your organization is interested in increasing readership for its research by starting a Research Paper Series, or sponsoring a Subject Matter eJournal, please email: RPS@SSRN.com Distributed byDirectorsPOLITICAL METHODS EJOURNALS DAVID A. LAKE MATHEW D. MCCUBBINS Please contact us at the above addresses with your comments, questions or suggestions for PSN-Sub.
|
|
| | |||||
