Measuring and Explaining Political Sophistication Through Textual Complexity

47 Pages Posted: 1 Nov 2017 Last revised: 25 May 2018

Kenneth Benoit

London School of Economics & Political Science (LSE); Trinity College Dublin

Kevin Munger

New York University (NYU)

Arthur Spirling

New York University

Date Written: May 22, 2018

Abstract

Political scientists lack domain-specific measures for the purpose of measuring the sophistication of political communication. We systematically review the shortcomings of existing approaches, before developing a new and better method along with software tools to apply it. We use crowdsourcing to perform thousands of pairwise comparisons of text snippets and incorporate these results into a statistical model of sophistication. This includes previously excluded features such as parts of speech, and a measure of word rarity derived from dynamic term frequencies in the Google books dataset. Our technique not only shows which features are appropriate to the political domain and how, but also provides a measure easily applied and re-scaled to political texts in a way that facilitates probabilistic comparisons. We reanalyze the State of the Union corpus to demonstrate how conclusions differ when using our improved approach, including the ability to compare complexity as a function of covariates.

Keywords: sophistication, text as data, political methodology, pairwise contests, crowdsourcing

Suggested Citation

Benoit, Kenneth and Munger, Kevin and Spirling, Arthur, Measuring and Explaining Political Sophistication Through Textual Complexity (May 22, 2018). Available at SSRN: https://ssrn.com/abstract=3062061 or http://dx.doi.org/10.2139/ssrn.3062061

Kenneth Benoit

London School of Economics & Political Science (LSE) ( email )

Houghton Street
London, WC2A 2AE
United Kingdom

Trinity College Dublin ( email )

3 College Green
Trinity College
Dublin, 2
Ireland

Kevin Munger

New York University (NYU) ( email )

New York, NY
United States

Arthur Spirling (Contact Author)

New York University ( email )

19 West 4th Street
New York, NY 10012
United States

HOME PAGE: http://https://www.nyu.edu/projects/spirling/

Register to save articles to
your library

Register

Paper statistics

Downloads
296
rank
94,389
Abstract Views
1,437
PlumX