Large Scale Online Text Analysis Using Lydia

35 Pages Posted: 19 Jul 2010 Last revised: 15 Sep 2010

Ellen Key

State University of New York (SUNY) - Department of Political Science

Leonie Huddy

State University of New York (SUNY), Stony Brook

Matthew J. Lebo

State University of New York (SUNY) at Stony Brook - Department of Political Science

Steve Skiena

affiliation not provided to SSRN

Date Written: 2010

Abstract

Political scientists have recognized that the news media serve as an important intervening variable between political issues and public opinion. The media can reinforce existing attitudes (Lazarsfeld, Berelson, and Gaudet 1944), increase the salience of issues (Iyengar and Kinder 1987; Romer, Jamieson, and Aday 2003), and set the agenda for public discourse through the priming and framing of issues (Iyengar 1987; Druckman 2001; Krosnick and Kinder 1990). Media coverage can also aid citizens in the evaluation of political candidates (Aldrich and Alvarez 1994; Ansolabehere and Iyengar 1994; Mendelsohn 1996) and influence voting decisions (Iyengar 1990a; Mendelsohn 1996).

The Lydia project is designed to tame the information tide emanating from news media and blogs in the US (and around the world) by building a relational model of people, places, and things through natural language processing (NLP) of news sources and the statistical analysis of entity frequencies and co-locations. In this paper, we will introduce the Lydia system and its method of collecting and classifying entities, describe each phase of the process, and present a practical application of the system for political scientists by examining the value of the Lydia system in predicting presidential approval for different partisan groups across time. Examining presidential approval as a function of presidential media sentiment and controlling for inflation and unemployment, we find media sentiment toward the president has a significant effect on presidential approval for different partisan groups We also find that media sentiment toward the president responds to changes in presidential approval.

Suggested Citation

Key, Ellen and Huddy, Leonie and Lebo, Matthew J. and Skiena, Steve, Large Scale Online Text Analysis Using Lydia (2010). APSA 2010 Annual Meeting Paper. Available at SSRN: https://ssrn.com/abstract=1642404

Ellen Key (Contact Author)

State University of New York (SUNY) - Department of Political Science ( email )

Stony Brook, 11794-4392
United States

Leonie Huddy

State University of New York (SUNY), Stony Brook ( email )

Stony Brook, NY 11794
United States

Matthew J. Lebo

State University of New York (SUNY) at Stony Brook - Department of Political Science ( email )

Stony Brook, 11794-4392
United States

Steve Skiena

affiliation not provided to SSRN

Paper statistics

Downloads
209
Rank
119,613
Abstract Views
1,287