Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis

Ceren Budak

University of Michigan at Ann Arbor

Sharad Goel

Stanford University

Justin M. Rao

Microsoft Research; Microsoft Corporation - Microsoft Research - Redmond

November 17, 2014

It is widely thought that news organizations exhibit ideological bias, but rigorously quantifying such slant has proven methodologically challenging. Through a combination of machine learning and crowdsourcing techniques, we investigate the selection and framing of political issues in 15 major U.S. news outlets. Starting with 803,146 news stories published over 12 months, we first used supervised learning algorithms to identify the 14% of articles pertaining to political events. We then recruited 749 online human judges to classify a random subset of 10,950 of these political articles according to topic and ideological position. Our analysis yields an ideological ordering of outlets consistent with prior work. We find, however, that news outlets are considerably more similar than generally believed. Specifically, with the exception of political scandals, we find that major news organizations present topics in a largely non-partisan manner, casting neither Democrats nor Republicans in a particularly favorable or unfavorable light. Moreover, again with the exception of political scandals, there is little evidence of systematic differences in story selection, with all major news outlets covering a wide variety of topics with frequency largely unrelated to the outlet's ideological position. Finally, we find that news organizations express their ideological bias not by directly advocating for a preferred political party, but rather by disproportionately criticizing one side, a convention that further moderates overall differences.

Number of Pages in PDF File: 37

Keywords: media bias, text analysis, crowdsourcing, big data

JEL Classification: D83, L10, L82

Open PDF in Browser Download This Paper

Date posted: November 19, 2014 ; Last revised: March 11, 2016

Suggested Citation

Budak, Ceren and Goel, Sharad and Rao, Justin M., Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis (November 17, 2014). Available at SSRN: https://ssrn.com/abstract=2526461 or http://dx.doi.org/10.2139/ssrn.2526461

Contact Information

Ceren Budak (Contact Author)
University of Michigan at Ann Arbor ( email )
110 Tappan Hall
855 S. University Ave
Ann Arbor, MI 48109
United States
Sharad Goel
Stanford University ( email )
475 Via Ortega
Stanford, CA 94305
United States
HOME PAGE: http://5harad.com
Justin M. Rao
Microsoft Research ( email )
641 Avenue of Americas
7th Floor
New York, NY 11249
United States
Microsoft Corporation - Microsoft Research - Redmond ( email )
Building 99
Redmond, WA
United States
Feedback to SSRN

Paper statistics
Abstract Views: 4,795
Downloads: 889
Download Rank: 19,438
Paper comments
No comments have been made on this paper