Fighting Misinformation on Social Media Using Crowdsourced Judgments of News Source Quality

Pennycook, G., & Rand. D. G. (2019). Fighting misinformation on social media using crowdsourced judgments of news source quality. Proceedings of the National Academy of Sciences. DOI: 10.1073/pnas.1806781116

85 Pages Posted: 17 Feb 2018 Last revised: 31 Jan 2019

See all articles by Gordon Pennycook

Gordon Pennycook

University of Regina

David G. Rand

Massachusetts Institute of Technology (MIT)

Date Written: January 29, 2019

Abstract

Reducing the spread of misinformation, especially on social media, is a major challenge. We investigate one potential approach: having social media platform algorithms preferentially display content from news sources which users rate as trustworthy. To do so, we ask whether crowdsourced trust ratings can effectively differentiate more versus less reliable sources. We ran two preregistered experiments (N=1,010 from Mechanical Turk and N=970 from Lucid) where individuals rated familiarity with, and trust in, 60 news sources from three categories: 1) mainstream media outlets, 2) hyper-partisan websites, and 3) websites that produce blatantly false content (“fake news”). Despite substantial partisan differences, we find that laypeople across the political spectrum rated mainstream sources as far more trustworthy than either hyper-partisan or fake news sources. Although this difference was larger for Democrats than Republicans – mostly due to distrust of mainstream sources by Republicans – every mainstream source (with one exception) was rated as more trustworthy than every hyper-partisan or fake news source across both studies when equally weighting ratings of Democrats and Republicans. Furthermore, politically balanced layperson ratings were strongly correlated (r=0.90) with ratings provided by professional fact-checkers. We also found that, particularly among liberals, individuals higher in cognitive reflection were better able to discern between low- and high-quality sources. Finally, we found that excluding ratings from participants who were not familiar with a given news source dramatically reduced the effectiveness of the crowd. Our findings indicate that having algorithms up-rank content from trusted media outlets may be a promising approach for fighting the spread of misinformation on social media.

Keywords: fake news, news media, social media, media trust, misinformation

Suggested Citation

Pennycook, Gordon and Rand, David G., Fighting Misinformation on Social Media Using Crowdsourced Judgments of News Source Quality (January 29, 2019). Pennycook, G., & Rand. D. G. (2019). Fighting misinformation on social media using crowdsourced judgments of news source quality. Proceedings of the National Academy of Sciences. DOI: 10.1073/pnas.1806781116. Available at SSRN: https://ssrn.com/abstract=3118471 or http://dx.doi.org/10.2139/ssrn.3118471

Gordon Pennycook (Contact Author)

University of Regina ( email )

3737 Wascana Parkway
Regina, Saskatchewan S4S OA2 S4S 0A1
Canada

David G. Rand

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

HOME PAGE: http://www.daverand.org

Register to save articles to
your library

Register

Paper statistics

Downloads
1,792
Abstract Views
9,399
rank
8,544
PlumX Metrics