Fighting Misinformation on Social Media Using Crowdsourced Judgments of News Source Quality
Pennycook, G., & Rand. D. G. (2019). Fighting misinformation on social media using crowdsourced judgments of news source quality. Proceedings of the National Academy of Sciences. DOI: 10.1073/pnas.1806781116
85 Pages Posted: 17 Feb 2018 Last revised: 31 Jan 2019
Date Written: January 29, 2019
Reducing the spread of misinformation, especially on social media, is a major challenge. We investigate one potential approach: having social media platform algorithms preferentially display content from news sources which users rate as trustworthy. To do so, we ask whether crowdsourced trust ratings can effectively differentiate more versus less reliable sources. We ran two preregistered experiments (N=1,010 from Mechanical Turk and N=970 from Lucid) where individuals rated familiarity with, and trust in, 60 news sources from three categories: 1) mainstream media outlets, 2) hyper-partisan websites, and 3) websites that produce blatantly false content (“fake news”). Despite substantial partisan differences, we find that laypeople across the political spectrum rated mainstream sources as far more trustworthy than either hyper-partisan or fake news sources. Although this difference was larger for Democrats than Republicans – mostly due to distrust of mainstream sources by Republicans – every mainstream source (with one exception) was rated as more trustworthy than every hyper-partisan or fake news source across both studies when equally weighting ratings of Democrats and Republicans. Furthermore, politically balanced layperson ratings were strongly correlated (r=0.90) with ratings provided by professional fact-checkers. We also found that, particularly among liberals, individuals higher in cognitive reflection were better able to discern between low- and high-quality sources. Finally, we found that excluding ratings from participants who were not familiar with a given news source dramatically reduced the effectiveness of the crowd. Our findings indicate that having algorithms up-rank content from trusted media outlets may be a promising approach for fighting the spread of misinformation on social media.
Keywords: fake news, news media, social media, media trust, misinformation
Suggested Citation: Suggested Citation