The Language That Drives Engagement: A Systematic Large-scale Analysis of Headline Experiments.

Abstract

We use a large-scale dataset of thousands of field experiments conducted on Upworthy.com, an online media platform, to investigate the cognitive, motivational, affective, and grammatical factors implementable in messages that increase engagement with online content. We map from textual cues measured with Natural Language Processing tools to constructs implied to be relevant by a broad range of prior research literatures. We validate the constructs with human judgment and then test which constructs causally impact click-through to articles when implemented in headlines. Our findings suggest that the use of textual cues identified in previous research and industry advice does impact the effectiveness of headlines overall, but the prior research and industry advice does not always provide useful guidance as to the direction of the effects. We identify which textual characteristics make headlines most effective at motivating engagement in our online news setting.

Keywords: Natural Language Processing; Information Processing; Cognitive Psychology; Linguistic Process; Social Psychology; A/B Testing; Text Analysis