All the Headlines that Are Fit to Change

Abstract

We study how the New York Times changes headlines after an article has been published. We track the 35,000 articles on the newspaper's home page between February 2021 and April 2022 (except July 2021), and find that 10% have an immediate headline change while 11% undergo an A/B test where two titles are trialed against one another over a short time window. We also collect every tweet of an article in our data. We first look at which articles get a headline change of some form. Article type matters: popularity on Twitter, hard news, negative sentiment headline, and mentions from more partisan Twitter accounts are associated with headline changes. Another factor is social pressure, negative tweets from users connected to the reporter or the newspaper, which leads to more immediate headline changes and to a lesser extent to A/B tests. Next we examine how headline changes impact an article's performance metrics. After accounting for selection, we find that headline changes increase an article's popularity and the negative sentiment when it is tweeted, but the impact on the liberal political slant is mixed. We also analyze headline changes at the Wall Street Journal for a shorter period, and find comparable behavior. Our results have implications for distinguishing supply and demand driven models of media bias, how digitization may foster media consolidation, and conditions under which economic or partisan motives drive newspaper decision-making.

Keywords: A/B Testing, Media Economics, Machine Learning, Social Media

JEL Classification: D83, L82