The Effects of Diversity in Algorithmic Recommendations on Digital Content Consumption: A Field Experiment

Abstract

Social media platforms such as TikTok and Facebook are criticized for trapping consumers in “filter bubbles” (Pariser 2011) through personalized recommendations based on users’ detailed individual information. Practitioners and regulators have been calling for platforms to tackle the problem by incorporating more diversified content in their recommender systems. We aim to study the causal effects of more diversified personalized recommendations on users’ behaviors in practice. By collaborating with NetEase Cloud Music, the world’s third-largest music-streaming service company, we developed a new recommender system with more content diversity based on their existing state-of-the-art recommender system. We then conducted a large-scale field experiment where hundreds of millions of users were randomly assigned to receive video recommendations either from the platform’s current recommender algorithm or our modified algorithm. Although the new algorithm increased the diversity of recommended content to users, overall there is no clear evidence that it increased the diversity of consumed content, but it decreased users’ consumption level. However, for active users, we find that a 1% increase in recommendation diversity boosted their consumption diversity by 0.55% without reducing their consumption or engagement level. We show that the accuracy of predicting users’ preferences is key for the new algorithm to increase the consumption diversity and, when users also highly value the platform, their consumption and engagement levels will not be hurt. The company eventually adopted our algorithm modification and now uses it to serve millions of customers daily.

Keywords: Recommender Systems, Filter Bubble, Social Media Platforms, Recommendation Diversification, Field Experiment