Adaptive Data Acquisition for Personalized Recommender Systems with Optimality Guarantees on Short-Form Video Platforms

48 Pages Posted: 11 May 2021 Last revised: 28 May 2022

See all articles by Junyu Cao

Junyu Cao

University of Texas at Austin - Red McCombs School of Business

Yan Leng

University of Texas at Austin - Red McCombs School of Business

Date Written: May 9, 2021

Abstract

The popularity of short-form video (SFV) has been exploding on digital platforms recently. The large number of videos and evolving topics on digital platforms pose technical challenges in making personalized recommendations. In this paper, we introduce a new pure exploration problem on SFV platforms for finding a $(K, \epsilon_H, \epsilon_L)$-optimal set comprising all recommendations that are within $\epsilon_L$-optimality gap and excluding all recommendations outside of the $\epsilon_H$-optimality gap, relative to the best arm with capacity at most K, for quality control. We propose an adaptive data acquisition method, called Adaptive Acquisition Tree (AAT), to jointly account for user preference heterogeneity and high-dimensional product characteristics. AAT adaptively segments users based on preference similarity and then learns a personalized transductive bandit policy that can be used on partially observed or even unobserved items to accommodate the dynamic trends on SFV platforms. We derive the sample complexity for identifying a $(K, \epsilon_H, \epsilon_L)$-optimal set for a single user and for all users, respectively. We further evaluate the algorithm via numerical experiments on data collected from the NetEase platform. Our result demonstrates that the proposed policy performs significantly better than several state-of-the-art benchmarks in four transductive scenarios for both spotlight recommendations (i.e., best-arm identifications) and $(K, \epsilon_H, \epsilon_L)$-optimal set recommendations. With the potential to improve the expected view time by 20-25%, our method produces both academic and practical value, given the increasing popularity and uniqueness of SFVs and, more broadly, user-generated content.

Keywords: short-form video, pure exploration, information acquisition, transductive learning, contextual bandits; interpretability; short-form video platforms; recommender systems

Suggested Citation

Cao, Junyu and Leng, Yan, Adaptive Data Acquisition for Personalized Recommender Systems with Optimality Guarantees on Short-Form Video Platforms (May 9, 2021). Available at SSRN: https://ssrn.com/abstract=3843086 or http://dx.doi.org/10.2139/ssrn.3843086

Junyu Cao (Contact Author)

University of Texas at Austin - Red McCombs School of Business ( email )

Austin, TX
United States

Yan Leng

University of Texas at Austin - Red McCombs School of Business ( email )

Austin, TX
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
218
Abstract Views
837
Rank
240,693
PlumX Metrics