Network-Enabled Sequential Data Acquisition for High-Dimensional Recommender Systems

Abstract

Consumer data are strategic assets for digital platforms. High-dimensional matrix completion is pervasive for e-commerce platforms, with notable examples including recommender systems, consumer profiling, and assortment personalization. We formulate a new sequential data acquisition problem for the high-dimensional recommender system to maximize the decision-centric cumulative utility for the platform. We propose to jointly consider the utility and the uncertainty reduction based on the entropy of a single data acquisition. Rooted in the information theory and three social network theories (i.e., homophily, structural equivalence, and information exposure), we develop DU-Net (Data-Utility NETwork-amplified) algorithm that exploits data network, static properties of and dynamic processes on social networks. To better estimate the uncertainty reduction, we develop a flexible and dynamic locally-smooth mechanism, which aggregates the low-dimensional node representation to capture homophily and structural equivalence, with varying aggregation strength depending on data density and estimation errors. To better predict the utility of a single acquisition action, we further incorporate the information exposure. We conduct an extensive evaluation with three canonical real-world recommendation datasets and show that DU-Net outperforms prevalent methods from representative previous research and salient industry practices. Our paper extends the data acquisition literature to consider data and social network effect jointly and shows that these two networks can effectively boost the cumulative utility in high-dimensional learning problems on digital platforms.

Keywords: Data acquisition; high-dimensional data; recommender system; design science; social network