How Does Competition Affect Exploration vs. Exploitation? A Tale of Two Recommendation Algorithms

50 Pages Posted:

See all articles by Huining Henry Cao

Huining Henry Cao

Cheung Kong Graduate School of Business

Liye Ma

University of Maryland - Department of Marketing

Z. Eddie Ning

Cheung Kong Graduate School of Business

Baohong Sun

Cheung Kong Graduate School of Business (New York)

Date Written: November 30, 2020

Abstract

Through repeated interactions with users, firms today refine their understanding of individual users' preferences adaptively for personalized targeting and recommendation. In this paper, we use a continuous-time bandit model to analyze firms that supply content to consumers, a representative setting for strategic learning of consumer preferences to maximize lifetime value. We compare a forward-looking recommendation algorithm that balances exploration and exploitation to a myopic algorithm that only maximizes the current quality of the recommendation in both monopoly and duopoly settings. Our analysis shows that competition can discourage learning. In a duopoly where firms compete for consumers' attention, firms focus more on exploitation than exploration in their recommendations than a monopoly would. Competition increases firms' incentives to develop myopic algorithms but decreases their incentives to develop forward-looking algorithms when users are impatient. Development of the optimal forward-looking algorithm may hurt users under monopoly but benefits users under competition. We are among the first to examine and compare the equilibrium of this multi-agent bandit problem under different competitive scenarios, and our results provide implications for firms on the adoption of AI strategy as well as for policy makers on the effect of market power on innovation and consumer welfare.

Keywords: AI, multi-agent bandit, recommendation algorithm, innovation, competition, reinforcement learning, experimentation, CLV, value of learning, forward-looking optimization

JEL Classification: C73, D40, D83, L10, M31

Suggested Citation

Cao, Huining Henry and Ma, Liye and Ning, Z. Eddie and Sun, Baohong, How Does Competition Affect Exploration vs. Exploitation? A Tale of Two Recommendation Algorithms (November 30, 2020). Available at SSRN: https://ssrn.com/abstract=

Huining Henry Cao

Cheung Kong Graduate School of Business ( email )

Oriental Plaza, Tower E3
One East Chang An Avenue
Beijing, 100738
China

Liye Ma

University of Maryland - Department of Marketing ( email )

College Park, MD 20742
United States
(301) 405-8982 (Phone)

Z. Eddie Ning (Contact Author)

Cheung Kong Graduate School of Business ( email )

Oriental Plaza, Tower E3
One East Chang An Avenue
Beijing, 100738
China

HOME PAGE: http://www.eddiening.com

Baohong Sun

Cheung Kong Graduate School of Business (New York) ( email )

Oriental Plaza, Tower E3
One East Chang An Avenue
Beijing, 100738
China

HOME PAGE: http://english.ckgsb.edu.cn/faculty/sun-baohong/

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
30
Abstract Views
81
PlumX Metrics