Sequential Choice Bandits: Learning with Marketing Fatigue

49 Pages Posted: 8 Apr 2019 Last revised: 25 Nov 2019

See all articles by Junyu Cao

Junyu Cao

University of California, Berkeley

Wei Sun

IBM Corporation - Thomas J. Watson Research Center

Zuo-Jun Max Shen

University of California, Berkeley - Department of Industrial Engineering & Operations Research (IEOR)

Date Written: March 18, 2019

Abstract

Motivated by the observation that overexposure to unwanted marketing activities can lead to customer dissatisfaction, we consider a setting where a platform offers a sequence of messages to its users and is penalized when users abandon the platform due to marketing fatigue. We propose a novel sequential choice model to capture multiple interactions taking place between the platform and its users: upon receiving a message, a user decides on whether to accept or reject the message. If she chooses to reject, she would then decide to either receive the next message in the sequence or abandon the platform. Based on user feedback, the platform dynamically learns users' abandonment distribution and the relevance of the recommended content. With a goal to maximize the cumulative payoff over a horizon of length T, the platform dynamically adjusts the sequence of messages and the order in which the messages are shown to a user. We refer to this online learning task as the sequential choice bandit (SC-Bandit) problem. For the offline combinatorial optimization problem, we show a polynomial-time algorithm. For the online problem, we consider two variants, depending on whether contexts are included, and propose algorithms that balance exploration and exploitation. Lastly, we evaluate the performance of our algorithms with both synthetic and real-world datasets.

Keywords: sequential choice, learning to rank, marketing fatigue, online learning, bandit

Suggested Citation

Cao, Junyu and Sun, Wei and Shen, Zuo-Jun Max, Sequential Choice Bandits: Learning with Marketing Fatigue (March 18, 2019). Available at SSRN: https://ssrn.com/abstract=3355211 or http://dx.doi.org/10.2139/ssrn.3355211

Junyu Cao (Contact Author)

University of California, Berkeley ( email )

4141 Etcheverry Hall
Berkeley, CA 94720
United States

Wei Sun

IBM Corporation - Thomas J. Watson Research Center ( email )

Route 134
Kitchawan Road
Yorktown Heights, NY 10598
United States

Zuo-Jun Max Shen

University of California, Berkeley - Department of Industrial Engineering & Operations Research (IEOR) ( email )

IEOR Department
4135 Etcheverry Hall
Berkeley, CA 94720
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
108
Abstract Views
572
rank
257,427
PlumX Metrics