Deep Reinforcement Learning for Sequential Targeting

Management Science, forthcoming

48 Pages Posted: 5 Dec 2019 Last revised: 3 May 2022

See all articles by Wen Wang

Wen Wang

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management

Beibei Li

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management

Xueming Luo

Temple University

xiao yi wang

affiliation not provided to SSRN

Date Written: December 5, 2021

Abstract

Deep reinforcement learning (DRL) has opened up many unprecedented opportunities in revolutionizing the digital marketing field. In this study, we designed a DRL-based personalized targeting strategy in a sequential setting. We show that the strategy is able to address three important challenges of sequential targeting. 1) Forward-looking: balancing between a firm’s current revenue and future revenues; 2) Earning-while-learning: maximizing profits while continuously learning through exploration-exploitation; 3) Scalability: coping with a high-dimensional state and policy space. We illustrate the above through a novel design of a DRL-based artificial intelligence (AI) agent. To better adapt DRL to complex consumer behavior dimensions, we proposed a quantization-based uncertainty learning heuristic for efficient exploration-exploitation. Our policy evaluation results through simulation suggest that the proposed DRL agent generates 26.75% more long-term revenues than can the non-DRL approaches on average, and learns 76.92% faster than the second-fastest model among all benchmarks. Further, in order to better understand the potential underlying mechanisms, we conducted multiple interpretability analyses to explain the patterns of learned optimal policy at both the individual and population levels. Our findings provide important managerial-relevant and theory-consistent insights. For instance, consecutive price promotions at the beginning can capture price-sensitive consumers’ immediate attention, while carefully spaced non-promotional “cool-down” periods between price promotions can allow consumers to adjust their reference points. Besides, consideration of future revenues is necessary from a long-term horizon, but weighing the future too much can also dampen revenues. In addition, analyses of heterogeneous treatment effects suggest that the optimal promotion sequence pattern highly varies across the consumer engagement stages. Overall, our study results demonstrate DRL’s potential to optimize these strategies’ combinations to maximize long-term revenues.

Keywords: Deep Reinforcement Learning, Field experiments, Targeting, Promotions, Sequential actions, Mobile

Suggested Citation

Wang, Wen and Li, Beibei and Luo, Xueming and wang, xiao yi, Deep Reinforcement Learning for Sequential Targeting (December 5, 2021). Management Science, forthcoming, Available at SSRN: https://ssrn.com/abstract=3487145 or http://dx.doi.org/10.2139/ssrn.3487145

Wen Wang (Contact Author)

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management ( email )

Pittsburgh, PA 15213
United States

Beibei Li

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management ( email )

Pittsburgh, PA 15213-3890
United States

Xueming Luo

Temple University ( email )

1810 N. 13th Street
Floor 2
Philadelphia, PA 19128
United States

HOME PAGE: http://www.fox.temple.edu/mcm_people/xueming-luo/

Xiao yi Wang

affiliation not provided to SSRN

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
911
Abstract Views
2,465
rank
37,290
PlumX Metrics