Using Algorithmic Scores to Measure the Impacts of Targeting Promotional Messages

29 Pages Posted: 19 Dec 2022

See all articles by Annie Shi

Annie Shi

Washington University in St. Louis - John M. Olin Business School

Dennis Zhang

Washington University in St. Louis - John M. Olin Business School

Tat Chan

Washington University in St. Louis - John M. Olin Business School

Haoyuan Hu

Alibaba Group

Binqiang Zhao

Alibaba Group

Date Written: October 30, 2022

Abstract

Nowadays, managers' targeting promotion decisions are often facilitated by machine learning (ML) algorithms. These algorithms often aggregate a large amount of customer demographic and behavioral information into several personalized scores, based on which managers then algorithmically make targeting or pricing decisions. We propose matching on these ML-generated scores used in targeting decisions to measure the effectiveness of targeting promotions. Our strategy can effectively address the issue of selection bias introduced by personalized targeting, and it could overcome the curse of dimensionality in matching when these ML systems use a large amount of consumer information. In order to test our proposed approach, we conducted a large field experiment on targeting promotions with a large retailing platform and analyzed customer behaviors. In our empirical application, customers are divided into two groups: the experimental group, where customers are randomly assigned to the treatment and control for receiving the targeting promotion, and the non-experimental (i.e., observational) group, where the algorithm will decide who receives the promotion. We first demonstrate that receiving the targeting promotions successfully increases store visits and purchases but also increases customers' likelihood of unsubscribing from the promotional message service. We then compare the results from experimental data and those by our matching approach using the non-experimental data. We showed that our proposed matching approach could effectively recover the same effects as the experimental data. By contrast, the estimated effects without matching are seriously over-biased, and traditional matching methods cannot mitigate such biases due to the curse of dimensionality.

Suggested Citation

Shi, Annie and Zhang, Dennis and Chan, Tat and Hu, Haoyuan and Zhao, Binqiang, Using Algorithmic Scores to Measure the Impacts of Targeting Promotional Messages (October 30, 2022). Available at SSRN: https://ssrn.com/abstract=4262156 or http://dx.doi.org/10.2139/ssrn.4262156

Annie Shi (Contact Author)

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Dennis Zhang

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Tat Chan

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Haoyuan Hu

Alibaba Group ( email )

Binqiang Zhao

Alibaba Group ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
45
Abstract Views
171
PlumX Metrics