Using Algorithmic Scores to Measure the Impacts of Targeting Promotional Messages

42 Pages Posted: 19 Dec 2022 Last revised: 7 May 2024

See all articles by Annie Shi

Annie Shi

Washington University in St. Louis - John M. Olin Business School

Tat Chan

Washington University in St. Louis - John M. Olin Business School

Dennis Zhang

Washington University in St. Louis - John M. Olin Business School

Date Written: April 24, 2024

Abstract

Targeting promotions on online platforms are often determined by artificial intelligence (AI) algorithms, which utilize extensive customer and seller information to generate various algorithmic scores for targeting. Effective targeting, however, will lead to selection bias when evaluating the causal effects of promotions. The authors analyzed 2,294 promotion experiments on a major online retail platform and found that traditional methods, such as propensity score matching and double machine learning, cannot accurately recover the true effects using readily available data.

To overcome this challenge, the authors propose an approach that logs and utilizes the algorithmic scores to match treated and untreated customers, effectively mitigating the selection bias and addressing the curse of dimensionality in matching. The authors validate this approach by analyzing the same set of experiments and demonstrate that the estimates from the proposed matching approach, based on algorithmic scores, closely align with the promotion effects estimated from the separately run randomized field experiments. This approach can assist platforms and sellers in accurately evaluating the value of targeted promotions. Additionally, it can be implemented easily and at a low cost since algorithmic scores are easy to store.

Suggested Citation

Shi, Annie and Chan, Tat and Zhang, Dennis, Using Algorithmic Scores to Measure the Impacts of Targeting Promotional Messages (April 24, 2024). Available at SSRN: https://ssrn.com/abstract=4262156 or http://dx.doi.org/10.2139/ssrn.4262156

Annie Shi (Contact Author)

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Tat Chan

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Dennis Zhang

Washington University in St. Louis - John M. Olin Business School ( email )

One Brookings Drive
Campus Box 1133
St. Louis, MO 63130-4899
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
366
Abstract Views
1,259
Rank
174,483
PlumX Metrics