Using Algorithmic Scores to Measure the Impacts of Targeting Promotional Messages

Abstract

Targeting promotions on online platforms are often determined by artificial intelligence (AI) algorithms, which utilize extensive customer and seller information to generate various algorithmic scores for targeting. Effective targeting, however, will lead to selection bias when evaluating the causal effects of promotions. The authors analyzed 2,294 promotion experiments on a major online retail platform and found that traditional methods, such as propensity score matching and double machine learning, cannot accurately recover the true effects using readily available data.

To overcome this challenge, the authors propose an approach that logs and utilizes the algorithmic scores to match treated and untreated customers, effectively mitigating the selection bias and addressing the curse of dimensionality in matching. The authors validate this approach by analyzing the same set of experiments and demonstrate that the estimates from the proposed matching approach, based on algorithmic scores, closely align with the promotion effects estimated from the separately run randomized field experiments. This approach can assist platforms and sellers in accurately evaluating the value of targeted promotions. Additionally, it can be implemented easily and at a low cost since algorithmic scores are easy to store.