Download This PaperWhere A-B Testing Goes Wrong: How Divergent Delivery Affects What Online Experiments Cannot (and Can) Tell You About How Customers Respond to Advertising
51 Pages Posted: 30 Jul 2021 Last revised: 5 May 2024
Date Written: May 2, 2024
Abstract
Marketers use online advertising platforms to compare user responses to different ad content. However, platforms’ experimentation tools deliver ads to distinct, optimized, undetectable mixes of users that vary across ads, even during the test. As a result, the estimated
Keywords: Targeted online advertising, A/B testing, measuring advertising effectiveness, causal inference, experimental design, Simpson's paradox, social media
JEL Classification: C9, M31, M37
Suggested Citation: Suggested Citation
Braun, Michael and Schwartz, Eric M., Where A-B Testing Goes Wrong: How Divergent Delivery Affects What Online Experiments Cannot (and Can) Tell You About How Customers Respond to Advertising (May 2, 2024). SMU Cox School of Business Research Paper No. 21-10, Available at SSRN: https://ssrn.com/abstract=3896024 or http://dx.doi.org/10.2139/ssrn.3896024
Eric M. Schwartz
University of Michigan, Stephen M. Ross School of Business ( email )
701 Tappan Street
Ann Arbor, MI MI 48109
United States
Do you have a job opening that you would like to promote on SSRN?
Feedback
Feedback to SSRN