The Unfavorable Economics of Measuring the Returns to Advertising
Randall A. Lewis
Justin M. Rao
September 18, 2014
Twenty-five large field experiments with major U.S. retailers and brokerages, each reaching millions of customers and collectively representing $2.8 million in advertising expenditure, reveal that measuring the returns to advertising is exceedingly difficult. The median confidence interval on ROI is over 100% wide, the smallest exceeds 50%. Detailed sales data show that, relative to the per capita cost of the advertising, individual-level sales are incredibly volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than ten million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The statistically small impact of profitable advertising amid such noise means that selection bias is a crippling concern for widely-employed observational methods. We discuss how these biases and weak informational feedback from experiments fundamentally impact both advertisers and publishers.
Number of Pages in PDF File: 35
Keywords: advertising, field experiments, causal inference, electronic commerce, return on investment, information
JEL Classification: L10, M37, C93
Date posted: December 14, 2013 ; Last revised: September 19, 2014
© 2015 Social Science Electronic Publishing, Inc. All Rights Reserved.
This page was processed by apollo6 in 0.422 seconds