On the Near Impossibility of Measuring the Returns to Advertising
Randall A Lewis
Justin M. Rao
December 12, 2013
Classical theories assume the firm has access to reliable signals to measure the causal impact of choice variables on profit. For advertising expenditure we show, using twenty-five online field experiments with major U.S. retailers and brokerages ($2.8 million expenditure), that this assumption typically does not hold. Evidence from the randomized trials is very weak because individual-level sales are incredibly volatile relative to the per capita cost of a campaign -- a "small'' impact on a noisy dependent variable can generate positive returns. A calibrated statistical argument shows that the required sample size for an experiment to generate informative confidence intervals is typically in excess of ten million person-weeks. This also implies that selection bias unaccounted for by observational methods only needs to explain a tiny fraction of sales variation to severely bias observational estimates. We discuss how weak informational feedback has shaped the current marketplace and the impact of technological advances moving forward.
Number of Pages in PDF File: 42
Keywords: advertising, field experiments, causal inference, electronic commerce, return on investment, information
JEL Classification: L10, M37, C93working papers series
Date posted: December 14, 2013
© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.
This page was processed by apollo3 in 0.344 seconds