Does Retail Advertising Work? Measuring the Effects of Advertising on Sales Via a Controlled Experiment on Yahoo!

37 Pages Posted: 21 Jun 2011

See all articles by Randall A. Lewis

Randall A. Lewis


David Reiley

Pandora Media, Inc.; UC Berkeley School of Information

Date Written: August 21, 2008


We measure the causal effects of online advertising on sales, using a randomized experiment performed in cooperation between Yahoo! and a major retailer. After identifying over one million customers matched in the databases of the retailer and Yahoo!, we randomly assign them to treatment and control groups. We analyze individual-level data on ad exposure and weekly purchases at this retailer, both online and in stores. We find statistically and economically significant impacts of the advertising on sales. The treatment effect persists for weeks after the end of an advertising campaign, and the total effect on revenues is estimated to be more than seven times the retailer’s expenditure on advertising during the study. Additional results explore differences in the number of advertising impressions delivered to each individual, online and offline sales, and the effects of advertising on those who click the ads versus those who merely view them. Power calculations show that, due to the high variance of sales, our large number of observations brings us just to the frontier of being able to measure economically significant effects of advertising. We also demonstrate that without an experiment, using industry-standard methods based on endogenous cross- sectional variation in advertising exposure, we would have obtained a wildly inaccurate estimate of advertising effectiveness.

Keywords: advertising, field experiment, electronic commerce

JEL Classification: M37, C93, L81

Suggested Citation

Lewis, Randall A. and Reiley, David H., Does Retail Advertising Work? Measuring the Effects of Advertising on Sales Via a Controlled Experiment on Yahoo! (August 21, 2008). Available at SSRN: or

Randall A. Lewis

Netflix ( email )

Los Gatos, CA
United States
312-RA-LEWIS (Phone)


David H. Reiley (Contact Author)

Pandora Media, Inc. ( email )

Oakland, CA 94612
United States

UC Berkeley School of Information ( email )

102 South Hall
Berkeley, CA 94720-4600
United States

Register to save articles to
your library


Paper statistics

Abstract Views
PlumX Metrics