Ad Attributes and Attribution: Large-Scale Field Experiments Measure Online Customer Acquisition

24 Pages Posted: 2 May 2012

See all articles by David Reiley

David Reiley

Pandora Media, Inc.; UC Berkeley School of Information

Randall A. Lewis

Amazon

Taylor Schreiner

affiliation not provided to SSRN

Date Written: May 1, 2012

Abstract

Using a large-scale experiment involving 3.7 million treated subjects on Yahoo!, we measure the ability of online display advertising to cause new account sign-ups at an online business. We experiment with two dimensions of media choice: banner-shaped ads versus large rectangular ads, and Yahoo! Mail placement versus Yahoo! Run-of-Network placement. The Run-of-Network advertisements succeeded in generating a statistically significant increase in sign-ups of approximately 11% relative to the control group, but the Yahoo! Mail ads did not produce a statistically significant increase in sign-ups for this advertiser. By contrast with the results for choice of pages, we find no effect of the form factor of the ads (banner versus large rectangular) in this campaign. We note the somewhat surprising fact that even with experimental sample sizes in the millions of subjects, our estimates remain quite noisy: for example, the upper bound of the 95% confidence interval for the statistically insignificant Yahoo! Mail campaign represents a 15% increase in new customers. Most importantly, our estimates call into question click-only attribution models, as restricting attention to the number of users that clicked on an ad and converted is less than 30% of the estimated treatment effect on conversions.

Keywords: Online advertising, advertising effectiveness, field experiment, click attribution

JEL Classification: C93, M37, L81

Suggested Citation

Reiley, David H. and Lewis, Randall A. and Schreiner, Taylor, Ad Attributes and Attribution: Large-Scale Field Experiments Measure Online Customer Acquisition (May 1, 2012). Available at SSRN: https://ssrn.com/abstract=2049457 or http://dx.doi.org/10.2139/ssrn.2049457

David H. Reiley (Contact Author)

Pandora Media, Inc. ( email )

2101 WEBSTER ST 16TH FLOOR
Oakland, CA 94612
United States

UC Berkeley School of Information ( email )

102 South Hall
Berkeley, CA 94720-4600
United States

Randall A. Lewis

Amazon ( email )

312-RA-LEWIS (Phone)

Taylor Schreiner

affiliation not provided to SSRN ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
466
Abstract Views
2,884
Rank
127,624
PlumX Metrics