Ad Attributes and Attribution: Large-Scale Field Experiments Measure Online Customer Acquisition

24 Pages Posted: 2 May 2012  

David Reiley Jr.

Google, Inc.

Randall A. Lewis

Netflix

Taylor A. Schreiner

affiliation not provided to SSRN

Date Written: May 1, 2012

Abstract

Using a large-scale experiment involving 3.7 million treated subjects on Yahoo!, we measure the ability of online display advertising to cause new account sign-ups at an online business. We experiment with two dimensions of media choice: banner-shaped ads versus large rectangular ads, and Yahoo! Mail placement versus Yahoo! Run-of-Network placement. The Run-of-Network advertisements succeeded in generating a statistically significant increase in sign-ups of approximately 11% relative to the control group, but the Yahoo! Mail ads did not produce a statistically significant increase in sign-ups for this advertiser. By contrast with the results for choice of pages, we find no effect of the form factor of the ads (banner versus large rectangular) in this campaign. We note the somewhat surprising fact that even with experimental sample sizes in the millions of subjects, our estimates remain quite noisy: for example, the upper bound of the 95% confidence interval for the statistically insignificant Yahoo! Mail campaign represents a 15% increase in new customers. Most importantly, our estimates call into question click-only attribution models, as restricting attention to the number of users that clicked on an ad and converted is less than 30% of the estimated treatment effect on conversions.

Keywords: Online advertising, advertising effectiveness, field experiment, click attribution

JEL Classification: C93, M37, L81

Suggested Citation

Reiley, David and Lewis, Randall A. and Schreiner, Taylor A., Ad Attributes and Attribution: Large-Scale Field Experiments Measure Online Customer Acquisition (May 1, 2012). Available at SSRN: https://ssrn.com/abstract=2049457 or http://dx.doi.org/10.2139/ssrn.2049457

David H. Reiley Jr. (Contact Author)

Google, Inc. ( email )

1600 Amphitheatre Parkway
Second Floor
Mountain View, CA 94043
United States

Randall A. Lewis

Netflix ( email )

Los Gatos, CA
United States
312-RA-LEWIS (Phone)

HOME PAGE: http://www.econinformatics.com/

Taylor A. Schreiner

affiliation not provided to SSRN ( email )

Paper statistics

Downloads
186
Rank
133,590
Abstract Views
988