Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions

50 Pages Posted: 16 Nov 2014 Last revised: 21 Nov 2014

Mathew Goldman

UC San Diego

Justin M. Rao

Microsoft Research; Microsoft Corporation - Microsoft Research - Redmond

Date Written: November 20, 2014


Google and Bing employ a generalized second price (GSP) auction to allocate billions of dollars of sponsored search advertising. Despite evolving from a naive, first-price, rank-by-bid auction, the GSP has been shown to achieve an efficient allocation and favorable revenue properties via a tight theoretical link with the equilibrium of the truthful Vickrey-Clarke-Groves mechanism. This link hinges on a critical assumption about the causal impact of ad position on user click probabilities: moving up increases click-through-rate by the same scaling factor for all ads. Because an ad's position is the endogenous outcome of an auction, these assumptions have gone largely untested. We overcome this identification problem by developing a novel method to re-purpose internal business experimentation at a major search engine. Using a broad cross-section of advertisers we strongly reject the conventional multiplicatively-separable model, instead finding substantial heterogeneity of the causal impact of position on click-through-rates. For brand queries, off-brand competitors benefit much more from position than the advertiser matching the query. For generic queries, higher quality and less-well known websites benefit more from position. The impact of position differs by up to 100% across ad types. These findings raise serious concerns about the efficiency properties of the GSP as currently employed.

Keywords: sponsored search, mechanism design, instrumental variables, regularization

JEL Classification: C57

Suggested Citation

Goldman, Mathew and Rao, Justin M., Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions (November 20, 2014). Available at SSRN: https://ssrn.com/abstract=2524688 or http://dx.doi.org/10.2139/ssrn.2524688

Mathew Goldman

UC San Diego ( email )

United States

Justin M. Rao (Contact Author)

Microsoft Research ( email )

641 Avenue of Americas
7th Floor
New York, NY 11249
United States

Microsoft Corporation - Microsoft Research - Redmond ( email )

Building 99
Redmond, WA
United States

Paper statistics

Abstract Views