Information Signals in Sponsored Search: Evidence from Google's BERT

Abstract

We study how improvements to search engine interpretation algorithms and the information signals they generate affect sponsored search markets. We focus on two outcomes: the number of advertisers bidding for a query (i.e., competition) and cost-per-click (CPC). We start by developing a theoretical auction model. We find that as the quality of a search engine's interpretation algorithm improves, the number of bidders allocated to auctions generally increases for all queries. Despite this, we find that changes in CPC depend on the prevalence of context in a query. For queries lacking context (e.g., shorter queries), CPC increases. However, CPC may decrease for queries with more contextual information (e.g., longer queries). This can occur when the new algorithm significantly improves contextual interpretation capabilities, leading to more precise advertiser relevancy scores. These scores can lower CPC but increase the average click-through rate (CTR). We then test the model predictions using a monthly dataset of competition scores and CPC for 12,000 queries, leveraging Google's October 2019 rollout of Bidirectional Encoder Representations from Transformers as a natural experiment. We find results consistent with the theoretical model. Our results provide insight into the economic impact of AI and Large Language Models on advertising markets.

Keywords: Large Language Models, Sponsored Search Advertising, Auction Markets, BERT, Advertising