Learning Product Characteristics and Consumer Preferences from Search Data
47 Pages Posted: 3 Jun 2021 Last revised: 8 Oct 2021
Date Written: June 1, 2021
A building block of many models in empirical industrial organization is a characteristic space, where products are modeled as a bundle of characteristics over which consumers have preferences. The ability of such models to predict counterfactual outcomes depends on how well this characteristic space representation can capture substitution patterns. A limitation of existing methods is that product characteristics must be observable. In this paper, we extend a machine learning approach (Bayesian Personalized Ranking) that allows us to jointly learn latent product characteristics and consumer preferences from search data. We then show how this can be combined with existing demand estimation approaches to predict demand. Our application is to the hotel market, where we combine two datasets: consumers' web browsing histories, and hotel prices and occupancy rates. Using an event study design, we show that closeness in latent characteristic space predicts competition: hotels that are close to new entrants lose the most market share post-entry. We take a more structural approach to the 2016 merger of Marriott and Starwood, demonstrating that by using latent characteristics and consumer preferences learned from search data, we can substantially improve post-merger predictions of demand relative to standard baselines.
Keywords: E-Commerce, Search, Demand Estimation, Transfer Learning, Embeddings
JEL Classification: C13, C38, C51, C52, L1, L22, L81
Suggested Citation: Suggested Citation