Deriving the Pricing Power of Product Features by Mining Consumer Reviews

Abstract:     
The growing pervasiveness of the Internet has changed the way that consumers shop for goods. Increasingly, user-generated product reviews serve as a valuable source of information for customers making product choices online. While there is a significant body of theory on multi-attribute choice under uncertainty, the literature that examines product reviews has not built on this stream of theory for a variety of reasons. Typically, the impact of product reviews has been incorporated by numeric variables representing the valence and volume of reviews. In this paper we posit that the information embedded in product reviews cannot be captured by a single scalar value. Rather, we argue that product reviews are multifaceted and hence, the textual content of product reviews is an important determinant of consumers' choices, over and above the valence and volume of reviews. We provide a text mining technique that allows us to incorporate text in choice and panel data models by decomposing textual reviews into segments, evaluating different product features. We test our pproach on a unique dataset collected from Amazon, and demonstrate how it can be used to learn consumers' relative preferences for different product features. The dataset used contains three different groups of products (digital cameras, camcorders, PDAs), associated sales data and consumer review data gathered over a 15-month period. Additionally, we present and discuss two experimental techniques that can be used to alleviate the problem of data sparsity and of omitted variables: the first technique models consumer opinions as elements of a tensor product of independent feature and evaluation spaces and the second technique clusters rare opinions based on pointwise mutual information. The paper concludes by discussing the managerial relevance of this work as a tool for extracting actionable business intelligence from user-generated content.

Keywords: user-generated content, consumer reviews, e-commerce, econometrics, electronic markets, sentiment analysis, text mining

Working Paper Series