• Selected
• Entire List

Abstract:
This paper examines the effect of recommender systems on the diversity of sales. Two anecdotal views exist about such effects. Some believe recommenders help consumers discover new products and thus increase sales diversity. Others believe recommenders only reinforce the popularity of already popular products. This paper seeks to reconcile these seemingly incompatible views. We explore the question in two ways. First, modeling recommender systems analytically allows us to explore their path dependent effects. Second, turning to simulation, we increase the realism of our results by combining choice models with actual implementations of recommender systems. We arrive at three main results. First, some well known recommenders can lead to a reduction in sales diversity. Because common recommenders (e.g., collaborative filters) recommend products based on sales and ratings, they cannot recommend products with limited historical data, even if they would be rated favorably. In turn, these recommenders can create a rich-get-richer effect for popular products and vice-versa for unpopular ones. This bias toward popularity can prevent what may otherwise be better consumer-product matches. That diversity can decrease is surprising to consumers who express that recommendations have helped them discover new products. In line with this, result two shows that it is possible for individual-level diversity to increase but aggregate diversity to decrease. Recommenders can push each person to new products, but they often push users toward the same products.. Third, we show how basic design choices affect the outcome, and thus managers can choose recommender designs that are more consistent with their sales goals and consumers’ preferences.

recommender systems, collaborative filtering, sales diversity, Lorenz curve, Gini coefficient, long tail, electronic commerce, path dependence, simulation, concentration, diversity

Abstract:
Recommender systems are becoming integral to how consumers discover media. The value that recommenders offer is personalization: in environments with many product choices, recommenders personalize the browsing and consumption experience to each user’s taste. Popular applications include product recommendations at e-commerce sites and online newspapers’ selecting articles to display based on the current reader’s interests. This ability to focus more closely on one's taste and filter all else out has spawned criticism that recommenders will fragment consumers. Critics say recommenders cause consumers to have less in common with one another and that the media should do more to increase exposure to a variety of content. Others, however, contend that recommenders do the opposite: they may homogenize users because they share information among those who would otherwise not communicate. These are opposing views, discussed in the literature for over ten years for which there is not yet empirical evidence. We present an empirical study of recommender systems in the music industry. In contrast to concerns that users are becoming more fragmented, we find that in our setting users become more similar to one another in their purchases. This increase in similarity occurs for two reasons, which we term volume and taste effects. The volume effect is that consumers simply purchase more after recommendations, increasing the chance of having more purchases in common. The taste effect is that, conditional on volume, consumers buy a more similar mix of products after recommendations. When we view consumers as a similarity network before versus after recommendations, we find that the network becomes denser and smaller, or characterized by shorter inter-user distances. These findings suggest that for this setting, recommender systems are associated with an increase in commonality among users and that concerns of fragmentation may be misplaced.

recommender systems, collaborative filtering, fragmentation, personalization, long tail

Abstract:
Cluster analysis is useful for data interpretation. Instead of studying thousands of records, one can create a smaller number of clusters and interpret a prototype for each. Often, however, the world being interpreted via clusters can change. The naive approach of independently reclustering the data each period has a significant drawback: even if the data's distribution is unchanged, sampling variation can cause cluster prototypes to differ from one period to the next, which creates difficulty in comparing cluster solutions. In this paper we present a method for clustering sequential data sets and comparing cluster solutions over time. At a macro level, we examine how cluster prototypes change over time; at a micro level, we examine how objects transition among these prototypes. The method works as follows. We take as given cluster prototypes from the first data set. In clustering the new data, the previous prototypes are constrained to remain unchanged; this enables consistency among old and new prototypes. However, to fit the new data well, the second clustering must be flexible enough to add new prototypes where needed. This amounts to an optimization criteria that trades off consistency (reuse of old prototypes) with model fit (cluster fit on the new data). We formulate this as a constrained optimization problem and present a solution technique. A feature of the technique is its ability to incorporate prior knowledge from the first period to define an appropriate consistency-fit tradeoff. We envision the method will have particular relevance for business, as firms increasingly manage their customers through segments for which new data arrives over time.

clustering, cluster analysis, resampling, penalty methods

Abstract:
Recommender systems typically work over sparse matrices. Although most methods assume so, these matrices' entries are often not missing at random (NMAR). How problematic is this? We present a puzzle. Some methods explicitly account for NMAR processes. This has been shown to improve predictions. Many methods, however, assume that entries are missing at random (MAR). While they may be wrong in that assumption, we show they may benefit nonetheless from its being violated. Given that some data must go missing, NMAR can often pick the "right" values to preserve (i.e. it preserves the more informative data). Thus despite the perception that NMAR is bad, it can often improve recommendations. This may explain some of the historical success of collaborative filtering even when this assumption has been violated.

recommender systems, collaborative filtering, predictive modeling, missing data