Culling the Herd: Using Real World Randomized Experiments to Measure Social Bias with Known Costly Goods

Management Science, Forthcoming

49 Pages Posted: 6 Oct 2013 Last revised: 15 Feb 2016

Miguel Godinho de Matos

School of Business and Economics, Católica-Lisbon; Carnegie Mellon University

Pedro Ferreira

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management; Carnegie Mellon University - Department of Engineering and Public Policy

Michael D. Smith

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management

Rahul Telang

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management

Date Written: February 25, 2016

Abstract

Peer ratings have become increasingly important sources of product information, particularly in markets for information goods. However, in spite of the increasing prevalence of this information, there are relatively few academic studies that analyze the impact of peer ratings on consumers transacting in “real-world” marketplaces. In this paper, we partner with a major telecommunications company to analyze the impact of peer ratings in a real-world video-on-demand market where consumer participation is organic and where movies are costly and well known to consumers. After experimentally changing the initial conditions of product information displayed to consumers, we find that, consistent with the prior literature, peer ratings influence consumer behavior independently from underlying product quality. However, we also find that, in contrast to the prior literature, there is little evidence of long-term bias as a result of herding effects, at least in our setting. Specifically, when movies are artificially promoted or demoted in peer rating lists, subsequent reviews cause them to return to their true quality position relatively quickly. One explanation for this difference is that consumers in our empirical setting likely had more outside information about the true quality of the products they were evaluating than did consumers in the studies reported in prior literature. Although tentative, this explanation suggests that in real-world marketplaces where consumers have sufficient access to outside information about true product quality, peer ratings may be more robust to herding effects and thus provide more reliable signals of true product quality than previously thought.

Keywords: Randomized Experiment, Likes, Ratings, Video-on-Demand

Suggested Citation

Godinho de Matos, Miguel and Ferreira, Pedro and Smith, Michael D. and Telang, Rahul, Culling the Herd: Using Real World Randomized Experiments to Measure Social Bias with Known Costly Goods (February 25, 2016). Management Science, Forthcoming. Available at SSRN: https://ssrn.com/abstract=2336158 or http://dx.doi.org/10.2139/ssrn.2336158

Miguel Godinho de Matos (Contact Author)

School of Business and Economics, Católica-Lisbon ( email )

Palma de Cima
Lisbon, Lisboa 1649-023
Portugal

Carnegie Mellon University ( email )

Pittsburgh, PA 15213
United States

Pedro Ferreira

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management ( email )

Pittsburgh, PA 15213-3890
United States

Carnegie Mellon University - Department of Engineering and Public Policy ( email )

Baker Hall 129
5000 Forbes Ave
Pittsburgh, PA 15213
United States

Michael D. Smith

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management ( email )

Pittsburgh, PA 15213-3890
United States

HOME PAGE: http://www.heinz.cmu.edu/~mds

Rahul Telang

Carnegie Mellon University - H. John Heinz III School of Public Policy and Management ( email )

4800 Forbes Ave
Pittsburgh, PA 15213-3890
United States
412-268-1155 (Phone)

Paper statistics

Downloads
143
Rank
168,105
Abstract Views
805