Incentives Can Reduce Bias in Online Reviews

45 Pages Posted: 8 Jan 2018 Last revised: 26 Feb 2018

Ioana Elena Marinescu

University of Pennsylvania - School of Social Policy & Practice; National Bureau of Economic Research (NBER)

Nadav Klein

University of Chicago

Andrew Chamberlain

Glassdoor, Inc.

Morgan Smart

Glassdoor, Inc.

Multiple version iconThere are 3 versions of this paper

Date Written: January 3, 2018

Abstract

Online reviews are a powerful means of propagating the reputations of products, services, and even employers. However, existing research suggests that online reviews often suffer from selection bias—people with extreme opinions are more motivated to share them than people with moderate opinions, resulting in biased distributions of reviews. Providing incentives for reviewing has the potential to reduce this selection bias, because incentives can mitigate the motivational deficit of people who hold moderate opinions. Using data from one of the leading employer review companies Glassdoor, we show that voluntary reviews have a different distribution from incentivized reviews. The likely bias in the distribution of voluntary reviews can affect workers’ choice of employers, because it changes the ranking of industries by average employee satisfaction. Because observational data from Glassdoor are not able to provide a measure of the true distribution of employer reviews, we complement our investigation with a randomized controlled experiment on MTurk. We find that when participants’ decision to review their employer is voluntary, the resulting distribution of reviews differs from the distribution of forced reviews. Moreover, providing relatively high monetary rewards or a pro-social cue as incentives for reviewing reduces this bias. We conclude that while voluntary employer reviews often suffer from selection bias, incentives can significantly reduce bias and help workers make more informed employer choices.

Keywords: incentives, online reviews, word of mouth, survey bias

JEL Classification: J28, C83

Suggested Citation

Marinescu, Ioana Elena and Klein, Nadav and Chamberlain, Andrew and Smart, Morgan, Incentives Can Reduce Bias in Online Reviews (January 3, 2018). Available at SSRN: https://ssrn.com/abstract=3092828 or http://dx.doi.org/10.2139/ssrn.3092828

Ioana Elena Marinescu (Contact Author)

University of Pennsylvania - School of Social Policy & Practice ( email )

3701 Locust Walk
Philadelphia, PA 19104-6214
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Nadav Klein

University of Chicago ( email )

1155 East 60th Street
Chicago, IL 60637
United States

Andrew Chamberlain

Glassdoor, Inc. ( email )

100 Shoreline Highway
Mill Valley, CA 94941
United States

HOME PAGE: http://www.glassdoor.com/research/

Morgan Smart

Glassdoor, Inc. ( email )

100 Shoreline Hwy
Mill Valley, CA 94941
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
220
rank
119,545
Abstract Views
1,330
PlumX