Quality Certifications Influence User-Generated Ratings

Abstract

Platforms present various certifications to signal the quality of their offerings to prospective consumers. For example, Airbnb.com designates some hosts as “Superhosts” to distinguish properties that provide superior experiences. Platforms also present user-generated ratings—typically elicited and presented as “star ratings”—from their customers for the same purpose. This research investigates the interaction of these signals of quality and suggests a potential downside to platform-provided certifications: They decrease subsequent ratings. In an analysis of over 1,500,000 ratings from Airbnb.com and three follow-up studies, we find that properties with the superhost designation receive lower ratings. We assess the robustness of this result in several ways, including comparing ratings on Airbnb with those for the same property of Vrbo. In three follow-up experiments, we find that the net effect of certifications can lead to reduced choice share: The positive effect of signaling quality is more than offset by the negative effect of reduced ratings. This suggests that consumers are not sufficiently aware of this effect of quality certifications on ratings when choosing.

Keywords: user-generated ratings, online ratings, evaluability