You Rate Me and I’ll Rate You: Mutual Rating Relationships in Multi-Rater Performance Evaluation Systems

Abstract

We examine rating behavior in multi-rater performance evaluation systems. Specifically, we study mutual rating relationships, where two employees rate each other contemporaneously. We use proprietary data from an online retailer and show that demographic similarity and organizational proximity are positively associated with the likelihood of a mutual rating relationship. Mutual ratings are higher on average than one-sided ratings; this premium is driven by both selection effects in relationship formation and reciprocal uprating. While rater nominations are more likely to be approved by supervisors if they would result in a mutual rating relationship, we find more discerning supervisors nonetheless appear to be cognizant of the fact that mutual ratings may not provide a balanced assessment. We find that they place less (implicit) weight on mutual ratings vis-à-vis one-sided ratings when arriving at their employee assessments. Overall, our study offers novel evidence on a phenomenon inherent in multi-rater systems: mutual rating relationships.

Keywords: Performance evaluation; multi-rater systems.

JEL Classification: M12; M40; M50