You Rate Me and I’ll Rate You: Mutual Rating Relationships in Multi-Rater Performance Evaluation Systems

59 Pages Posted: 27 Dec 2022

See all articles by Martin Artz

Martin Artz

University of Münster - Accounting Center

Carolyn Deller

University of Pennsylvania - The Wharton School

Sinja Leonelli

University of Chicago - Booth School of Business

Date Written: December 23, 2022

Abstract

We examine rating behavior in multi-rater performance evaluation systems. Specifically, we study mutual rating relationships, where two employees rate each other contemporaneously. We use proprietary data from an online retailer and show that demographic similarity and organizational proximity are positively associated with the likelihood of a mutual rating relationship. Mutual ratings are higher on average than one-sided ratings; this premium is driven by both selection effects in relationship formation and reciprocal uprating. While rater nominations are more likely to be approved by supervisors if they would result in a mutual rating relationship, we find more discerning supervisors nonetheless appear to be cognizant of the fact that mutual ratings may not provide a balanced assessment. We find that they place less (implicit) weight on mutual ratings vis-à-vis one-sided ratings when arriving at their employee assessments. Overall, our study offers novel evidence on a phenomenon inherent in multi-rater systems: mutual rating relationships.

Keywords: Performance evaluation; multi-rater systems.

JEL Classification: M12; M40; M50

Suggested Citation

Artz, Martin and Deller, Carolyn and Leonelli, Sinja, You Rate Me and I’ll Rate You: Mutual Rating Relationships in Multi-Rater Performance Evaluation Systems (December 23, 2022). Available at SSRN: https://ssrn.com/abstract=4310277 or http://dx.doi.org/10.2139/ssrn.4310277

Martin Artz

University of Münster - Accounting Center ( email )

Universitätsstr. 14-16
Münster, 48143
Germany

Carolyn Deller (Contact Author)

University of Pennsylvania - The Wharton School ( email )

1311 Steinberg Hall - Dietrich Hall
3620 Locust Walk
Philadelphia, PA 19104-6365
United States

Sinja Leonelli

University of Chicago - Booth School of Business ( email )

5807 S Woodlawn Ave
Chicago, IL 60637
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
54
Abstract Views
218
Rank
590,037
PlumX Metrics