Performance Evaluation: Subjectivity, Bias and Judgement Style in Sport Data

26 Pages Posted: 28 Jan 2019

Date Written: January 24, 2019


The number of situations that require individual judgements and evaluations, and that can be object of different sources of conscious and unconscious biases is endless. This paper proposes a practical scores aggregation procedure that attempts to reduce and mitigate the influence of bias in subjective judgements. The argument is based on the idea that bias is associated with deviations from the panel mean and/or deviations from the judge grading style. Consequently, the procedure is not specific to a particular type of bias (e.g., nationalistic or reputation bias), but it addresses general forms of bias. We also discuss a set of desirable properties. The proposed scores aggregation procedure is then applied to a unique data set from the 2000 Summer Olympic Games diving competition. The proposed procedure captures the possibility of bias by adjusting the gap of scores between divers.

Keywords: Scores aggregation; Subjective judgements; Biased judgements; Bias correction; Sports data

JEL Classification: D71, D72, D91

Suggested Citation

Osório Costa, António Miguel, Performance Evaluation: Subjectivity, Bias and Judgement Style in Sport Data (January 24, 2019). Available at SSRN: or

António Miguel Osório Costa (Contact Author)

Universitat Rovira i Virgili ( email )

Av. de la Universitat 1
Reus, Tarragona 43204
(+34)977759891 (Phone)


Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics