Judgement and Ranking: Living with Hidden Bias
24 Pages Posted: 28 Sep 2016
Date Written: September 27, 2016
Abstract
The complexity and subjectivity of the judgement task conceals the existence of biases that undermines the quality of the process. This paper presents a weighted aggregation function that attempts to reduce the influence of biased judgements on the final score. We also discuss a set of desirable properties. The proposed weighted aggregation function is able to correct the "nationalism bias" found by Emerson et al. (2009) in the 2000 Olympic Games diving competition and suggest the possibility of a reputation bias. Our results can be applied to judgement sports and other activities that require the aggregation of several personal evaluations.
Keywords: Weighted aggregation function, Judgement by Grading, Nationalism Bias, Reputation Bias, Bias Correction, Olympic Games
JEL Classification: D72, D81, D03
Suggested Citation: Suggested Citation