Do Rankings Reflect Research Quality?

42 Pages Posted: 31 Oct 2008

See all articles by Bruno S. Frey

Bruno S. Frey

CREMA; University of Basel

Katja Rost

University of Zurich - Institute for Organization and Administrative Science

Multiple version iconThere are 2 versions of this paper

Date Written: October 2008

Abstract

Publication and citation rankings have become major indicators of the scientific worth of universities and countries, and determine to a large extent the career of individual scholars. We argue that such rankings do not effectively measure research quality, which should be the essence of evaluation. For that reason, an alternative ranking is developed as a quality indicator, based on membership on academic editorial boards of professional journals. It turns out that especially the ranking of individual scholars is far from objective. The results differ markedly, depending on whether research quantity or research quality is considered. Even quantity rankings are not objective; two citation rankings, based on different samples, produce entirely different results. It follows that any career decisions based on rankings are dominated by chance and do not reflect research quality. Instead of propagating a ranking based on board membership as the gold standard, we suggest that committees make use of this quality indicator to find members who, in turn, evaluate the research quality of individual scholars.

Keywords: rankings, universities, scholars, publications, citations

JEL Classification: H43, L15, O38

Suggested Citation

Frey, Bruno S. and Rost, Katja, Do Rankings Reflect Research Quality? (October 2008). CESifo Working Paper Series No. 2443, Available at SSRN: https://ssrn.com/abstract=1291645 or http://dx.doi.org/10.2139/ssrn.1291645

Bruno S. Frey (Contact Author)

CREMA ( email )

Südstrasse 11
Zurich, CH 8008
Switzerland
+41 44 380 00 78 (Phone)

University of Basel ( email )

Peter Merian-Weg 6
Basel, 4002
Switzerland

Katja Rost

University of Zurich - Institute for Organization and Administrative Science ( email )

Plattenstrasse 14
CH-8032 Zurich
Switzerland

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
228
Abstract Views
1,784
Rank
125,447
PlumX Metrics