Quantitative and Qualitative Rankings of Scholars

Schmalenbach Business Review, Vol. 63, pp. 63-91, January 2011

29 Pages Posted: 5 Jul 2011

See all articles by Katja Rost

Katja Rost

University of Zurich - Institute for Organization and Administrative Science

Bruno S. Frey

CREMA; University of Basel

Date Written: January 1, 2011

Abstract

In a former article we started to argue that publication and citation rankings of individual scholars do not effectively measure research quality, which should in fact be the essence of evaluation (Frey and Rost (2010)). For the field of economics we show that an alternative ranking based on membership on academic editorial boards of professional journals is randomly correlated with citation and publication rankings of these scholars. In this article we go a step further by hypothesizing a systematic, inverted U-shaped relationship between quantitative and qualitative rankings. By relying on a longitudinal data set of management scholars who are part of the international top community in organization science the findings support this multi-tasking effect. While a certain amount of publications indeed reflects aspects of research quality, it also suggests that maximizing publications ignores other essential aspects of research quality that are doubtlessly hard to measure. It follows that if career decisions are only based on high scores in publication rankings the result will be not only haphazard but may be even counterproductive for science.

Keywords: evaluations, rankings, research quality, scholars, universities

JEL Classification: H43, L15, O38

Suggested Citation

Rost, Katja and Frey, Bruno S., Quantitative and Qualitative Rankings of Scholars (January 1, 2011). Schmalenbach Business Review, Vol. 63, pp. 63-91, January 2011, Available at SSRN: https://ssrn.com/abstract=1879167

Katja Rost (Contact Author)

University of Zurich - Institute for Organization and Administrative Science ( email )

Plattenstrasse 14
CH-8032 Zurich
Switzerland

Bruno S. Frey

CREMA ( email )

Südstrasse 11
Zurich, CH 8008
Switzerland
+41 44 380 00 78 (Phone)

University of Basel ( email )

Peter Merian-Weg 6
Basel, 4002
Switzerland