Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA

34 Pages Posted: 9 Jan 2021

See all articles by Loet Leydesdorff

Loet Leydesdorff

University of Amsterdam - Amsterdam School of Communication Research (ASCoR)

Caroline Wagner

Ohio State University (OSU)

Lin Zhang

KU Leuven - Faculty of Business and Economics (FEB)

Multiple version iconThere are 2 versions of this paper

Date Written: November 18, 2020

Abstract

Purpose: We address the question of whether differences are statistically significant in the rankings of universities. We propose methods measuring the statistical significance among different universities and illustrate the results by empirical data.

Design/methodology/approach: Based on z-testing and overlapping confidence intervals, and using data about 205 Chinese universities included in the Leiden Rankings 2020, we argue that three main groups of Chinese research universities can be distinguished.

Findings: When the sample of 205 Chinese universities is merged with the 197 US universities included in Leiden Rankings 2020, the results similarly indicate three main groups: high, middle, low. Using this data (Leiden Rankings and Web-of-Science), the z-scores of the Chinese universities are significantly below those of the US universities albeit with some overlap. Research limitations: We show empirically that differences in ranking may be due to changes in the data, the models, or the modeling effects on the data. The scientometric groupings are not always stable when we use different methods.

R&D policy implications: Differences among universities can be tested for their statistical significance. The statistics relativize the values of decimals in the rankings. One can operate with a scheme of low/middle/high in policy debates and leave the more fine-grained rankings of individual universities to operational management and local settings.

Originality/value: In the discussion about the rankings of universities, the question of whether differences are statistically significant, is, in our opinion, insufficiently addressed.

Keywords: ranking, university, China, significance of differences, indicators

Suggested Citation

Leydesdorff, Loet and Wagner, Caroline and Zhang, Lin, Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA (November 18, 2020). Available at SSRN: https://ssrn.com/abstract=3732825 or http://dx.doi.org/10.2139/ssrn.3732825

Loet Leydesdorff

University of Amsterdam - Amsterdam School of Communication Research (ASCoR) ( email )

PO Box 15793
Amsterdam, 1001 NG
Netherlands

HOME PAGE: http://www.leydesdorff.net

Caroline Wagner

Ohio State University (OSU) ( email )

Page Hall
1810 College Road N
Columbus, OH OH 43210
United States
6142927791 (Phone)

HOME PAGE: http://glenn.osu.edu

Lin Zhang (Contact Author)

KU Leuven - Faculty of Business and Economics (FEB) ( email )

Naamsestraat 69
Leuven, B-3000
Belgium

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
44
Abstract Views
336
PlumX Metrics