Methods to Account for Citation Inflation in Research Evaluation

14 Pages Posted: 24 Jun 2018 Last revised: 2 May 2019

See all articles by Alexander Michael Petersen

Alexander Michael Petersen

University of California Merced, Ernest and Julio Gallo Management Program

Raj Pan

Aalto University

Fabio Pammolli

Polytechnic University of Milan - Department of Management, Economics and Industrial Engineering; CERM Foundation

Santo Fortunato

Aalto University

Date Written: June 10, 2018

Abstract

Quantitative research evaluation requires measures that are transparent, relatively simple, and free of disciplinary and temporal bias. We document and provide solution to a hitherto unaddressed temporal bias – citation inflation – which arises from the basic fact that scientific publication is steadily growing at roughly 4% per year. Moreover, because the total production of citations grows by a factor of 2 every 12 years, this means that the real value of a citation depends on when it was produced. Consequently, failing to convert nominal citation values into real citation values produces significant mis-measurement of scientific impact. To address this problem, we develop a citation deflator method, outline the steps to generalize and implement it using the Web of Science portal, and analyze a large set of researchers from biology and physics to demonstrate how two common evaluation metrics – total citations and h-index – can differ by a remarkable amount depending on whether the underlying citation counts are deflated or not. In particular, our results show that the scientific impact of prior generations is likely to be significantly under- estimated when citations are not deflated, often by 100% or more of the nominal value. Thus, our study points to the need for a systemic overhaul of the counting methods used evaluating citation impact – especially in the case of researchers, journals, and institutions – which can span several decades and thus several doubling periods.

Keywords: Research Evaluation, Career Evaluation, Citation Analysis, Science Policy

JEL Classification: D39, J24, J44, C40

Suggested Citation

Petersen, Alexander Michael and Pan, Raj and Pammolli, Fabio and Fortunato, Santo, Methods to Account for Citation Inflation in Research Evaluation (June 10, 2018). Research Policy, Forthcoming, Available at SSRN: https://ssrn.com/abstract=3193712 or http://dx.doi.org/10.2139/ssrn.3193712

Alexander Michael Petersen (Contact Author)

University of California Merced, Ernest and Julio Gallo Management Program ( email )

School of Engineering
Science & Engineering 2, Suite 315
Merced, CA 95343
United States

Raj Pan

Aalto University ( email )

P.O. Box 21210
Helsinki, 00101
Finland

Fabio Pammolli

Polytechnic University of Milan - Department of Management, Economics and Industrial Engineering ( email )

Via Lambruschini 4C - building 26/A
Milano, 20156
Italy

CERM Foundation ( email )

Via Fiorentina, 1
Siena, Siena 53100
Italy

HOME PAGE: http://www.cermlab.it

Santo Fortunato

Aalto University ( email )

P.O. Box 21210
Helsinki, 00101
Finland

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
75
Abstract Views
648
rank
342,662
PlumX Metrics