The Disruption Index Suffers From Citation Inflation and Is Confounded by Shifts in Scholarly Citation Practice

9 Pages Posted: 26 Jun 2023 Last revised: 25 Oct 2023

See all articles by Alexander Michael Petersen

Alexander Michael Petersen

University of California Merced, Department of Management of Complex Systems

Felber Arroyave

University of California, Merced

Fabio Pammolli

Polytechnic University of Milan - Department of Management, Economics and Industrial Engineering; CERM Foundation

Date Written: June 20, 2023

Abstract

Measuring the rate of innovation in academia and industry is fundamental to monitoring the efficiency and competitiveness of the knowledge economy. To this end, a disruption index (CD) was recently developed and applied to publication and patent citation networks (Funk & Owen-Smith, 2017; Wu et al., 2019; Park et al., 2023). Here we show that CD systematically decreases over time due to secular growth in research and patent production, following two distinct mechanisms unrelated to innovation – one structural and the other behavioral. The structural explanation follows from ‘citation inflation’ (Petersen et al., 2018), an inextricable feature of real citation networks attributable to increasing reference list lengths, which causes CD to systematically decrease. We demonstrate this causal link by way of mathematical deduction, computational simulation, multi-variate regression, and quasi- experimental juxtaposition of the disruptiveness of PNAS versus PNAS Plus articles, which differ primarily in their article lengths. The behavioral explanation reflects shifts associated with techno-social factors (e.g. self-citation practices) that increase the rate of triadic closure in citation networks and confounds efforts to measure disruptive innovation using CD. Combined, these two mechanisms render CD unsuitable for cross-temporal analysis, and thereby call into question a growing body of research employing CD to measure innovation trends and identify co-factors associated with team assembly.

Keywords: Disruption index, citation inflation, confounded variable, measurement error, Research Evaluation, Citation Analysis, Science Policy

JEL Classification: D39, J24, J44, C40

Suggested Citation

Petersen, Alexander Michael and Arroyave, Felber and Pammolli, Fabio, The Disruption Index Suffers From Citation Inflation and Is Confounded by Shifts in Scholarly Citation Practice (June 20, 2023). Available at SSRN: https://ssrn.com/abstract=4486421 or http://dx.doi.org/10.2139/ssrn.4486421

Alexander Michael Petersen (Contact Author)

University of California Merced, Department of Management of Complex Systems ( email )

School of Engineering
Science & Engineering 2, Suite 315
Merced, CA 95343
United States

Felber Arroyave

University of California, Merced ( email )

P.O. Box 2039
Merced, CA 95344
United States

Fabio Pammolli

Polytechnic University of Milan - Department of Management, Economics and Industrial Engineering ( email )

Via Lambruschini 4C - building 26/A
Milano, 20156
Italy

CERM Foundation ( email )

Via Fiorentina, 1
Siena, Siena 53100
Italy

HOME PAGE: http://www.cermlab.it

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
131
Abstract Views
649
Rank
414,957
PlumX Metrics