Bias Against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators

30 Pages Posted: 5 Jan 2016  

Jian Wang

KU Leuven - Department of Managerial Economics, Strategy, and Innovation

Reinhilde Veugelers

Catholic University of Leuven (KUL) - Department of Applied Economics; Centre for Economic Policy Research (CEPR)

Paula E. Stephan

Georgia State University - Department of Economics; National Bureau of Economic Research (NBER)

Multiple version iconThere are 4 versions of this paper

Date Written: December 2015

Abstract

Research which explores unchartered waters has a high potential for major impact but also carries a high uncertainty of having minimal impact. Such explorative research is often described as taking a novel approach. This study examines the complex relationship between pursuing a novel approach and impact. We measure novelty by examining the extent to which a published paper makes first time ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to a set of one million research articles across all scientific disciplines. We find that highly novel papers, defined to be those that make more (distinct) new combinations, have more than a triple probability of being a top 1% highly cited paper when using a sufficiently long citation time window to assess impact. Moreover, follow-on papers that cite highly novel research are themselves more likely to be highly cited. However, novel research is also risky as it has a higher variance in the citation performance. These findings are consistent with the “high risk/high gain” characteristic of novel research. We also find that novel papers are typically published in journals with a lower than expected Impact Factor and are less cited when using a short time window. Our findings suggest that science policy, in particular funding decisions which are over reliant on traditional bibliometric indicators based on short-term direct citation counts and Journal Impact Factors, may be biased against novelty.

Suggested Citation

Wang, Jian and Veugelers, Reinhilde and Stephan, Paula E., Bias Against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators (December 2015). Available at SSRN: https://ssrn.com/abstract=2710572 or http://dx.doi.org/10.2139/ssrn.2710572

Jian Wang (Contact Author)

KU Leuven - Department of Managerial Economics, Strategy, and Innovation ( email )

Naamsestraat 69 bus 3500
Leuven, 3000
Belgium

Reinhilde Veugelers

Catholic University of Leuven (KUL) - Department of Applied Economics ( email )

Leuven, B-3000
Belgium
+32 16 32 6908 (Phone)
+32 16 32 6732 (Fax)

Centre for Economic Policy Research (CEPR)

London
United Kingdom

Paula E. Stephan

Georgia State University - Department of Economics ( email )

P.O. Box 3992
Atlanta, GA 30302-3992
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
104
rank
183,169
Abstract Views
618
PlumX