Quality to Impact, Text to Metadata: Publication and Evaluation in the Age of Metrics

KNOW: A Journal on the Formation of Knowledge 2(2) 2018

32 Pages Posted: 8 Apr 2019

See all articles by Mario Biagioli

Mario Biagioli

University of California, Davis - School of Law

Date Written: December 29, 2018

Abstract

The evaluation of scholarly works used to be interpretively complex but technologically simple. One read and evaluated an author’s publication, manuscript, or grant proposal together with the evidence it contained or referred to. Scholars have been doing this for centuries, by themselves, from their desks, best if in the proximity of a good library. Peer review — the epitome of academic judgment and its independence — slowly grew from this model of scholarly evaluation by scholars.

Things have dramatically changed in recent years. The assessment of scholars and their work may now start and end with a simple Google Scholar search or other quantitative, auditing-like techniques that make reading publications superfluous. This is a world of evaluation not populated by scholars practicing peer review, but by a variety of methods and actors dispersed across academic institutions, data analytics companies, and media outlets tracking anything from citation counts (of books, journals, and conference abstracts) and journal impact factors, to a variety of indicators like H-index, Eigenfactor, CiteScore, SCImago Journal Rank, as well as altmetrics. We have moved from descriptive metrics used by scientists and scholars, to evaluative metrics used by outsiders who typically do not have technical knowledge of the field they seek to evaluate. This is a shift that reflects a fundamental and increasingly naturalized assumption that the number or frequency of citations received by a publication is, somehow, an index of its quality or value.

Keywords: peer review, metrics, evaluation, big data, academia, rankings

Suggested Citation

Biagioli, Mario, Quality to Impact, Text to Metadata: Publication and Evaluation in the Age of Metrics (December 29, 2018). KNOW: A Journal on the Formation of Knowledge 2(2) 2018. Available at SSRN: https://ssrn.com/abstract=3307615 or http://dx.doi.org/10.2139/ssrn.3307615

Mario Biagioli (Contact Author)

University of California, Davis - School of Law ( email )

Martin Luther King, Jr. Hall
Davis, CA CA 95616-5201
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
18
Abstract Views
126
PlumX Metrics
!

Under construction: SSRN citations while be offline until July when we will launch a brand new and improved citations service, check here for more details.

For more information