Measuring the Developmental Function of Peer Review: A Multi-Dimensional, Cross-Disciplinary Analysis of Peer Review Reports from 740 Academic Journals

18 Pages Posted: 30 Aug 2021

See all articles by Daniel Garcıa-Costa

Daniel Garcıa-Costa

University of Valencia

Flaminio Squazzoni

University of Milan

Bahar Mehmani

Elsevier

Francisco Grimaldo

Departament d'Informàtica - Universitat de València

Date Written: August 27, 2021

Abstract

Reviewers do not only help to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on reports and journals without any optimal benchmark. To fill this gap, we adapted a recently proposed quality assessment tool and tested it on a sample of 1.3 million reports submitted to 740 Elsevier journals in 2018-2020. Results showed that the developmental standards of peer review are shared across areas of research, yet with remarkable differences. Reports submitted to social science and economics journals show the highest developmental standards. Reports from youngest reviewers, women and reviewers from Western Europe are generally more developmental than those from senior, men and reviewers working in academic institutions outside Western regions. Our findings suggest that increasing the standards of peer review at journals requires effort to assess interventions and measure practices with context-specific and multi-dimensional frameworks.

Keywords: Peer review, reviewers, standards, academic journals, natural language processing

JEL Classification: C55, C63, J16, O30

Suggested Citation

Garcıa-Costa, Daniel and Squazzoni, Flaminio and Mehmani, Bahar and Grimaldo, Francisco, Measuring the Developmental Function of Peer Review: A Multi-Dimensional, Cross-Disciplinary Analysis of Peer Review Reports from 740 Academic Journals (August 27, 2021). Available at SSRN: https://ssrn.com/abstract=3912607 or http://dx.doi.org/10.2139/ssrn.3912607

Daniel Garcıa-Costa

University of Valencia ( email )

Avda. de los Naranjos s/n
Valencia, Valencia E-46022
Spain

Flaminio Squazzoni (Contact Author)

University of Milan ( email )

Via Conservatorio 7
Milan, 20122
Italy
+393396584139 (Phone)

HOME PAGE: http://behavelab.org/flaminio-squazzoni/

Bahar Mehmani

Elsevier ( email )

Francisco Grimaldo

Departament d'Informàtica - Universitat de València ( email )

Av. de la Universitat, s/n
Burjassot, Valencia 46100
Spain

HOME PAGE: http://www.uv.es/grimo

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
170
Abstract Views
852
rank
226,891
PlumX Metrics