Is Novel Research Worth Doing? Evidence from Journal Peer Review

39 Pages Posted: 13 Sep 2021 Last revised: 26 Oct 2021

See all articles by Misha Teplitskiy

Misha Teplitskiy

University of Michigan School of Information

Hao Peng

University of Michigan at Ann Arbor - School of Information

Andrea Blasco

Harvard University - Institute for Quantitative Social Science

Karim R. Lakhani

Harvard Business School - Technology and Operations Management Group; Harvard Institute for Quantitative Social Science; Harvard University - Berkman Klein Center for Internet & Society

Date Written: September 9, 2021

Abstract

Scientific institutions like journals and funding agencies often express desire for novel ideas, but there are long-standing concerns that in practice they favor conservative ones. Here, we examine the association between novelty and acceptance among 21,406 manuscripts submitted between 2013-2018 to one field-leading and one middle-tier life sciences journal. Measuring the novelty of submissions as atypical combinations of journals in their reference lists, we find no evidence of conservatism. Instead, at the top journal, submissions in the top quintile of novelty are 18.5 percentage points more likely than bottom quintile ones to get accepted, while the middle-tier journal shows no systematic favor for or against novel submissions. Separating plausible mechanisms into supply-side (characteristics of the submission pool) and demand-side (judgements by editors and reviewers) shows the importance of the latter, and particularly editor discretion. On the supply-side, novelty is not strongly associated with submissions’ risk or reward, measured with citations. On the demand-side, at the top journal peer reviewers do not select for or against novelty, while editors select strongly for novelty, even conditional on quality. In further support of editor discretion, different editors show different novelty responses to the manuscripts that happened to be reviewed by both journals. Overall, the findings show that journal peer review, particularly at the top, incentivizes novel work.

Keywords: novelty, evaluation, peer review

Suggested Citation

Teplitskiy, Misha and Peng, Hao and Blasco, Andrea and Lakhani, Karim R., Is Novel Research Worth Doing? Evidence from Journal Peer Review (September 9, 2021). Available at SSRN: https://ssrn.com/abstract=3920711 or http://dx.doi.org/10.2139/ssrn.3920711

Misha Teplitskiy (Contact Author)

University of Michigan School of Information ( email )

Ann Arbor, MI
United States

HOME PAGE: http://www.misha.mx

Hao Peng

University of Michigan at Ann Arbor - School of Information ( email )

304 West Hall
550 East University
Ann Arbor, MI 48109-1092
United States

Andrea Blasco

Harvard University - Institute for Quantitative Social Science ( email )

1737 Cambridge St.
Cambridge, MA 02138
United States

Karim R. Lakhani

Harvard Business School - Technology and Operations Management Group ( email )

Boston, MA 02163
United States
617-495-6741 (Phone)

Harvard Institute for Quantitative Social Science ( email )

1737 Cambridge St.
Cambridge, MA 02138
United States

Harvard University - Berkman Klein Center for Internet & Society ( email )

Harvard Law School
23 Everett, 2nd Floor
Cambridge, MA 02138
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
225
Abstract Views
1,657
rank
192,504
PlumX Metrics