The bias-reducing effect of voluntary anonymization of authors’ identities: Evidence from peer review

Abstract

A persistent concern in formal evaluation processes like scientific peer review is that evaluators favor projects by prestigious producers. While anonymizing producers’ identities can reduce prestige bias, achieving anonymization can be difficult because explicit and subtle signals of identities can appear throughout projects, and enforcing compliance can be prohibitively expensive. An attractive alternative is to encourage—“nudge”—producers to anonymize their submissions without checking for compliance. However, encouraging anonymization could fail to reduce bias if low-prestige producers do not anonymize because they do not anticipate bias (“information frictions”) or high-prestige producers seek to benefit from the bias (“strategic non-anonymization”). We test these arguments in a quasi-experiment with IOP Publishing, one of the largest academic publishers, which adopted a policy encouraging anonymization and rolled it out across its portfolio of 57 physical sciences journals. Examining 156,015 submissions and measuring first author prestige with citations, we identify causal effects of voluntary anonymization. We find that neither information frictions nor strategy fully explain anonymization rates: highest-prestige authors anonymized less often but still substantially (19%), and the rate varied greatly by geography and gender. More importantly, the nudging policy increased positive peer reviews of low-prestige authors by 2.4% and acceptance by 5.6%. For middle- and high-prestige authors, the policy decreased positive reviews (1.8% and 1% [n.s.]) and final acceptance (4.6% and 2.2% [n.s.]). Overall, simply nudging producers to anonymize their submissions can help reduce prestige bias and should be considered by organizations for which enforcing full anonymization is impractical.

Keywords: evaluation processes, peer review, prestige bias, voluntary anonymization, causal evidence