Download This PaperDo Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?
57 Pages Posted: 9 Aug 2022 Last revised: 16 Dec 2022
There are 2 versions of this paper
Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?
Do Pre-Registration and Pre-Analysis Plans Reduce P-Hacking and Publication Bias?
Date Written: December 15, 2022
Abstract
Randomized controlled trials (RCTs) are increasingly prominent in economics, with pre-registration and pre-analysis plans (PAPs) promoted as important in ensuring the credibility of findings. We investigate whether these tools reduce the extent of p-hacking and publication bias by collecting and studying the universe of test statistics, 15,992 in total, from RCTs published in 15 leading economics journals from 2018 through 2021. In our primary analysis, we find no meaningful difference in the distribution of test statistics from pre-registered studies, compared to their non-pre-registered counterparts. However, pre-registered studies that have a complete PAP are significantly less p-hacked. These results point to the importance of PAPs, rather than pre-registration in itself, in ensuring credibility.
Keywords: Pre-analysis Plan, Pre-registration, p-Hacking, Publication Bias, Research Credibility
JEL Classification: B41, C13, C40, C93
Suggested Citation: Suggested Citation
