Rounding and Other Pitfalls in Meta-Studies on P-Hacking and Publication Bias: A Comment on Brodeur Et Al. (2020)
37 Pages Posted: 24 May 2021 Last revised: 18 Nov 2021
Date Written: November 17, 2021
Abstract
Brodeur et al. (2020) study hypothesis tests from economic articles and find evidence for p-hacking and publication bias, in particular for IV and DID studies. We adjust for rounding errors in reported estimates and standard errors using different approaches. Statistical evidence for p-hacking from randomization tests and caliper tests at the 5% significance threshold robustly vanishes for DID studies but remains for IV studies. In addition, BCH derive latent distribution of z-statistics absent publication bias using two different approaches. We show that neither approach is applicable due to reasons unrelated to rounding.
Keywords: p-hacking, publication bias, meta-study
JEL Classification: A14, C12, C52
Suggested Citation: Suggested Citation