Rounding and Other Pitfalls in Meta-Studies on P-Hacking and Publication Bias: A Comment on Brodeur Et Al. (2020)

37 Pages Posted: 24 May 2021 Last revised: 18 Nov 2021

Date Written: November 17, 2021

Abstract

Brodeur et al. (2020) study hypothesis tests from economic articles and find evidence for p-hacking and publication bias, in particular for IV and DID studies. We adjust for rounding errors in reported estimates and standard errors using different approaches. Statistical evidence for p-hacking from randomization tests and caliper tests at the 5% significance threshold robustly vanishes for DID studies but remains for IV studies. In addition, BCH derive latent distribution of z-statistics absent publication bias using two different approaches. We show that neither approach is applicable due to reasons unrelated to rounding.

Keywords: p-hacking, publication bias, meta-study

JEL Classification: A14, C12, C52

Suggested Citation

Kranz, Sebastian and Pütz, Peter, Rounding and Other Pitfalls in Meta-Studies on P-Hacking and Publication Bias: A Comment on Brodeur Et Al. (2020) (November 17, 2021). Available at SSRN: https://ssrn.com/abstract=3848786 or http://dx.doi.org/10.2139/ssrn.3848786

Sebastian Kranz (Contact Author)

Ulm University ( email )

Albert-Einstein-Alee 11
Ulm, D-89081
Germany

HOME PAGE: http://www.uni-ulm.de/mawi/mawi-wiwi/mitarbeiter/skranz.html

Peter Pütz

Bielefeld University ( email )

Universitätsstraße 25
Bielefeld, NRW 33613
Germany

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
163
Abstract Views
670
rank
252,137
PlumX Metrics