P-Hacking in Experimental Accounting Studies

59 Pages Posted: 12 Jan 2021 Last revised: 16 Aug 2021

See all articles by Xin Chang

Xin Chang

Nanyang Business School, Nanyang Technological University

Huasheng Gao

Fanhai International School of Finance, Fudan University

Wei Li

Shanghai University of Finance and Economics

Date Written: September 5, 2017

Abstract

We study the extent and determinants of p-hacking in experimental accounting studies. Based on p-values in experimental studies published in top accounting journals, we find an unusual abundance of p-values that are just significant: the observed frequency of p-values equal to 0.05 is 22% to 30% higher than what would be expected based on the frequency of other p-values. This suggests that some researchers may have p-hacked to obtain “just significant” results. Further analyses reveal that p-hacking is more evident in articles by junior researchers, authors from highly ranked schools, male authors, or sole authors. Moreover, the p-hacking evidence is stronger in articles with more experiment participants or in tests concerning key results. More cited articles exhibit a greater extent of p-hacking, indicating that p-hacking helps attract citations. Lastly, we provide the first evidence that the disclosure policy adopted by the Journal of Accounting Research in 2014 mitigates p-hacking.

Keywords: P-hacking; Experimental accounting; NHST; Statistical significance; Hypothesis testing

Suggested Citation

Chang, Xin and Gao, Huasheng and Li, Wei, P-Hacking in Experimental Accounting Studies (September 5, 2017). Nanyang Business School Research Paper No. 21-01, Available at SSRN: https://ssrn.com/abstract=3762342 or http://dx.doi.org/10.2139/ssrn.3762342

Xin Chang (Contact Author)

Nanyang Business School, Nanyang Technological University ( email )

Singapore, 639798
Singapore

HOME PAGE: https://personal.ntu.edu.sg/changxin/

Huasheng Gao

Fanhai International School of Finance, Fudan University ( email )

Beijing West District Baiyun Load 10th
Shanghai, 100045
China
2165642222 (Phone)
2165642222 (Fax)

Wei Li

Shanghai University of Finance and Economics ( email )

777 Guoding Road
Shanghai, AK Shanghai 200433
China

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
252
Abstract Views
1,422
rank
177,985
PlumX Metrics