Discontinuous Distribution of Test Statistics Around Significance Thresholds in Empirical Accounting Studies

71 Pages Posted: 12 Jan 2021 Last revised: 30 May 2024

See all articles by Xin Chang

Xin Chang

Nanyang Business School, Nanyang Technological University

Huasheng Gao

Fanhai International School of Finance, Fudan University

Wei Li

Shanghai University of Finance and Economics

Date Written: September 5, 2017

Abstract

Examining test statistics from articles in six leading accounting journals, we detect discontinuities in their distributions around conventional significance thresholds (p-values of 0.05 and 0.01) and find an unusual abundance of test statistics that are just significant. Further analysis reveals that these discontinuities are more prominent in studies with smaller samples and are more salient in experimental than in archival studies. The discontinuity discrepancy between experimental and archival studies relates to several proxies for researcher degrees of freedom. Nevertheless, this does not imply that experimental research is more prone to questionable research practices than archival studies. Overall, we provide suggestive evidence that some accounting researchers exercise undisclosed discretion to obtain and report statistically significant results. Thus, a healthy skepticism of some just-significant test statistics is warranted.

Keywords: p-value discontinuity; Experimental accounting; Archival accounting; Researcher degrees of freedom

Suggested Citation

Chang, Xin and Gao, Huasheng and Li, Wei, Discontinuous Distribution of Test Statistics Around Significance Thresholds in Empirical Accounting Studies (September 5, 2017). Nanyang Business School Research Paper No. 21-01, Available at SSRN: https://ssrn.com/abstract=3762342 or http://dx.doi.org/10.2139/ssrn.3762342

Xin Chang (Contact Author)

Nanyang Business School, Nanyang Technological University ( email )

Singapore, 639798
Singapore

HOME PAGE: https://personal.ntu.edu.sg/changxin/

Huasheng Gao

Fanhai International School of Finance, Fudan University ( email )

Beijing West District Baiyun Load 10th
Shanghai, 100045
China
2165642222 (Phone)
2165642222 (Fax)

Wei Li

Shanghai University of Finance and Economics ( email )

777 Guoding Road
Shanghai, AK Shanghai 200433
China

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
294
Abstract Views
2,141
Rank
194,694
PlumX Metrics