Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence

33 Pages Posted: 8 Sep 2017 Last revised: 9 Jan 2018

See all articles by Jae H. Kim

Jae H. Kim

affiliation not provided to SSRN

Philip Ji

Monash University - Department of Accounting; Financial Research Network (FIRN)

Kamran Ahmed

La Trobe Business School, La Trobe University

Date Written: September 5, 2017

Abstract

From a survey of the papers published in leading accounting journals in 2014, we find that accounting researchers conduct significance testing almost exclusively at a conventional level of significance, without considering the key factors such as sample size or power of a test. We present evidence that a vast majority of the accounting studies favour large or massive sample sizes and conduct significance tests with the power extremely close to or equal to one. As a result, statistical inference is severely biased towards Type I error, frequently rejecting the true null hypotheses. Under the ‘p-value less than 0.05’ criterion for statistical significance, more than 90% of the surveyed papers report statistical significance. However, under alternative criteria, only 40% of the results are statistically significant. We propose that substantial changes be made to the current practice of significance testing for more credible empirical research in accounting.

Keywords: Bayesian inference, Research credibility, Sample size, Statistical significance, Statistical power

JEL Classification: C12, M40

Suggested Citation

Kim, Jae H. and Ji, Philip Inyeob and Ahmed, Kamran, Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence (September 5, 2017). 9th Conference on Financial Markets and Corporate Governance (FMCG) 2018, Available at SSRN: https://ssrn.com/abstract=3032438 or http://dx.doi.org/10.2139/ssrn.3032438

Jae H. Kim

affiliation not provided to SSRN

Philip Inyeob Ji

Monash University - Department of Accounting ( email )

Building 11E
Clayton, Victoria 3800
Australia

Financial Research Network (FIRN)

C/- University of Queensland Business School
St Lucia, 4071 Brisbane
Queensland
Australia

HOME PAGE: http://www.firn.org.au

Kamran Ahmed (Contact Author)

La Trobe Business School, La Trobe University ( email )

Kingsbury Drive
Melbourne, VIC 3086
Australia
0403038387 (Phone)
3105 (Fax)

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
280
Abstract Views
1,987
Rank
231,664
PlumX Metrics