Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence
33 Pages Posted: 8 Sep 2017 Last revised: 9 Jan 2018
Date Written: September 5, 2017
Abstract
From a survey of the papers published in leading accounting journals in 2014, we find that accounting researchers conduct significance testing almost exclusively at a conventional level of significance, without considering the key factors such as sample size or power of a test. We present evidence that a vast majority of the accounting studies favour large or massive sample sizes and conduct significance tests with the power extremely close to or equal to one. As a result, statistical inference is severely biased towards Type I error, frequently rejecting the true null hypotheses. Under the ‘p-value less than 0.05’ criterion for statistical significance, more than 90% of the surveyed papers report statistical significance. However, under alternative criteria, only 40% of the results are statistically significant. We propose that substantial changes be made to the current practice of significance testing for more credible empirical research in accounting.
Keywords: Bayesian inference, Research credibility, Sample size, Statistical significance, Statistical power
JEL Classification: C12, M40
Suggested Citation: Suggested Citation