High Impact = High Statistical Standards? Not Necessarily So
Posted: 19 Aug 2012
Date Written: August 17, 2012
Which are the statistical practices of articles published in journals with high impact factor? Are there differences compared with articles published in journals with lower impact factor which have adopted editorial policies to reduce the impact of limitations of Null Hypothesis Significance Testing? The current study analyzes all articles related to psychological, neuropsychological and medical issues, published in 2011 in four journals with high impact factor: Science, Nature, NEJM and The Lancet, and three journals with lower impact factor: Neuropsychology, Journal of Experimental Psychology-Applied and the American Journal of Public Health. Results show that Null Hypothesis Significance Testing without any use of confidence intervals, effect size, prospective power and model estimation, is the prevalent statistical practice used in articles published in Nature, followed by articles published in Science. In contrast, in all other journals, most articles report confidence intervals and/or effect size measures. We interpreted these differences as consequences of the editorial policies adopted by the journal editors, which are probably the most efficient means to improve the statistical practices in journals with high or low impact factors.
Keywords: NHST, statistical practice, Impact factor, Confidence intervals, effect size
Suggested Citation: Suggested Citation