Neglect of Multiplicity When Testing Families of Related Hypotheses*
46 Pages Posted: 3 Sep 2009
Date Written: September 1, 2009
Many accounting research papers report statistical results involving families of related tests of hypotheses. Unlike many scientific disciplines, accounting researchers rarely attend to a familywise Type I error concept; for example, control of the probability of incorrectly declaring at least one significant result in the family. Neglecting this principle can lead to an excessive number of false declarations of significance, and therefore, spurious inferences. We review some simple established methods for controlling for multiplicity. A few examples from the “Top 3” accounting journals illustrate that inattention to familywise control can affect primary study conclusions. A compromise approach is suggested for those who choose to present results either with or without recognition of familywise Type I error control.
Keywords: multiple comparisons, Bonferroni, Holm, publication bias, journal policies
Suggested Citation: Suggested Citation