Combining P-Values Via Averaging
29 Pages Posted: 17 May 2018 Last revised: 14 Dec 2019
Date Written: April 20, 2018
This paper proposes general methods for the problem of multiple testing of a single hypothesis, with a standard goal of combining a number of p-values without making any assumptions about their dependence structure. An old result by Ruschendorf and, independently, Meng implies that the p-values can be combined by scaling up their arithmetic mean by a factor of 2 (and no smaller factor is sufficient in general). Based on more recent developments in mathematical finance, specifically, robust risk aggregation techniques, we show that K p-values can be combined by scaling up their geometric mean by a factor of e (for all K) and by scaling up their harmonic mean by a factor of ln K (asymptotically as K -> infinity). These and other results lead to a generalized version of the Bonferroni-Holm method. A simulation study compares the performance of various averaging methods.
Keywords: hypothesis testing, multiple hypothesis testing, multiple testing of a single hypothesis, robust risk aggregation
JEL Classification: C12
Suggested Citation: Suggested Citation