Trade-off between validity and efficiency of merging p-values under arbitrary dependence

40 Pages Posted: 1 May 2020 Last revised: 17 Aug 2021

See all articles by Yuyu Chen

Yuyu Chen

University of Waterloo - Department of Statistics and Actuarial Science

Peng Liu

University of Essex

Ken Seng Tan

University of Waterloo

Ruodu Wang

University of Waterloo - Department of Statistics and Actuarial Science

Date Written: April 6, 2020

Abstract

Various methods of combining individual p-values into one p-value are widely used in many areas of statistical applications. We say that a combining method is valid for arbitrary dependence (VAD) if it does not require any assumption on the dependence structure of the p-values, whereas it is valid for some dependence (VSD) if it requires some specific, perhaps realistic but unjustifiable, dependence structures.The trade-off between validity and efficiency of these methods is studied via analyzing the choices of critical values under different dependence assumptions.
We introduce the notions of independence-comonotonicity balance (IC-balance)
and the price for validity. In particular, IC-balanced methods always produce an identical critical value for independent and perfectly positively dependent p-values, a specific type of insensitivity to a family of dependence assumptions. We show that, among two very general classes of merging methods commonly used in practice, the Cauchy combination method and the Simes method are the only IC-balanced ones. Simulation studies and a real data analysis are conducted to analyze the sizes and powers of various combining methods in the presence of weak and strong dependence.

Keywords: Hypothesis testing; multiple hypothesis testing; validity; efficiency

Suggested Citation

Chen, Yuyu and Liu, Peng and Tan, Ken Seng and Wang, Ruodu, Trade-off between validity and efficiency of merging p-values under arbitrary dependence (April 6, 2020). Available at SSRN: https://ssrn.com/abstract=3569329 or http://dx.doi.org/10.2139/ssrn.3569329

Yuyu Chen

University of Waterloo - Department of Statistics and Actuarial Science ( email )

200 University Avenue West
Waterloo, Ontario N2L 3G1
Croatia

Peng Liu (Contact Author)

University of Essex ( email )

Wivenhoe Park
Colchester, CO4
United Kingdom

Ken Seng Tan

University of Waterloo ( email )

Waterloo, Ontario N2L 3G1
Canada

Ruodu Wang

University of Waterloo - Department of Statistics and Actuarial Science ( email )

Waterloo, Ontario N2L 3G1
Canada

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
66
Abstract Views
559
rank
484,862
PlumX Metrics