Stepwise Multiple Testing as Formalized Data Snooping

UPF Working Paper No. 712

36 Pages Posted: 11 Jul 2004

See all articles by Michael Wolf

Michael Wolf

University of Zurich - Department of Economics

Joseph P. Romano

Stanford University - Department of Statistics

Date Written: October 2003

Abstract

It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. In this paper, we suggest a stepwise multiple testing procedure which asymptotically controls the familywise error rate at a desired level. Compared to related single-step methods, our procedure is more powerful in the sense that it often will reject more false hypotheses.

Unlike some stepwise methods, our method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses. We prove our method asymptotically controls the familywise error rate under minimal assumptions. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.

Keywords: Bootstrap, data snooping, familywise error, multiple testing, step-down method

JEL Classification: C12, C14, C52

Suggested Citation

Wolf, Michael and Romano, Joseph P., Stepwise Multiple Testing as Formalized Data Snooping (October 2003). UPF Working Paper No. 712, Available at SSRN: https://ssrn.com/abstract=563209 or http://dx.doi.org/10.2139/ssrn.563209

Michael Wolf (Contact Author)

University of Zurich - Department of Economics ( email )

Wilfriedstrasse 6
Zurich, 8032
Switzerland

Joseph P. Romano

Stanford University - Department of Statistics ( email )

Stanford, CA 94305
United States