Stepwise Multiple Testing as Formalized Data Snooping
UPF Working Paper No. 712
36 Pages Posted: 11 Jul 2004
Date Written: October 2003
Abstract
It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. In this paper, we suggest a stepwise multiple testing procedure which asymptotically controls the familywise error rate at a desired level. Compared to related single-step methods, our procedure is more powerful in the sense that it often will reject more false hypotheses.
Unlike some stepwise methods, our method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses. We prove our method asymptotically controls the familywise error rate under minimal assumptions. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.
Keywords: Bootstrap, data snooping, familywise error, multiple testing, step-down method
JEL Classification: C12, C14, C52
Suggested Citation: Suggested Citation
Do you have negative results from your research you’d like to share?
Recommended Papers
-
Improved Nonparametric Confidence Intervals in Time Series Regressions
By Joseph P. Romano and Michael Wolf
-
By James J. Heckman, Seong Moon, ...
-
Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing
By Joseph P. Romano and Michael Wolf
-
Balanced Control of Generalized Error Rates
By Joseph P. Romano and Michael Wolf
-
Control of the False Discovery Rate Under Dependence Using the Bootstrap and Subsampling
By Joseph P. Romano, Azeem Shaikh, ...
-
Hypothesis Testing in Econometrics
By Joseph P. Romano, Azeem Shaikh, ...
-
Inference with Imperfect Randomization: The Case of the Perry Preschool Program
By James J. Heckman, Rodrigo R. Pinto, ...
-
Primate Evidence on the Late Health Effects of Early Life Adversity
By Gabriella Conti, Christopher Hansman, ...
-
By Ashok Kaul and Michael Wolf
-
Optimal Testing of Multiple Hypotheses with Common Effect Direction
By Richard M. Bittman, Joseph P. Romano, ...