An Evaluation of Alternative Multiple Testing Methods for Finance Applications
51 Pages Posted: 13 Dec 2019 Last revised: 2 Feb 2020
Date Written: February 2, 2020
In almost every area of empirical finance, researchers are confronted with multiple tests. One high profile example is the identification of investment managers that outperform. Many beat their benchmarks purely by luck. Multiple testing methods are designed to control for luck. Factor selection is another glaring case. However, there are numerous other applications that do not get as much attention. Importantly, for example, in a simple regression model where, say, five variables are tested, a t-statistic of 2.0 is not enough to establish significance — because five variables were tried. Our paper provides a guide to various multiple testing methods and details a number of applications. We provide simulation evidence on the relative performance of different methods across a variety of testing environments. The goal of our paper is to provide a menu that researchers can choose from to improve inference in financial economics.
Keywords: Multiple hypothesis testing, False rejections, False discovery rate, False non-discovery rate, False omission rate, Family-wise error rate, Data mining, Data snooping, Type I error, Type II error, False discovery control, Luck, Test power
JEL Classification: G0, G1, G2, G3, G5, M4, C1, C5
Suggested Citation: Suggested Citation