An Evaluation of Alternative Multiple Testing Methods for Finance Applications

51 Pages Posted: 13 Dec 2019 Last revised: 2 Feb 2020

See all articles by Campbell R. Harvey

Campbell R. Harvey

Duke University - Fuqua School of Business; National Bureau of Economic Research (NBER)

Yan Liu

Purdue University

Alessio Saretto

University of Texas at Dallas - School of Management - Department of Finance & Managerial Economics

Date Written: February 2, 2020

Abstract

In almost every area of empirical finance, researchers are confronted with multiple tests. One high profile example is the identification of investment managers that outperform. Many beat their benchmarks purely by luck. Multiple testing methods are designed to control for luck. Factor selection is another glaring case. However, there are numerous other applications that do not get as much attention. Importantly, for example, in a simple regression model where, say, five variables are tested, a t-statistic of 2.0 is not enough to establish significance — because five variables were tried. Our paper provides a guide to various multiple testing methods and details a number of applications. We provide simulation evidence on the relative performance of different methods across a variety of testing environments. The goal of our paper is to provide a menu that researchers can choose from to improve inference in financial economics.

Keywords: Multiple hypothesis testing, False rejections, False discovery rate, False non-discovery rate, False omission rate, Family-wise error rate, Data mining, Data snooping, Type I error, Type II error, False discovery control, Luck, Test power

JEL Classification: G0, G1, G2, G3, G5, M4, C1, C5

Suggested Citation

Harvey, Campbell R. and Liu, Yan and Saretto, Alessio, An Evaluation of Alternative Multiple Testing Methods for Finance Applications (February 2, 2020). Available at SSRN: https://ssrn.com/abstract=3480087 or http://dx.doi.org/10.2139/ssrn.3480087

Campbell R. Harvey (Contact Author)

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States
919-660-7768 (Phone)

HOME PAGE: http://www.duke.edu/~charvey

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Yan Liu

Purdue University ( email )

West Lafayette, IN 47907-1310
United States

HOME PAGE: http://yliu1.com

Alessio Saretto

University of Texas at Dallas - School of Management - Department of Finance & Managerial Economics ( email )

800 Campbell Road
SM 31
Richardson, TX 75080
United States
972-883-5907 (Phone)

HOME PAGE: http://www.utdallas.edu/~axs125732

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
230
Abstract Views
881
rank
142,556
PlumX Metrics