False (and Missed) Discoveries in Financial Economics

89 Pages Posted: 21 Nov 2017 Last revised: 15 Aug 2019

See all articles by Campbell R. Harvey

Campbell R. Harvey

Duke University - Fuqua School of Business; National Bureau of Economic Research (NBER)

Yan Liu

Purdue University; Texas A&M University, Department of Finance

Date Written: August 14, 2019

Abstract

The multiple testing problem plagues many important issues in finance such as fund and factor selection. Many look good purely by luck. There are a number of statistical techniques to control for multiplicity that reduce Type I errors - but it is unknown by how much. We propose a new way to calibrate both Type I and Type II errors. We start with the researcher's prior belief on the proportion of managers that are skilled. Using a double bootstrap method, we then establish a t-statistic hurdle that is associated with a specific false discovery rate (e.g., 5%). We also establish a t-statistic hurdle that is associated with a certain acceptable ratio of misses to false discoveries (Type II error scaled by Type I error) | effectively allowing for differential costs of the two types of mistakes. Evaluating current methods within our framework, we find that they lack the power to detect outperforming managers.

Keywords: Type I, Type II, Multiple testing, False discoveries, Odds ratio, Power, Mutual funds, Smart beta, Anomalies, Bayesian, Factors, Backtesting, Factor Zoo

JEL Classification: G12, G14, C12, C21, C22, C31, C32

Suggested Citation

Harvey, Campbell R. and Liu, Yan, False (and Missed) Discoveries in Financial Economics (August 14, 2019). Available at SSRN: https://ssrn.com/abstract=3073799 or http://dx.doi.org/10.2139/ssrn.3073799

Campbell R. Harvey (Contact Author)

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States
919-660-7768 (Phone)

HOME PAGE: http://www.duke.edu/~charvey

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Yan Liu

Purdue University ( email )

West Lafayette, IN 47907-1310
United States

HOME PAGE: http://yliu1.com

Texas A&M University, Department of Finance ( email )

Wehner 401Q, MS 4353
College Station, TX 77843-4218
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
748
Abstract Views
3,885
rank
32,418
PlumX Metrics