False (and Missed) Discoveries in Financial Economics

30 Pages Posted: 21 Nov 2017 Last revised: 3 Dec 2017

Campbell R. Harvey

Duke University - Fuqua School of Business; National Bureau of Economic Research (NBER); Duke Innovation & Entrepreneurship Initiative

Yan Liu

Texas A&M University, Department of Finance

Date Written: November 18, 2017


Investors make two types of mistakes. First, they erroneously allocate to an asset manager (or a “smart” beta) that underperforms because the asset manager lacks skill. Second, investors might miss out allocating to a good manager. The first mistake is difficult to deal with given there are thousands of managers and many look good purely by luck. We introduce a new technique that optimizes the threshold for a prespecified false discovery rate (i.e., chance of the first mistake), at say 5%. Our method also allows for heterogeneous false discoveries – we should not treat all bad managers the same because some are really, really bad. Next, we focus on the second type of error where investors miss out on good managers. It is routine to ignore this type of mistake. Our results show that current research methods have little or no power to detect good managers. Finally, our method allows for the asymmetric treatment of false discoveries and misses – generally, investing in a bad manager is more costly than missing a good manager. We also offer a way to select managers whereby the investor can prespecify the ratio of false discoveries to misses to accommodate these differential costs. For instance, we can accommodate a decision rule whereby the investor is willing to miss ten good managers to avoid the mistake of selecting a bad manager.

Keywords: Type I, Type II, Multiple testing, False discoveries, Odds ratio, Power, Mutual funds, Smart beta, Anomalies, Bayesian, Factors, Backtesting, Factor Zoo

JEL Classification: G12, G14, C12, C21, C22, C31, C32

Suggested Citation

Harvey, Campbell R. and Liu, Yan, False (and Missed) Discoveries in Financial Economics (November 18, 2017). Available at SSRN: https://ssrn.com/abstract=3073799 or http://dx.doi.org/10.2139/ssrn.3073799

Campbell R. Harvey (Contact Author)

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States
919-660-7768 (Phone)
919-660-8030 (Fax)

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Duke Innovation & Entrepreneurship Initiative ( email )

215 Morris St., Suite 300
Durham, NC 27701
United States

Yan Liu

Texas A&M University, Department of Finance ( email )

Wehner 401Q, MS 4353
College Station, TX 77843-4218
United States

Register to support our free research


Paper statistics

Abstract Views