Abstract

http://ssrn.com/abstract=2308659
 


 



Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance


David H. Bailey


Lawrence Berkeley National Laboratory; University of California, Davis

Jonathan M. Borwein


University of Newcastle (Australia); Royal Society of Canada; Australian Academy of Science

Marcos Lopez de Prado


Guggenheim Partners, LLC; Lawrence Berkeley National Laboratory; RCC at Harvard University

Qiji Jim Zhu


Western Michigan University

April 1, 2014

Notices of the American Mathematical Society, 61(5), May 2014, pp.458-471

Abstract:     
We prove that high simulated performance is easily achievable after backtesting a relatively small number of alternative strategy configurations, a practice we denote “backtest overfitting”. The higher the number of configurations tried, the greater is the probability that the backtest is overfit. Because most financial analysts and academics rarely report the number of configurations tried for a given backtest, investors cannot evaluate the degree of overfitting in most investment proposals.

The implication is that investors can be easily misled into allocating capital to strategies that appear to be mathematically sound and empirically supported by an outstanding backtest. Under memory effects, backtest overfitting leads to negative expected returns out-of-sample, rather than zero performance. This may be one of several reasons why so many quantitative funds appear to fail.

Number of Pages in PDF File: 14

Keywords: backtest, historical simulation, probability of backtest over-fitting, investment strategy, optimization, Sharpe ratio, minimum backtest length, performance degradation

JEL Classification: G0, G1, G2, G15, G24, E44

Accepted Paper Series


Download This Paper

Date posted: August 12, 2013 ; Last revised: April 14, 2014

Suggested Citation

Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim, Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance (April 1, 2014). Notices of the American Mathematical Society, 61(5), May 2014, pp.458-471. Available at SSRN: http://ssrn.com/abstract=2308659 or http://dx.doi.org/10.2139/ssrn.2308659

Contact Information

David H. Bailey
Lawrence Berkeley National Laboratory ( email )
One Cyclotron Road
Berkeley, CA 94720
United States
HOME PAGE: http://www.davidhbailey.com
University of California, Davis ( email )
One Shields Avenue
Davis, CA 95616
United States
HOME PAGE: http://www.davidhbailey.com
Jonathan M. Borwein
University of Newcastle (Australia) ( email )
University Drive
Callaghan, NSW 2308
Australia
+61 2 4921 5535 (Phone)
HOME PAGE: http://www.carma.newcastle.edu.au/jon
Royal Society of Canada ( email )
282 Somerset West
Ottawa, Ontario K2P 0J6
Canada
Australian Academy of Science ( email )
Ian Potter House
Gordon Street
Acton, ACT 2601
Australia
Marcos Lopez de Prado (Contact Author)
Guggenheim Partners, LLC ( email )
330 Madison Avenue
New York, NY 10017
United States
HOME PAGE: http://www.QuantResearch.info
Lawrence Berkeley National Laboratory ( email )
One Cyclotron Road
Berkeley, CA 94720
United States
HOME PAGE: http://www.lbl.gov
RCC at Harvard University ( email )
26 Trowbridge Street
Cambridge, MA 02138
United States
HOME PAGE: http://www.rcc.harvard.edu
Qiji Jim Zhu
Western Michigan University ( email )
Kalamazoo, MI 49008
United States
Feedback to SSRN


Paper statistics
Abstract Views: 9,538
Downloads: 3,722
Download Rank: 1,102

© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.  FAQ   Terms of Use   Privacy Policy   Copyright   Contact Us
This page was processed by apollo4 in 1.766 seconds