Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance
David H. Bailey
Lawrence Berkeley National Laboratory
Jonathan M. Borwein
University of Newcastle (Australia); Royal Society of Canada; Australian Academy of Science
Marcos Lopez de Prado
Hess Energy Trading Company; Lawrence Berkeley National Laboratory; RCC at Harvard University
Qiji Jim Zhu
Western Michigan University
January 26, 2014
Notices of the American Mathematical Society, 61(5), May 2014, Forthcoming
Recent computational advances allow investment managers to search for profitable investment strategies. In many instances, that search involves a pseudo-mathematical argument, which is spuriously validated through a simulation of its historical performance (also called backtest).
We prove that high performance is easily achievable after backtesting a relatively small number of alternative strategy configurations, a practice we denote “backtest overfitting”. The higher the number of configurations tried, the greater is the probability that the backtest is overfit. Because financial analysts rarely report the number of configurations tried for a given backtest, investors cannot evaluate the degree of overfitting in most investment proposals.
The implication is that investors can be easily misled into allocating capital to strategies that appear to be mathematically sound and empirically supported by an outstanding backtest. This practice is particularly pernicious, because due to the nature of financial time series, backtest overfitting has a detrimental effect on the future strategy’s performance.
Number of Pages in PDF File: 34
Keywords: backtest, historical simulation, probability of backtest over-fitting, investment strategy, optimization, Sharpe ratio, minimum backtest length, performance degradation
JEL Classification: G0, G1, G2, G15, G24, E44Accepted Paper Series
Date posted: August 12, 2013 ; Last revised: January 31, 2014
© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.
This page was processed by apollo6 in 0.407 seconds