Telling the Good from the Bad and the Ugly: How to Evaluate Backtested Investment Strategies

Abstract

A growing share of the world’s trading activity is generated by algorithmic investment strategies. Algorithms require development, backtesting, and investors that assume the initial performance risk. Evaluating the likelihood that backtested strategies will maintain their risk return profile in the future is an endeavor that requires experience and insight. In this paper, we describe a practical, non-technical approach to evaluating backtests that should help investors weed out those strategies the least likely to be profitable once launched in the marketplace. We also try to provide insight into why for most experienced investors the details of the statistical methods used to develop backtests are not the most critical immediate considerations when evaluating a potential investment.

Keywords: back-test, historical simulation, probability of back-test over-fitting, investment strategy, optimization, Sharpe ratio, minimum back-test length, performance degradation

JEL Classification: G0, G1, G2, G15, G24, E44