A Bayesian Approach to Backtest Overfitting

Posted: 19 Jul 2017 Last revised: 21 Jun 2018

See all articles by Jiri Witzany

Jiri Witzany

University of Economics in Prague

Date Written: June 30, 2017

Abstract

Quantitative investment strategies are often selected from a broad class of candidate models estimated and tested on historical data. Standard statistical technique to prevent model overfitting such as out-sample back-testing turns out to be unreliable in the situation when selection is based on results of too many models tested on the holdout sample. There is an ongoing discussion how to estimate the probability of back-test overfitting and adjust the expected performance indicators like Sharpe ratio in order to reflect properly the effect of multiple testing. We propose a consistent Bayesian approach that consistently yields the desired robust estimates based on an MCMC simulation. The approach is tested on a class of technical trading strategies where a seemingly profitable strategy can be selected in the naïve approach.

Keywords: Backtest, Multiple Testing, Bootstrapping, Cross-Validation, Probability of Backtest Overfitting, Investment Strategy, Optimization, Sharpe Ratio, Bayesian Probability, MCMC

JEL Classification: G1, G2, C5, G24, C11, C12, C52

Suggested Citation

Witzany, Jiri, A Bayesian Approach to Backtest Overfitting (June 30, 2017). Available at SSRN: https://ssrn.com/abstract=3002503 or http://dx.doi.org/10.2139/ssrn.3002503

Jiri Witzany (Contact Author)

University of Economics in Prague ( email )

Winston Churchilla Sq. 4
Prague 3, 130 67
Czech Republic

Register to save articles to
your library

Register

Paper statistics

Abstract Views
346
PlumX Metrics