The Probability of Backtest Overfitting

31 Pages Posted: 21 Sep 2016

See all articles by David H. Bailey

David H. Bailey

Lawrence Berkeley National Laboratory; University of California, Davis

Jonathan Borwein

University of Newcastle (Australia); Royal Society of Canada; Australian Academy of Science (deceased)

Marcos Lopez de Prado

Cornell University - Operations Research & Industrial Engineering; AQR Capital Management, LLC

Qiji Jim Zhu

Western Michigan University

Multiple version iconThere are 2 versions of this paper

Date Written: September 19, 2016

Abstract

Many investment firms and portfolio managers rely on backtests (ie, simulations of performance based on historical market data) to select investment strategies and allocate capital. Standard statistical techniques designed to prevent regression overfitting, such as hold-out, tend to be unreliable and inaccurate in the context of investment backtests. We propose a general framework to assess the probability of backtest overfitting (PBO).We illustrate this framework with specific generic, model-free and nonparametric implementations in the context of investment simulations; we call these implementations combinatorially symmetric cross-validation (CSCV). We show that CSCV produces reasonable estimates of PBO for several useful examples.

Keywords: backtest, overfitting, investment strategy, Sharpe ratio optimization, performance degradation

Suggested Citation

Bailey, David H. and Borwein, Jonathan and López de Prado, Marcos and Zhu, Qiji Jim, The Probability of Backtest Overfitting (September 19, 2016). Journal of Computational Finance, Forthcoming. Available at SSRN: https://ssrn.com/abstract=2840838

David H. Bailey

Lawrence Berkeley National Laboratory ( email )

1 Cyclotron Road
Berkeley, CA 94720
United States

HOME PAGE: http://www.davidhbailey.com

University of California, Davis ( email )

One Shields Avenue
Davis, CA 95616
United States

HOME PAGE: http://www.davidhbailey.com

Jonathan Borwein

University of Newcastle (Australia)

Royal Society of Canada

Australian Academy of Science (deceased)

Marcos López de Prado (Contact Author)

Cornell University - Operations Research & Industrial Engineering ( email )

237 Rhodes Hall
Ithaca, NY 14853
United States

HOME PAGE: http://www.orie.cornell.edu

AQR Capital Management, LLC

One Greenwich Plaza
Greenwich, CT 06830
United States

HOME PAGE: http://www.aqr.com

Qiji Jim Zhu

Western Michigan University ( email )

Kalamazoo, MI 49008
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
0
Abstract Views
1,318
PlumX Metrics