Backtesting Correlated Quantities

12 Pages Posted: 26 Sep 2023

See all articles by Nikolai Nowaczyk

Nikolai Nowaczyk

Independent

Vladimir Piterbarg

NatWest Markets; Imperial College London

Date Written: September 13, 2023

Abstract

Backtesting financial models over long horizons inevitably leads to correlated samples due to overlapping windows. As a result, the null distributions of popular test statistics like exceedence counting or chi-squared are no longer analytically given. A standard technique to solve this problem is to calculate these distributions numerically via Monte Carlo simulation. While this automatically accounts for the correlation in the samples, the resulting distributions can sometimes have very long tails, which results in significant loss of discriminatory power of the test. We introduce a simple yet effective pre-processing technique based on decorrelating the samples, hence making them compatible with standard statistical tests. We provide various numerical examples showing that this leads to more stable distributions, higher discriminatory power and natural generalizations to jointly backtesting multiple correlated quantities in a clean consistent framework.

Keywords: Backtesting, CCR, statistics, hypothesis test, discriminatory power, model validation

JEL Classification: C12, C15, C22, C52

Suggested Citation

Nowaczyk, Nikolai and Piterbarg, Vladimir, Backtesting Correlated Quantities (September 13, 2023). Available at SSRN: https://ssrn.com/abstract=4571812 or http://dx.doi.org/10.2139/ssrn.4571812

Vladimir Piterbarg

NatWest Markets ( email )

250 Bishopsgate
London, EC2M 4AA
United Kingdom

Imperial College London ( email )

South Kensington Campus
Imperial College
LONDON, SW7 2AZ
United Kingdom

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
506
Abstract Views
914
Rank
97,314
PlumX Metrics