Backtesting Correlated Quantities

12 Pages Posted: 26 Sep 2023

See all articles by Nikolai Nowaczyk

Nikolai Nowaczyk


Vladimir Piterbarg

NatWest Markets; Imperial College London

Date Written: September 13, 2023


Backtesting financial models over long horizons inevitably leads to correlated samples due to overlapping windows. As a result, the null distributions of popular test statistics like exceedence counting or chi-squared are no longer analytically given. A standard technique to solve this problem is to calculate these distributions numerically via Monte Carlo simulation. While this automatically accounts for the correlation in the samples, the resulting distributions can sometimes have very long tails, which results in significant loss of discriminatory power of the test. We introduce a simple yet effective pre-processing technique based on decorrelating the samples, hence making them compatible with standard statistical tests. We provide various numerical examples showing that this leads to more stable distributions, higher discriminatory power and natural generalizations to jointly backtesting multiple correlated quantities in a clean consistent framework.

Keywords: Backtesting, CCR, statistics, hypothesis test, discriminatory power, model validation

JEL Classification: C12, C15, C22, C52

Suggested Citation

Nowaczyk, Nikolai and Piterbarg, Vladimir, Backtesting Correlated Quantities (September 13, 2023). Available at SSRN: or

Vladimir Piterbarg

NatWest Markets ( email )

250 Bishopsgate
London, EC2M 4AA
United Kingdom

Imperial College London ( email )

South Kensington Campus
Imperial College
United Kingdom

Do you have negative results from your research you’d like to share?

Paper statistics

Abstract Views
PlumX Metrics