Pitfalls in Backtesting Historical Simulation VAR Models
Juan Carlos Escanciano
Indiana University Bloomington - Department of Economics
Central University of Finance and Economics (CUFE) - Chinese Academy of Finance and Development
February 21, 2012
Center for Applied Economics and Policy Research Working Paper No. 2012-003
Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most widely used Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our findings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical findings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. Finally, our theoretical findings are confirmed in a Monte Carlo simulation study and an empirical application with three U.S. stocks. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.
Number of Pages in PDF File: 41
Keywords: Backtesting, Basel Accord, Risk management, Value at Risk, Conditional quantile, Market Risk Capital requirements
JEL Classification: C52, C32, G21, G32working papers series
Date posted: March 21, 2012
© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.
This page was processed by apollo1 in 0.891 seconds