Pitfalls in Backtesting Historical Simulation VAR Models

Center for Applied Economics and Policy Research Working Paper No. 2012-003

41 Pages Posted: 21 Mar 2012  

Juan Carlos Escanciano

Indiana University Bloomington - Department of Economics

Pei Pei

Central University of Finance and Economics (CUFE) - Chinese Academy of Finance and Development

Date Written: February 21, 2012

Abstract

Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most widely used Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our findings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical findings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. Finally, our theoretical findings are confirmed in a Monte Carlo simulation study and an empirical application with three U.S. stocks. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.

Keywords: Backtesting, Basel Accord, Risk management, Value at Risk, Conditional quantile, Market Risk Capital requirements

JEL Classification: C52, C32, G21, G32

Suggested Citation

Escanciano, Juan Carlos and Pei, Pei, Pitfalls in Backtesting Historical Simulation VAR Models (February 21, 2012). Center for Applied Economics and Policy Research Working Paper No. 2012-003. Available at SSRN: https://ssrn.com/abstract=2026537 or http://dx.doi.org/10.2139/ssrn.2026537

Juan Carlos Escanciano

Indiana University Bloomington - Department of Economics ( email )

Wylie Hall
Bloomington, IN 47405-6620
United States
812-855-7925 (Phone)
812-855-3736 (Fax)

Pei Pei (Contact Author)

Central University of Finance and Economics (CUFE) - Chinese Academy of Finance and Development ( email )

39 South College Road
Beijing
China

Paper statistics

Downloads
400
Rank
59,536
Abstract Views
1,693