A New Approach for Detecting Shifts in Forecast Accuracy

28 Pages Posted: 18 Apr 2018

Date Written: April 13, 2018

Abstract

Forecasts play a critical role at inflation-targeting central banks, such as the Bank of England. Breaks in the forecast performance of a model can potentially incur important policy costs. Commonly used statistical procedures, however, implicitly put a lot of weight on type I errors (or false positives), which result in a relatively low power of tests to identify forecast breakdowns in small samples. We develop a procedure which aims at capturing the policy cost of missing a break. We use data-based rules to find the test size that optimally trades off the costs associated with false positives with those that can result from a break going undetected for too long. In so doing, we also explicitly study forecast errors as a multivariate system. The covariance between forecast errors for different series, though often overlooked in the forecasting literature, not only enables us to consider testing in a multivariate setting but also increases the test power. As a result, we can tailor the choice of the critical values for each series not only to the in-sample properties of each series but also to how the series for forecast errors covary.

Keywords: Forecast breaks, statistical decision making, central banking

JEL Classification: C53, E47, E58

Suggested Citation

Chiu, Ching-Wai (Jeremy) and Hayes, Simon and Kapetanios, George and Theodoridis, Konstantinos, A New Approach for Detecting Shifts in Forecast Accuracy (April 13, 2018). Bank of England Working Paper No. 721. Available at SSRN: https://ssrn.com/abstract=3163414 or http://dx.doi.org/10.2139/ssrn.3163414

Ching-Wai (Jeremy) Chiu (Contact Author)

Bank of England ( email )

Threadneedle Street
London, EC2R 8AH
United Kingdom

Simon Hayes

Bank of England ( email )

Threadneedle Street
London, EC2R 8AH
United Kingdom

George Kapetanios

King's College, London ( email )

30 Aldwych
London, WC2B 4BG
United Kingdom
+44 20 78484951 (Phone)

Konstantinos Theodoridis

Cardiff University ( email )

Aberconway Building
Colum Drive
Cardiff, Wales CF10 3EU
United Kingdom

Register to save articles to
your library

Register

Paper statistics

Downloads
10
Abstract Views
151
PlumX Metrics