Detecting and Predicting Forecast Breakdowns

Abstract

We propose a theoretical framework for assessing whether a forecast model estimated over one period can provide good forecasts over a subsequent period. We formalize this idea by defining a forecast breakdown as a situation in which the out-of-sample performance of the model, judged by some loss function, is significantly worse than its in-sample performance. Our framework, which is valid under general conditions, can be used not only to detect past forecast breakdowns but also to predict future ones. We show that main causes of forecast breakdowns are instabilities in the data generating process and relate the properties of our forecast breakdown test to those of existing structural break tests. The empirical application finds evidence of a forecast breakdown in the Phillips' curve forecasts of U.S. inflation, and links it to inflation volatility and to changes in the monetary policy reaction function of the Fed.

Keywords: Structural change, Forecast evaluation, Forecast rationality testing, In-sample evaluation, Out-of-sample evaluation

JEL Classification: C22, C52, C53