Comparing Forecast Accuracy: A Monte Carlo Investigation

Abstract

The size and power properties of several tests of equal Mean Square Prediction Error (MSPE) and of Forecast Encompassing (FE) are evaluated, using Monte Carlo simulations, in the context of dynamic regressions. For nested models, the F-type test of forecast encompassing proposed by Clark and McCracken (2001) displays overall the best properties. However its power advantage tends to become smaller as the prediction sample increases and for multi-step ahead predictions; in these cases a standard FE test based on Gaussian critical values becomes relatively more attractive. The ranking among the tests remains broadly unaltered for one-step and multi-step ahead predictions, for partially misspecified models and for highly persistent data. A similar setup is then used to analyze the case of non-nested models. Again it is found that FE tests have a significantly better performance than tests of equal MSPE for discriminating between correct and misspecified models. An empirical application evaluates the predictive ability of nested and non-nested models for GDP in Italy and the euro-area.

Keywords: Forecast encompassing, Model evaluation, Nested models, Non-nested models, Equal predictive ability

JEL Classification: C12, C52, C53