Adjusting for Information Content When Comparing Forecast Performance

Abstract

Cross institutional forecast evaluations may be severely distorted by the fact that forecasts are made at different points in time, and thus with different amount of information. This paper proposes a method to account for these differences. The method computes the timing effect and the forecaster's ability simultaneously. Monte Carlo simulation demonstrate that evaluations that do not adjust for the differences in information content may be misleading. In addition, the method is applied on a real-world data set of 10 Swedish forecasters for the period 1999-2015. The results show that the ranking of the forecasters is affected by the proposed adjustment.

Keywords: Forecast error, Forecast comparison, Publication time, Evaluation, Error component model, Panel data

JEL Classification: C23, C53, E37