Statistical Tests for Multiple Forecast Comparison

Abstract

We consider a multivariate version of the Diebold–Mariano test for equal predictive ability of three or more forecasting models. The Wald-type test, S, which has a null distribution that is asymptotically chi-squared, is shown to be generally invariant with respect to the ordering of the models being compared. Finite-sample corrections for the test are also developed. Monte Carlo simulations indicate that S has reasonable size properties in large samples but tends to be oversized in moderate samples. The finite-sample correction succeeds in correcting for size, but only partially. For the size-adjusted tests, power increases with sample size, as expected. It is speculated that further finite-sample improvements can be achieved using Hotelling’s T-square or bootstrap critical values.

Keywords: forecast comparison, multivariate tests of equal predictive ability, Diebold–Mariano test, finite-sample correction