Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons
International Journal of Forecasting, Vol. 8, pp. 69-80, 1992
18 Pages Posted: 7 Feb 2005 Last revised: 22 Jul 2008
Abstract
This study evaluated measures for making comparisons of errors across time series. We analyzed 90 annual and 101 quarterly economic time series. We judged error measures on reliability, construct validity, sensitivity to small changes, protection against outliers, and their relationship to decision making. The results lead us to recommend the Geometric Mean of the Relative Absolute Error (GMRAE) when the task involves calibrating a model for a set of time series. The GMRAE compares the absolute error of a given method to that from the random walk forecast. For selecting the most accurate methods, we recommend the Median RAE (MdRAE) when few series are available and the Median Absolute Percentage Error (MdAPE) otherwise. The Root Mean Square Error (RMSE) is not reliable, and is therefore inappropriate for comparing accuracy across series.
Keywords: Forecast accuracy, M-Competition, Relative absolute error, Theil's U
Suggested Citation: Suggested Citation
Do you have a job opening that you would like to promote on SSRN?
Recommended Papers
-
By Fred Collopy and J. Scott Armstrong
-
By J. Scott Armstrong and Robert Carbone
-
By J. Scott Armstrong and Fred Collopy
-
Causal Forces: Structuring Knowledge for Time-Series Extrapolation
By J. Scott Armstrong and Fred Collopy
-
Expert Opinions About Extrapolation and the Mystery of the Overlooked Discontinuities
By Fred Collopy and J. Scott Armstrong
-
Beyond Accuracy: Comparison of Criteria Used to Select Forecasting Methods
By Thomas Yokum and J. Scott Armstrong
-
Structuring Knowledge Retrieval: An Analysis of Decomposed Quantitative Judgments
By Donald G. Macgregor, Sarah Lichtenstein, ...
-
Identification of Asymmetric Prediction Intervals Through Causal Forces
By J. Scott Armstrong and Fred Collopy