Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons

International Journal of Forecasting, Vol. 8, pp. 69-80, 1992

18 Pages Posted: 7 Feb 2005 Last revised: 1 Jan 2012

See all articles by J. Scott Armstrong

J. Scott Armstrong

University of Pennsylvania - Marketing Department

Fred Collopy

Case Western Reserve University - Department of Information Systems

Abstract

This study evaluated measures for making comparisons of errors across time series. We analyzed 90 annual and 101 quarterly economic time series. We judged error measures on reliability, construct validity, sensitivity to small changes, protection against outliers, and their relationship to decision making. The results lead us to recommend the Geometric Mean of the Relative Absolute Error (GMRAE) when the task involves calibrating a model for a set of time series. The GMRAE compares the absolute error of a given method to that from the random walk forecast. For selecting the most accurate methods, we recommend the Median RAE (MdRAE) when few series are available and the Median Absolute Percentage Error (MdAPE) otherwise. The Root Mean Square Error (RMSE) is not reliable, and is therefore inappropriate for comparing accuracy across series.

Keywords: Forecast accuracy, M-Competition, Relative absolute error, Theil's U

Suggested Citation

Armstrong, J. Scott and Collopy, Fred, Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons. International Journal of Forecasting, Vol. 8, pp. 69-80, 1992. Available at SSRN: https://ssrn.com/abstract=662701

J. Scott Armstrong (Contact Author)

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States
215-898-5087 (Phone)
215-898-2534 (Fax)

HOME PAGE: http://marketing.wharton.upenn.edu/people/faculty/armstrong.cfm

Fred Collopy

Case Western Reserve University - Department of Information Systems ( email )

10900 Euclid Ave.
Cleveland, OH 44106-7235
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
266
Abstract Views
1,654
rank
114,569
PlumX Metrics