Findings from Evidence-Based Forecasting: Methods for Reducing Forecast Error

Abstract

Empirical comparisons of reasonable approaches provide evidence on the best forecasting procedures to use under given conditions. Based on this evidence, I summarize the progress made over the past quarter century with respect to methods for reducing forecasting error. Seven well-established methods have been shown to improve accuracy: combining forecasts and Delphi help for all types of data; causal modeling, judgmental bootstrapping and structured judgment help with cross-sectional data; and causal models and trend-damping help with time-series data. Promising methods for cross-sectional data include damped causality, simulated interaction, structured analogies, and judgmental decomposition; for time-series data, they include segmentation, rule-based forecasting, damped seasonality, decomposition by causal forces, damped trend with analogous data, and damped seasonality. The testing of multiple hypotheses has also revealed methods where gains are limited: these include data mining, neural nets, and Box-Jenkins methods. Multiple hypotheses tests should be conducted on widely used but relatively untested methods such as prediction markets, conjoint analysis, diffusion models, and game theory.

Keywords: Box-Jenkins, causal forces, causal models, combining forecasts, complex series, conjoint analysis, contrary series, damped seasonality, damped trend, data mining, Delphi, diffusion, game theory, judgmental decomposition, multiple hypotheses, neural nets, prediction markets, rule-based forecasting