Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests
18 Pages Posted: 11 Sep 2012
There are 3 versions of this paper
Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests
Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests
Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests
Date Written: September 7, 2012
Abstract
The Diebold-Mariano (DM) test was intended for comparing forecasts; it has been, and remains, useful in that regard. The DM test was not intended for comparing models. Unfortunately, however, much of the large subsequent literature uses DM-type tests for comparing models, in (pseudo-) out-of-sample environments. In that case, much simpler yet more compelling full-sample model comparison procedures exist; they have been, and should continue to be, widely used. The hunch that (pseudo-) out-of-sample analysis is somehow the “only," or “best," or even a “good" way to provide insurance against in-sample over fitting in model comparisons proves largely false. On the other hand, (pseudo-) out-of-sample analysis may be useful for learning about comparative historical predictive performance.
Keywords: Forecasting, model comparison, model selection, out-of-sample tests
JEL Classification: C01, C53
Suggested Citation: Suggested Citation
Do you have negative results from your research you’d like to share?
Recommended Papers
-
Tests of Equal Forecast Accuracy and Encompassing for Nested Models
-
Long Swings in the Exchange Rate: are They in the Data and Do Markets Know it?
-
Exchange Rates and Fundamentals
By Charles M. Engel and Kenneth D. West
-
Exchange Rates and Fundamentals
By Charles M. Engel and Kenneth D. West
-
Empirical Exchange Rate Models of the Nineties: Are Any Fit to Survive?
-
Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?
By Lutz Kilian
-
Empirical Exchange Rate Models of the Nineties: Are Any Fit to Survive?
By Yin-wong Cheung, Menzie David Chinn, ...