Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests

18 Pages Posted: 11 Sep 2012

See all articles by Francis X. Diebold

Francis X. Diebold

University of Pennsylvania - Department of Economics; National Bureau of Economic Research (NBER)

Multiple version iconThere are 3 versions of this paper

Date Written: September 7, 2012

Abstract

The Diebold-Mariano (DM) test was intended for comparing forecasts; it has been, and remains, useful in that regard. The DM test was not intended for comparing models. Unfortunately, however, much of the large subsequent literature uses DM-type tests for comparing models, in (pseudo-) out-of-sample environments. In that case, much simpler yet more compelling full-sample model comparison procedures exist; they have been, and should continue to be, widely used. The hunch that (pseudo-) out-of-sample analysis is somehow the “only," or “best," or even a “good" way to provide insurance against in-sample over fitting in model comparisons proves largely false. On the other hand, (pseudo-) out-of-sample analysis may be useful for learning about comparative historical predictive performance.

Keywords: Forecasting, model comparison, model selection, out-of-sample tests

JEL Classification: C01, C53

Suggested Citation

Diebold, Francis X., Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests (September 7, 2012). PIER Working Paper No. 12-035, Available at SSRN: https://ssrn.com/abstract=2144210 or http://dx.doi.org/10.2139/ssrn.2144210

Francis X. Diebold (Contact Author)

University of Pennsylvania - Department of Economics ( email )

Ronald O. Perelman Center for Political Science
133 South 36th Street
Philadelphia, PA 19104-6297
United States
215-898-1507 (Phone)
215-573-4217 (Fax)

HOME PAGE: http://www.ssc.upenn.edu/~fdiebold/

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
102
Abstract Views
796
Rank
240,169
PlumX Metrics