Reality Checks and Nested Forecast Model Comparisons

42 Pages Posted: 5 Oct 2010

See all articles by Todd E. Clark

Todd E. Clark

Federal Reserve Bank of Cleveland

Michael W. McCracken

Federal Reserve Banks - Federal Reserve Bank of St. Louis

Date Written: September 1, 2010

Abstract

This paper develops a novel and effective bootstrap method for simulating asymptotic critical values for tests of equal forecast accuracy and encompassing among many nested models. The bootstrap, which combines elements of fixed regressor and wild bootstrap methods, is simple to use. We first derive the asymptotic distributions of tests of equal forecast accuracy and encompassing applied to forecasts from multiple models that nest the benchmark model – that is, reality check tests applied to nested models. We then prove the validity of the bootstrap for these tests. Monte Carlo experiments indicate that our proposed bootstrap has better finite-sample size and power than other methods designed for comparison of non-nested models. We conclude with empirical applications to multiple-model forecasts of commodity prices and GDP growth.

Keywords: Prediction, forecast evaluation, equal accuracy

JEL Classification: C53, C12, C52

Suggested Citation

Clark, Todd E. and McCracken, Michael W., Reality Checks and Nested Forecast Model Comparisons (September 1, 2010). Available at SSRN: https://ssrn.com/abstract=1687219 or http://dx.doi.org/10.2139/ssrn.1687219

Todd E. Clark

Federal Reserve Bank of Cleveland ( email )

P.O. Box 6387
Cleveland, OH 44101
United States
216-579-2015 (Phone)

Michael W. McCracken (Contact Author)

Federal Reserve Banks - Federal Reserve Bank of St. Louis ( email )

411 Locust St
Saint Louis, MO 63011
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
98
Abstract Views
893
Rank
486,005
PlumX Metrics