Robust Forecast Comparison

57 Pages Posted: 14 May 2015 Last revised: 15 Mar 2016

See all articles by Sainan Jin

Sainan Jin

Singapore Management University

Valentina Corradi

University of Surrey - School of Economics

Norman R. Swanson

Rutgers University - Department of Economics; Rutgers, The State University of New Jersey - Department of Economics

Date Written: February 24, 2016

Abstract

Forecast accuracy is typically measured in terms of a given loss function. However, as a consequence of the use of misspecified models in multiple model comparisons, relative forecast rankings are loss function dependent. This paper addresses this issue by using a novel criterion for forecast evaluation which is based on the entire distribution of forecast errors. We introduce the concepts of general-loss (GL) forecast superiority and convex-loss (CL) forecast superiority; and develop tests for GL (CL) superiority that are based on an out-of-sample generalization of the tests introduced by Linton, Maasoumi and Whang (2005). The asymptotic null distributions of our test statistics are nonstandard, and resampling procedures are used to obtain critical values. Additionally, the tests are consistent and have nontrivial local power under a sequence of local alternatives. In addition to the stationary case, we outline theory extending our tests to the case of heterogeneity induced by distributional change over time. Monte Carlo simulations suggest that the tests perform reasonably well in finite samples; and an application to exchange rate data indicates that our tests can help identify superior forecasting models, regardless of loss function.

Keywords: Convex loss function, Empirical processes, Forecast superiority, General loss function

JEL Classification: C12, C22

Suggested Citation

Jin, Sainan and Corradi, Valentina and Swanson, Norman Rasmus and Swanson, Norman Rasmus, Robust Forecast Comparison (February 24, 2016). Available at SSRN: https://ssrn.com/abstract=2605927 or http://dx.doi.org/10.2139/ssrn.2605927

Sainan Jin (Contact Author)

Singapore Management University ( email )

Li Ka Shing Library
70 Stamford Road
Singapore 178901, 178899
Singapore

Valentina Corradi

University of Surrey - School of Economics ( email )

Guildford
Guildford, Surrey GU2 5XH
United Kingdom

Norman Rasmus Swanson

Rutgers, The State University of New Jersey - Department of Economics ( email )

75 Hamilton Street
New Brunswick, NJ 08901
United States
848-932-7432 (Phone)

HOME PAGE: http://econweb.rutgers.edu/nswanson/

Rutgers University - Department of Economics ( email )

NJ
United States

HOME PAGE: http://econweb.rutgers.edu/nswanson/

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
123
Abstract Views
983
Rank
444,818
PlumX Metrics