Testing Forecast Accuracy of Expectiles and Quantiles with the Extremal Consistent Loss Functions

85 Pages Posted: 16 Nov 2016 Last revised: 28 Nov 2020

See all articles by Yu-Min Yen

Yu-Min Yen

Department of International Business, National Chengchi University

Tso-Jung Yen

Academia Sinica - Institute of Statistical Science

Date Written: July 15, 2018

Abstract

Forecast evaluations aim to choose an accurate forecast for making decisions by using loss functions. However, different loss functions often generate different ranking results for forecasts, which complicates the task of comparisons. In this paper, we develop statistical tests for comparing performances of forecasting expectiles and quantiles of a random variable under consistent loss functions. The test statistics are constructed with the extremal consistent loss functions of Ehm et al. (2016). The null hypothesis of the tests is that a benchmark forecast at least performs equally well as a competing one under all extremal consistent loss functions. It can be shown that if such a null holds, the benchmark will also perform at least equally well as the competitor under all consistent loss functions. Thus under the null, when different consistent loss functions are used, the result that the competitor does not outperform the benchmark will not be altered. We establish asymptotic properties of the proposed test statistics and propose to use the re-centered bootstrap to construct their empirical distributions. Through simulations, we show the proposed test statistics perform reasonably well. We then apply the proposed method on evaluations of several different forecast methods.

Keywords: Consistent loss function, Expectile, Extremal consistent loss function, Quantile

JEL Classification: C12, C53, E17

Suggested Citation

Yen, Yu-Min and Yen, Tso-Jung, Testing Forecast Accuracy of Expectiles and Quantiles with the Extremal Consistent Loss Functions (July 15, 2018). International Journal of Forecasting, accepted, Available at SSRN: https://ssrn.com/abstract=2869657 or http://dx.doi.org/10.2139/ssrn.2869657

Yu-Min Yen (Contact Author)

Department of International Business, National Chengchi University ( email )

64, Section 2, Zhi-nan Road
Wenshan
Taipei, 116
Taiwan

Tso-Jung Yen

Academia Sinica - Institute of Statistical Science ( email )

Nankang
Taipei, 11529
Taiwan

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
63
Abstract Views
475
Rank
627,735
PlumX Metrics