Measurement Error Sensitivity of Loss Functions for Distribution Forecasts

45 Pages Posted: 8 Nov 2019 Last revised: 25 Feb 2021

See all articles by Onno Kleen

Onno Kleen

Erasmus University Rotterdam (EUR) - Erasmus School of Economics (ESE)

Date Written: February 25, 2021

Abstract

I examine the sensitivity of loss functions—equivalently called scoring rules—for distribution forecasts in two dimensions: sensitivity to linear rescaling of the data and the influence of measurement error on the forecast evaluation outcome. First, I show that all commonly used scoring rules for distribution forecasts are robust to rescaling the data. Second, it is revealed that the forecast ranking based on the continuous ranked probability score is less sensitive to measurement error than the ranking based on the log score. The theoretical results are complemented by a simulation study aligned with frequently revised quarterly US GDP growth data and an empirical application forecasting realized variances of S&P 100 constituents. In line with its proven gross-error insensitivity, the ranking of the continuous ranked probability score is the most consistent between evaluations based on the true outcome and the observations with measurement error.

Keywords: Forecast evaluation, measurement error, distribution forecasts, proper scoring rules

JEL Classification: C50, C52, C53

Suggested Citation

Kleen, Onno, Measurement Error Sensitivity of Loss Functions for Distribution Forecasts (February 25, 2021). Available at SSRN: https://ssrn.com/abstract=3476461 or http://dx.doi.org/10.2139/ssrn.3476461

Onno Kleen (Contact Author)

Erasmus University Rotterdam (EUR) - Erasmus School of Economics (ESE) ( email )

P.O. Box 1738
3000 DR Rotterdam, NL 3062 PA
Netherlands

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
66
Abstract Views
408
rank
398,778
PlumX Metrics