Measurement Error Sensitivity of Loss Functions for Distribution Forecasts

56 Pages Posted: 8 Nov 2019

See all articles by Onno Kleen

Onno Kleen

Heidelberg University - Department of Economics

Date Written: November 15, 2019

Abstract

Economic variables are often reported on different scales or with measurement error, e.g. in macroeconomic and financial applications. We examine the sensitivity of scoring rules for distribution forecasts in two dimensions: linear rescaling of the data and the influence of noise on the forecast evaluation outcome. First, we show that all commonly used scoring rules for distribution forecasts are robust to rescaling the data. Second, it is revealed that the forecast ranking based on the continuous ranked probability score is less sensitive to measurement error than the log score. Our theoretical results are complemented by a simulation study based on forecasting quarterly GDP growth and an empirical application forecasting realized variances of 28 DJIA constituents. In line with its proven gross-error-insensitivity, the ranking of the continuous ranked probability score is the most consistent between evaluations based on the true outcome and the observations with measurement error.

Keywords: Forecast evaluation, measurement error, distribution forecasts, proper scoring rules

JEL Classification: C50, C52, C53

Suggested Citation

Kleen, Onno, Measurement Error Sensitivity of Loss Functions for Distribution Forecasts (November 15, 2019). Available at SSRN: https://ssrn.com/abstract=3476461 or http://dx.doi.org/10.2139/ssrn.3476461

Onno Kleen (Contact Author)

Heidelberg University - Department of Economics ( email )

Bergheimer Strasse 58
Heidelberg, BW 69115
Germany

HOME PAGE: http://onnokleen.de

Register to save articles to
your library

Register

Paper statistics

Downloads
14
Abstract Views
73
PlumX Metrics