Combining Prediction Intervals in the M4 Competition

16 Pages Posted: 14 Feb 2019 Last revised: 10 Apr 2019

See all articles by Yael Grushka-Cockayne

Yael Grushka-Cockayne

University of Virginia - Darden School of Business; Harvard University - Business School (HBS)

Victor Richmond Jose

Georgetown University - McDonough School of Business

Date Written: February 5, 2019

Abstract

The 2018 M4 Forecasting Competition was the first M-Competition to elicit prediction intervals in addition to point estimates. We take a closer look at the twenty valid interval submissions by examining the prediction intervals' calibration and accuracy and evaluating their performance over different time horizons. Overall, the submissions fail to estimate the uncertainty properly. Importantly, we investigate the benefits of interval combination using six recently proposed heuristics that can be applied prior to learning about the quantities' realizations. Our results suggest that interval aggregation offers improvements both in terms of calibration and in terms of accuracy. While averaging interval endpoints maintains its practical appeal as simple to implement and performs quite well when data sets are large, the median and the interior trimmed average are found to be robust aggregators for the prediction interval submissions across all 100,000 time series.

Keywords: hit rates, interval combination methods, calibration, mean scaled interval score, interior trimming, overconfidence

Suggested Citation

Grushka-Cockayne, Yael and Jose, Victor Richmond, Combining Prediction Intervals in the M4 Competition (February 5, 2019). Available at SSRN: https://ssrn.com/abstract=3329413 or http://dx.doi.org/10.2139/ssrn.3329413

Yael Grushka-Cockayne (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Harvard University - Business School (HBS) ( email )

Soldiers Field Road
Morgan 270C
Boston, MA 02163
United States

HOME PAGE: http://https://www.hbs.edu/faculty/Pages/profile.aspx?facId=263650

Victor Richmond Jose

Georgetown University - McDonough School of Business ( email )

544 Hariri Bldg
37th and O Sts NW
Washington, DC 20057
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
78
Abstract Views
585
rank
316,605
PlumX Metrics