Bayesian Ensembles of Binary-Event Forecasts: When Is It Appropriate to Extremize or Anti-Extremize?

32 Pages Posted: 27 Mar 2017 Last revised: 1 Apr 2018

See all articles by Kenneth C. Lichtendahl

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business

Yael Grushka-Cockayne

University of Virginia - Darden School of Business; Harvard Business School

Victor Richmond Jose

Georgetown University - McDonough School of Business

Robert L. Winkler

Duke University - Fuqua School of Business

Date Written: March 27, 2018

Abstract

Many organizations face critical decisions that rely on forecasts of binary events. In these situations, organizations often gather forecasts from multiple experts or models and average those forecasts to produce a single aggregate forecast. Because the average forecast is known to be underconfident, methods have been proposed that create an aggregate forecast more extreme than the average forecast. But is it always appropriate to extremize the average forecast? And if not, when is it appropriate to anti-extremize (i.e., to make the aggregate forecast less extreme)? To answer these questions, we introduce a class of optimal aggregators. These aggregators are Bayesian ensembles because they follow from a Bayesian model of the underlying information experts have. Each ensemble is a generalized additive model of experts' probabilities that first transforms the experts' probabilities into their corresponding information states, then linearly combines these information states, and finally transforms the combined information states back into the probability space. Analytically, we find that these optimal aggregators do not always extremize the average forecast, and when they do, they can run counter to existing methods. On two publicly available datasets, we demonstrate that these new ensembles are easily fit to real forecast data and are more accurate than existing methods.

Keywords: Forecast aggregation; linear opinion pool; generalized additive model; generalized linear model; stacking.

Suggested Citation

Lichtendahl, Kenneth C. and Grushka-Cockayne, Yael and Jose, Victor Richmond and Winkler, Robert L., Bayesian Ensembles of Binary-Event Forecasts: When Is It Appropriate to Extremize or Anti-Extremize? (March 27, 2018). Available at SSRN: https://ssrn.com/abstract=2940740 or http://dx.doi.org/10.2139/ssrn.2940740

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Yael Grushka-Cockayne (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Harvard Business School ( email )

Soldiers Field Road
Morgan 270C
Boston, MA 02163
United States

HOME PAGE: http://https://www.hbs.edu/faculty/Pages/profile.aspx?facId=263650

Victor Richmond Jose

Georgetown University - McDonough School of Business ( email )

544 Hariri Bldg
37th and O Sts NW
Washington, DC 20057
United States

Robert L. Winkler

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
58
Abstract Views
525
rank
360,658
PlumX Metrics