Extremizing and Anti-Extremizing in Bayesian Ensembles of Binary-Event Forecasts

29 Pages Posted: 27 Mar 2017 Last revised: 24 Apr 2021

See all articles by Kenneth C. Lichtendahl

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business

Yael Grushka-Cockayne

University of Virginia - Darden School of Business; Harvard University - Business School (HBS)

Victor Richmond Jose

Georgetown University - McDonough School of Business

Robert L. Winkler

Duke University - Fuqua School of Business

Date Written: August 25, 2020

Abstract

Probability forecasts of binary events are often gathered from multiple models or experts and averaged to provide inputs regarding uncertainty in important decision-making problems. Averages of well-calibrated probabilities are underconfident and methods have been proposed to make them more extreme. To aggregate probabilities, we introduce a class of ensembles that are generalized additive models. These ensembles are based on Bayesian principles and can help us learn why and when extremizing is appropriate. Extremizing is typically viewed as shifting the average probability farther from one-half; we emphasize that it is more suitable to define extremizing as shifting it farther from the base rate. We introduce the notion of anti-extremizing to learn instances where it might be beneficial to make average probabilities less extreme.

Analytically, we find that our Bayesian ensembles often extremize the average forecast, but sometimes anti-extremize instead. On several publicly available datasets, we demonstrate that our Bayesian ensemble performs well and anti-extremizes anywhere from 18% to 73% of the cases. It anti-extremizes much more often when there is bracketing with respect to the base rate among the probabilities being aggregated than with no bracketing, suggesting that bracketing is a promising indicator of when we should consider anti-extremizing

Keywords: Forecast aggregation; linear opinion pool; generalized linear model; extremizing and anti-extremizing; bracketing; probit ensemble

Suggested Citation

Lichtendahl, Kenneth C. and Grushka-Cockayne, Yael and Jose, Victor Richmond and Winkler, Robert L., Extremizing and Anti-Extremizing in Bayesian Ensembles of Binary-Event Forecasts (August 25, 2020). Available at SSRN: https://ssrn.com/abstract=2940740 or http://dx.doi.org/10.2139/ssrn.2940740

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Yael Grushka-Cockayne (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Harvard University - Business School (HBS) ( email )

Soldiers Field Road
Morgan 270C
Boston, MA 02163
United States

HOME PAGE: http://https://www.hbs.edu/faculty/Pages/profile.aspx?facId=263650

Victor Richmond Jose

Georgetown University - McDonough School of Business ( email )

544 Hariri Bldg
37th and O Sts NW
Washington, DC 20057
United States

Robert L. Winkler

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
103
Abstract Views
1,261
rank
315,597
PlumX Metrics