Regularized Aggregation of One-off Probability Predictions

69 Pages Posted: 11 Mar 2021 Last revised: 10 Aug 2021

See all articles by Ville Satopää

Ville Satopää

INSEAD - Technology and Operations Management

Date Written: January 20, 2021

Abstract

Forecasters predicting the chances of a future event may disagree due to differing evidence or noise. To harness the collective evidence of the crowd, we propose a Bayesian aggregator that is regularized by analyzing the forecasters’ disagreement and ascribing over-dispersion to noise. Our aggregator requires no user intervention and can be computed efficiently even for a large numbers of predictions. To illustrate, we evaluate our aggregator on subjective probability predictions collected during a four-year forecasting tournament sponsored by the US intelligence community. Our aggregator improves the squared error (a.k.a, the Brier score) of simple averaging by around 20% and other commonly used aggregators by 10 − 25%. This advantage stems almost exclusively from improved calibration. An R package called braggR implements our method and is available on CRAN.

Keywords: Judgmental forecasting, Information aggregation, Objective Bayes, Overdispersion, Wisdom of the Crowds

Suggested Citation

Satopää, Ville, Regularized Aggregation of One-off Probability Predictions (January 20, 2021). Available at SSRN: https://ssrn.com/abstract=3769945 or http://dx.doi.org/10.2139/ssrn.3769945

Ville Satopää (Contact Author)

INSEAD - Technology and Operations Management ( email )

Boulevard de Constance
77 305 Fontainebleau Cedex
France

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
169
Abstract Views
784
rank
248,289
PlumX Metrics