Extending the Wisdom of Crowds: Quantifying Uncertainty Using the Mean and Variance of a Collection of Point Estimates

19 Pages Posted: 7 Aug 2017 Last revised: 18 Jan 2018

See all articles by Asa Palley

Asa Palley

Indiana University - Kelley School of Business - Department of Operation & Decision Technologies

Date Written: August 5, 2017

Abstract

The wisdom of crowds—combining information from a collection of individual judgments—offers a useful technique to quantify an unknown variable. Averaging point estimates has proven to be effective in reducing error in the consensus estimate. However, in many managerial problems, the decision maker requires an assessment of the full distribution of uncertainty rather than just a single number. In practice, some managers have used dispersion in point estimates as a cue to uncertainty in the variable of interest, but a characterization of the exact relationship between the two has not yet been established. Using a stylized Bayesian model of overlapping information spread across a collection of judges, I show that the variance of the variable of interest can be estimated with a multiple of the variance of the individual judgments, and establish an analytical expression for the consensus predictive distribution. I then present a procedure that can be used to learn about this variance inflation factor using past judgments and realizations, and derive the resulting distribution for the new variable of interest. This aggregation method is easy to implement in practice to estimate a predictive distribution from a collection of individual judgments. Application of the procedure to forecasts available through the Survey of Professional Forecasters suggests that the resulting predictions are well-calibrated and outperform natural existing alternatives.

Keywords: Judgment Aggregation, Wisdom of Crowds, Forecasting, Combining Information, Estimation

JEL Classification: D83, C53, C91

Suggested Citation

Palley, Asa, Extending the Wisdom of Crowds: Quantifying Uncertainty Using the Mean and Variance of a Collection of Point Estimates (August 5, 2017). Available at SSRN: https://ssrn.com/abstract=3014058 or http://dx.doi.org/10.2139/ssrn.3014058

Asa Palley (Contact Author)

Indiana University - Kelley School of Business - Department of Operation & Decision Technologies ( email )

Hodge Hall 4100
1275 E 10th St.
Bloomington, IN 47405
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
100
Abstract Views
551
rank
288,622
PlumX Metrics