Boosting the Wisdom of Crowds Within a Single Judgment Problem: Selective Averaging Based on Peer Predictions

47 Pages Posted: 1 Jan 2020 Last revised: 19 Nov 2020

See all articles by Asa Palley

Asa Palley

Indiana University - Kelley School of Business - Department of Operation & Decision Technologies

Ville Satopää

INSEAD - Technology and Operations Management

Date Written: November 18, 2020

Abstract

Combining point estimates from multiple judges often provides a more accurate aggregate estimate than using a point estimate from a single judge, a phenomenon called "the wisdom of crowds." However, if the judges use shared information when forming their estimates, the simple average will end up over-emphasizing this common component at the expense of the judges’ private information. A decision maker could in theory obtain a more accurate estimate by appropriately combining all information in the judges’ opinions. Although this information is embedded within the judges’ individual estimates, it is typically unobservable and thus cannot be directly aggregated by a decision maker. In this article, we propose a weighting of judges’ individual estimates that appropriately combines their collective information within a single estimation problem. Judges are asked to provide both a point estimate of the quantity of interest and a prediction of the average estimate that will be given by all other judges. Predictions of others are then used as part of a criterion to determine weights that are applied to each judge’s estimate to form an aggregate estimate. Our weighting procedure is robust to noise in the judges’ responses and can be expressed in closed form. We use both simulation and data from six experimental studies to illustrate that our procedure outperforms existing averaging-like methods.

Keywords: Forecasting, Estimation, Judgment Aggregation, Wisdom of Crowds, Shared Information

JEL Classification: D83, C53, C91

Suggested Citation

Palley, Asa and Satopää, Ville, Boosting the Wisdom of Crowds Within a Single Judgment Problem: Selective Averaging Based on Peer Predictions (November 18, 2020). Available at SSRN: https://ssrn.com/abstract=3504286 or http://dx.doi.org/10.2139/ssrn.3504286

Asa Palley (Contact Author)

Indiana University - Kelley School of Business - Department of Operation & Decision Technologies ( email )

Hodge Hall 4100
1275 E 10th St.
Bloomington, IN 47405
United States

Ville Satopää

INSEAD - Technology and Operations Management ( email )

Boulevard de Constance
77 305 Fontainebleau Cedex
France

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
99
Abstract Views
584
rank
304,622
PlumX Metrics