Abstract

http://ssrn.com/abstract=2175093
 


 



Trimmed Opinion Pools and the Crowd's Calibration Problem


Victor Richmond R. Jose


Georgetown University - McDonough School of Business

Yael Grushka-Cockayne


University of Virginia - Darden School of Business

Kenneth C. Lichtendahl Jr.


University of Virginia - Darden School of Business

November 13, 2012

Management Science, Forthcoming
Darden Business School Working Paper No. 2175093

Abstract:     
We introduce an alternative to the popular linear opinion pool for combining individual probability forecasts. One of the well-known problems with the linear opinion pool is that it can be poorly calibrated. It tends toward underconfidence as the crowd’s diversity increases, i.e., as the variance in the individuals’ means increases. To address this calibration problem, we propose the exterior-trimmed opinion pool. To form this pool, forecasts with low and high means, or cumulative distribution function (cdf) values, are trimmed away from a linear opinion pool. Exterior-trimming decreases the pool’s variance and improves its calibration. A linear opinion pool, however, will remain overconfident when individuals are overconfident and not very diverse. For these situations, we suggest trimming away forecasts with moderate means, or cdf values. This interior-trimming increases variance and reduces overconfidence. Using probability forecast data from US and European Surveys of Professional Forecasters, we present empirical evidence that trimmed opinion pools can outperform the linear opinion pool.

Number of Pages in PDF File: 22

Keywords: trimming, probability forecasts, expert combination, linear opinion pool, underconfidence, overconfidence, scoring rules, wisdom of crowds, diversity

JEL Classification: C10, C53, E17

Accepted Paper Series


Download This Paper

Date posted: November 15, 2012 ; Last revised: July 13, 2013

Suggested Citation

Jose, Victor Richmond R. and Grushka-Cockayne, Yael and Lichtendahl, Kenneth C., Trimmed Opinion Pools and the Crowd's Calibration Problem (November 13, 2012). Management Science, Forthcoming; Darden Business School Working Paper No. 2175093. Available at SSRN: http://ssrn.com/abstract=2175093 or http://dx.doi.org/10.2139/ssrn.2175093

Contact Information

Victor Richmond R. Jose
Georgetown University - McDonough School of Business ( email )
544 Hariri Bldg
37th and O Sts NW
Washington, DC 20057
United States
Yael Grushka-Cockayne (Contact Author)
University of Virginia (UVA) - Darden School of Business ( email )
P.O. Box 6550
Charlottesville, VA 22906-6550
United States
Kenneth C. Lichtendahl Jr.
University of Virginia - Darden School of Business ( email )
P.O. Box 6550
Charlottesville, VA 22906-6550
United States
Feedback to SSRN


Paper statistics
Abstract Views: 556
Downloads: 95
Download Rank: 164,001

© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.  FAQ   Terms of Use   Privacy Policy   Copyright   Contact Us
This page was processed by apollo3 in 0.469 seconds