Abstract

https://ssrn.com/abstract=2411649
 


 



Probabilistic Coherence Weighting for Optimizing Expert Forecasts


Christopher W. Karvetski


George Mason University

Kenneth C. Olson


George Mason University

David R. Mandel


York University - Department of Psychology

Charles R. Twardy


George Mason University

July 26, 2013

Decision Analysis, 10(4), 305-326

Abstract:     
Methods for eliciting and aggregating expert judgment are necessary when decision-relevant data are scarce. Such methods have been used for aggregating the judgments of a large, heterogeneous group of forecasters, as well as the multiple judgments produced from an individual forecaster. This paper addresses how multiple related individual forecasts can be used to improve aggregation of probabilities for a binary event across a set of forecasters. We extend previous efforts that use probabilistic incoherence of an individual forecaster’s subjective probability judgments to weight and aggregate the judgments of multiple forecasters for the goal of increasing the accuracy of forecasts. With data from two studies, we describe an approach for eliciting extra probability judgments to (i) adjust the judgments of each individual forecaster, and (ii) assign weights to the judgments to aggregate over the entire set of forecasters. We show improvement of up to 30% over the established benchmark of a simple equal-weighted averaging of forecasts. We also describe how this method can be used to remedy the “fifty–fifty blip” that occurs when forecasters use the probability value of 0.5 to represent epistemic uncertainty.

Number of Pages in PDF File: 23

Keywords: probabilistic coherence, forecast aggregation, crowdsourcing, linear opinion pool, fifty–fifty blip, practice


Open PDF in Browser Download This Paper

Date posted: March 21, 2014  

Suggested Citation

Karvetski, Christopher W. and Olson, Kenneth C. and Mandel, David R. and Twardy, Charles R., Probabilistic Coherence Weighting for Optimizing Expert Forecasts (July 26, 2013). Decision Analysis, 10(4), 305-326. Available at SSRN: https://ssrn.com/abstract=2411649

Contact Information

Christopher W. Karvetski
George Mason University ( email )
4400 University Drive
Fairfax, VA 22030
United States
Kenneth C. Olson
George Mason University ( email )
4400 University Drive
Fairfax, VA 22030
United States
David R. Mandel (Contact Author)
York University - Department of Psychology ( email )
4700 Keele St.
Toronto, Ontario M3J 1P3
Canada
HOME PAGE: http://https://sites.google.com/site/themandelian/
Charles R. Twardy
George Mason University ( email )
4400 University Drive
Fairfax, VA 22030
United States
Feedback to SSRN


Paper statistics
Abstract Views: 163
Downloads: 26