Management Science, Early View, Forthcoming
37 Pages Posted: 15 Sep 2015 Last revised: 17 Jun 2016
Date Written: October 8, 2015
We report the results of the first large-scale, long-term, experimental test between two crowd sourcing methods – prediction markets and prediction polls. More than 2,400 participants made forecasts on 261 events over two seasons of a geopolitical prediction tournament. Some forecasters traded in a continuous double auction market and were ranked based on earnings. Others submitted probability judgments, independently or in teams, and were ranked based on Brier scores. In both seasons of the tournament, last day prices from the prediction market were more accurate than the simple mean of forecasts from prediction polls. However, team prediction polls outperformed prediction markets when poll forecasts were aggregated with algorithms using temporal decay, performance weighting and recalibration. The biggest advantage of prediction polls occurred at the start of long-duration questions. Prediction polls with proper scoring, algorithmic aggregation and teaming offer an attractive alternative to prediction markets for distilling the wisdom of crowds.
Keywords: Prediction, Forecasting, Prediction Markets, Crowdsourcing
JEL Classification: C82, C93, D84
Suggested Citation: Suggested Citation
Atanasov, Pavel D. and Rescober, Phillip and Stone, Eric and Swift, Samuel A. and Servan-Schreiber, Emile and Tetlock, Philip E. and Ungar, Lyle and Mellers, Barbara, Distilling the Wisdom of Crowds: Prediction Markets versus Prediction Polls (October 8, 2015). Management Science, Early View, Forthcoming. Available at SSRN: https://ssrn.com/abstract=2660628