The Promise of Prediction Contests

33 Pages Posted: 30 Aug 2013

See all articles by Phillip E. Pfeifer

Phillip E. Pfeifer

University of Virginia - Darden School of Business

Yael Grushka-Cockayne

University of Virginia - Darden School of Business

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business

Date Written: August 28, 2013

Abstract

This paper examines the prediction contest as a vehicle for aggregating the opinions of a crowd of experts. After proposing a general definition distinguishing prediction contests from other mechanisms for harnessing the wisdom of crowds, we examine two common types of prediction contests. For both point forecasting contests and pick-the-winners contests, we illustrate the incentive for forecasters to submit reports that exaggerate in the direction of their private information. In contrast to conventional wisdom, this non-truthful reporting usually improves the accuracy of the resulting crowd forecast. In the context of a simple theoretical model of overlapping information and forecaster behaviors, we present closed-form expressions for the mean squared error of the crowd forecasts which help identify the situations in which prediction contests will be most useful. The paper also reports on an empirical investigation into the accuracy of a pick-the-winners contest involving hundreds of experts. In that example, the resulting crowd forecasts were found to be more accurate than forecasts from a sophisticated statistical model.

Keywords: Forecast Aggregation, Forecasting Contests, Picking Contests, Public Knowledge Bias, Wisdom of Crowds

JEL Classification: D41, D82

Suggested Citation

Pfeifer, Phillip E. and Grushka-Cockayne, Yael and Lichtendahl, Kenneth C., The Promise of Prediction Contests (August 28, 2013). Darden Business School Working Paper No. 2317477, Available at SSRN: https://ssrn.com/abstract=2317477 or http://dx.doi.org/10.2139/ssrn.2317477

Phillip E. Pfeifer

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States
434-924-4803 (Phone)

HOME PAGE: http://www.darden.virginia.edu/faculty/Pfeifer.htm

Yael Grushka-Cockayne (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
75
Abstract Views
565
Rank
572,227
PlumX Metrics