The Promise of Prediction Contests

33 Pages Posted: 30 Aug 2013

See all articles by Phillip E. Pfeifer

Phillip E. Pfeifer

University of Virginia - Darden School of Business

Yael Grushka-Cockayne

University of Virginia - Darden School of Business; Harvard University - Business School (HBS)

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business

Date Written: August 28, 2013

Abstract

This paper examines the prediction contest as a vehicle for aggregating the opinions of a crowd of experts. After proposing a general definition distinguishing prediction contests from other mechanisms for harnessing the wisdom of crowds, we examine two common types of prediction contests. For both point forecasting contests and pick-the-winners contests, we illustrate the incentive for forecasters to submit reports that exaggerate in the direction of their private information. In contrast to conventional wisdom, this non-truthful reporting usually improves the accuracy of the resulting crowd forecast. In the context of a simple theoretical model of overlapping information and forecaster behaviors, we present closed-form expressions for the mean squared error of the crowd forecasts which help identify the situations in which prediction contests will be most useful. The paper also reports on an empirical investigation into the accuracy of a pick-the-winners contest involving hundreds of experts. In that example, the resulting crowd forecasts were found to be more accurate than forecasts from a sophisticated statistical model.

Keywords: Forecast Aggregation, Forecasting Contests, Picking Contests, Public Knowledge Bias, Wisdom of Crowds

JEL Classification: D41, D82

Suggested Citation

Pfeifer, Phillip E. and Grushka-Cockayne, Yael and Lichtendahl, Kenneth C., The Promise of Prediction Contests (August 28, 2013). Darden Business School Working Paper No. 2317477. Available at SSRN: https://ssrn.com/abstract=2317477 or http://dx.doi.org/10.2139/ssrn.2317477

Phillip E. Pfeifer

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States
434-924-4803 (Phone)

HOME PAGE: http://www.darden.virginia.edu/faculty/Pfeifer.htm

Yael Grushka-Cockayne (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Harvard University - Business School (HBS) ( email )

Soldiers Field Road
Morgan 270C
Boston, MA 02163
United States

HOME PAGE: http://https://www.hbs.edu/faculty/Pages/profile.aspx?facId=263650

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
45
Abstract Views
316
PlumX Metrics