Comparing Face-to-Face Meetings, Nominal Groups, Delphi and Prediction Markets on an Estimation Task

Posted: 3 Mar 2010

See all articles by Andreas Graefe

Andreas Graefe

Macromedia University of Applied Sciences

J. Scott Armstrong

University of Pennsylvania - Marketing Department

Date Written: March 2, 2010

Abstract

We conducted laboratory experiments to analyze the accuracy of three structured approaches (nominal groups, Delphi, and prediction markets) compared to traditional face-to-face meetings (FTF). We recruited 227 participants (11 groups per method) that had to solve a quantitative judgment task that did not involve distributed knowledge. This task consisted of ten factual questions, which required percentage estimates. While, overall, we did not find statistically significant differences in accuracy between the four methods, the results differed somewhat at the individual question level. Delphi was as accurate as FTF for eight questions and outperformed FTF for two questions. By comparison, prediction markets were unable to outperform FTF for any of the ten questions but were inferior for three questions. The relative performance of nominal groups and FTF was mixed and differences were small. We also compared the results from the three structured approaches to prior individual estimates and staticized groups. The three structured approaches were more accurate than participants’ prior individual estimates. Delphi was also more accurate than staticized groups. Nominal groups and prediction markets provided little additional value compared to a simple average of forecast. In addition, we examined participants’ perceptions of the group and the group process.

Participants rated personal communication more favorable than computer-mediated interaction. Group interaction in FTF and nominal groups was perceived as highly cooperative and effective. Prediction markets were rated least favorable. Prediction market participants were least satisfied with the group process and perceived their method as most difficult.

Keywords: forecast accuracy, group decision-making, method comparison, satisfaction, combining

JEL Classification: C92, D7, C42

Suggested Citation

Graefe, Andreas and Armstrong, J. Scott, Comparing Face-to-Face Meetings, Nominal Groups, Delphi and Prediction Markets on an Estimation Task (March 2, 2010). International Journal of Forecasting, Forthcoming. Available at SSRN: https://ssrn.com/abstract=1562481

Andreas Graefe (Contact Author)

Macromedia University of Applied Sciences ( email )

Sandstrasse 9
Munich, Bavaria 80337
Germany

HOME PAGE: http://www.andreas-graefe.org

J. Scott Armstrong

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States
215-898-5087 (Phone)
215-898-2534 (Fax)

HOME PAGE: http://marketing.wharton.upenn.edu/people/faculty/armstrong.cfm

Register to save articles to
your library

Register

Paper statistics

Abstract Views
418
PlumX Metrics