Review of: Philip E. Tetlock. 2005. Expert Political Judgment: How Good is it? How Can We Know?

4 Pages Posted: 1 Jul 2008

See all articles by Adrian E. Tschoegl

Adrian E. Tschoegl

University of Pennsylvania - Management Department

J. Scott Armstrong

University of Pennsylvania - Marketing Department

Date Written: June 29, 2008

Abstract

Typically, researchers report new findings in scholarly journals and Tetlock (1998, 1999) has done so for of some part of the findings of his study. Still, Tetlock has gone beyond journal articles, turning to a book to report on his large-scale and important study. Publishing a book has allowed him to deal with issues in detail and provide a full report, to reach a larger audience, and to have a stronger impact one hopes on experts and the consumers of their services, including political leaders and the voting public.

The book assaults common sense with evidence. In order to mount his assault on accepted wisdom, Tetlock spends more some 238 pages of text explaining his methods and findings, and considering and refuting many alternative explanations, and adds some 75 pages of technical appendices. The downside of this approach is that some readers may find the book too demanding. That would be a pity as his findings are important.

Tetlock's book reports the results of a two-decade long study of expert predictions. He recruited 284 people whose professions included "commenting or offering advice on political and economic trends." He asked them to forecast the probability that various situations would or would not occur, picking areas (geographic and substantive) within and outside their areas of expertise. In addition to eliciting forecasts, Tetlock also asked questions aimed at understanding how the forecasters came to formulate their forecasts, how they dealt with the failure of their forecasts, how they responded to contradictory information, and how they evaluated the probable accuracy of others' theories and predictions. By 2003, he had accumulated 82,361 forecasts, which provided him a database to evaluate. He then evaluated experts' predictions against outcomes, and against various alternate predictions that he derived from simple statistical procedures, from uninformed non-experts, and from well-informed non-experts.

Suggested Citation

Tschoegl, Adrian E. and Armstrong, J. Scott, Review of: Philip E. Tetlock. 2005. Expert Political Judgment: How Good is it? How Can We Know? (June 29, 2008). Available at SSRN: https://ssrn.com/abstract=1153126 or http://dx.doi.org/10.2139/ssrn.1153126

Adrian E. Tschoegl

University of Pennsylvania - Management Department ( email )

The Wharton School
Philadelphia, PA 19104-6370
United States

J. Scott Armstrong (Contact Author)

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States
215-898-5087 (Phone)
215-898-2534 (Fax)

HOME PAGE: http://marketing.wharton.upenn.edu/people/faculty/armstrong.cfm

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
116
Abstract Views
1,041
PlumX Metrics