False Dichotomy Alert: Improving Subjective-Probability Estimates vs. Raising Awareness of Systemic Risk

18 Pages Posted: 28 Jan 2022 Last revised: 18 Feb 2022

See all articles by Philip Tetlock

Philip Tetlock

University of Pennsylvania

Yunzi Lu

University of Pennsylvania

Barbara Mellers

University of Pennsylvania, Psychology; University of Pennsylvania, Wharton School

Date Written: January 20, 2022

Abstract

Taleb et al. (2022) portray the superforecasting research program as a masquerade that purports to build “survival functions for tail assessments via sports-like tournaments.’’ But that never was the goal. The program was designed to help intelligence analysts make better probability judgments, which required posing rapidly resolvable questions. From a signal detection theory perspective, the superforecasting and Taleb et al. programs are complementary, not contradictory (a point Taleb and Tetlock (2013) recognized). The superforecasting program aims at achieving high Hit rates at low cost in False-Positives whereas Taleb et al prioritize alerting us to systemic risk, even if that entails a high False-Positive rate. Proponents of each program should however acknowledge weaknesses in their cases. It is unclear: (a) how Taleb et al. (2022) can justify extreme error- avoidance trade-offs, without tacit probability judgments of rare, high-impact events; (b) how much superforecasting interventions can improve probability judgments of such events.

Keywords: superforecasting, systemic risk, fat-tailed dstributions, signal detection, forecasting tournaments, proper scoring rules

Suggested Citation

Tetlock, Philip and Lu, Yunzi and Mellers, Barbara, False Dichotomy Alert: Improving Subjective-Probability Estimates vs. Raising Awareness of Systemic Risk (January 20, 2022). Available at SSRN: https://ssrn.com/abstract=4013831 or http://dx.doi.org/10.2139/ssrn.4013831

Philip Tetlock (Contact Author)

University of Pennsylvania ( email )

Philadelphia, PA 19104
United States

Yunzi Lu

University of Pennsylvania ( email )

Philadelphia, PA 19104
United States

Barbara Mellers

University of Pennsylvania, Psychology ( email )

3815 Walnut Street
Philadelphia, PA 19104-6196
United States

University of Pennsylvania, Wharton School ( email )

3641 Locust Walk
Philadelphia, PA 19104-6365
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
593
Abstract Views
2,972
Rank
82,469
PlumX Metrics