Modeling and Interpreting Expert Disagreement About Artificial Superintelligence

Informatica, vol. 41, no. 7 (December 2017), pages 419-428

10 Pages Posted: 27 Jan 2018

See all articles by Seth Baum

Seth Baum

Global Catastrophic Risk Institute

Anthony Barrett

Global Catastrophic Risk Institute

Roman V. Yampolskiy

University of Louisville

Date Written: August 31, 2017

Abstract

Artificial superintelligence (ASI) is artificial intelligence (AI) with capabilities that are significantly greater than human capabilities across a wide range of domains. A hallmark of the ASI issue is disagreement among experts. This paper demonstrates and discusses methodological options for modeling and interpreting expert disagreement about the risk of ASI catastrophe. Using a new model called ASI-PATH, the paper models a well-documented recent disagreement between Nick Bostrom and Ben Goertzel, two distinguished ASI experts. Three points of disagreement are considered: (1) the potential for humans to evaluate the values held by an AI, (2) the potential for humans to create an AI with values that humans would consider desirable, and (3) the potential for an AI to create for itself values that humans would consider desirable. An initial quantitative analysis shows that accounting for variation in expert judgment can have a large effect on estimates of the risk of ASI catastrophe. The risk estimates can in turn inform ASI risk management strategies, which the paper demonstrates via an analysis of the strategy of AI confinement. The paper find the optimal strength of AI confinement to depend on the balance of risk parameters (1) and (2).

Keywords: artificial intelligence, artificial superintelligence, risk analysis, expert disagreement

Suggested Citation

Baum, Seth and Barrett, Anthony and Yampolskiy, Roman, Modeling and Interpreting Expert Disagreement About Artificial Superintelligence (August 31, 2017). Informatica, vol. 41, no. 7 (December 2017), pages 419-428. Available at SSRN: https://ssrn.com/abstract=3104645

Seth Baum (Contact Author)

Global Catastrophic Risk Institute ( email )

Anthony Barrett

Global Catastrophic Risk Institute ( email )

Roman Yampolskiy

University of Louisville ( email )

Louisville, KY 40292
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
121
rank
217,434
Abstract Views
478
PlumX