A Narrowing of AI Research?

58 Pages Posted: 13 Nov 2020

See all articles by Joel Klinger

Joel Klinger

National Endowment for Science, Technology and the Arts (NESTA)

Juan C Mateos-Garcia

National Endowment for Science, Technology and the Arts (NESTA)

Konstantinos Stathoulopoulos

National Endowment for Science, Technology and the Arts (NESTA)

Date Written: September 24, 2020

Abstract

Artificial Intelligence (AI) is being hailed as the latest example of a General Purpose Technology that could transform productivity and help tackle important societal challenges. This outcome is however not guaranteed: a myopic focus on short-term benefits could lock AI into technologies that turn out to be sub-optimal in the longer-run. For this reason, it may be valuable to preserve diversity in the AI trajectories that are explored until there is more information about their relative merits and dangers. Recent controversies about the dominance of deep learning methods and private labs in AI research suggest that the field may be getting narrower, but the evidence base is lacking. We seek to address this gap with an analysis of the thematic diversity of AI research in arXiv, a widely used pre-prints site. Having identified 110,000 AI papers in this corpus, we use hierarchical topic modelling to estimate the thematic composition of AI research, and this composition to calculate various metrics of research diversity. Our analysis suggests that diversity in AI research has stagnated in recent years, and that AI research involving private sector organisations tends to be less diverse than research in academia. This appears to be driven by a small number of prolific and narrowly-focused technology companies. Diversity in academia is bolstered by smaller institutions and research groups that may have less incentives to race and lower levels of collaboration with the private sector. We also find that private sector AI researchers tend to specialize in data and computationally intensive deep learning methods at the expense of research involving other (symbolic and statistical) AI methods, and of research that considers the societal and ethical implications of AI or applies it in domains like health. Our results suggest that there may be a rationale for policy action to prevent a premature narrowing of AI research that could reduce its societal benefits, but we note the incentive, information and scale hurdles standing in the way of such interventions.

Keywords: artificial intelligence, technological change, diversity

JEL Classification: O31, O33

Suggested Citation

Klinger, Joel and Mateos-Garcia, Juan C and Stathoulopoulos, Konstantinos, A Narrowing of AI Research? (September 24, 2020). Available at SSRN: https://ssrn.com/abstract=3698698 or http://dx.doi.org/10.2139/ssrn.3698698

Joel Klinger

National Endowment for Science, Technology and the Arts (NESTA) ( email )

United Kingdom

Juan C Mateos-Garcia (Contact Author)

National Endowment for Science, Technology and the Arts (NESTA) ( email )

United Kingdom

Konstantinos Stathoulopoulos

National Endowment for Science, Technology and the Arts (NESTA) ( email )

United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
150
Abstract Views
860
Rank
292,909
PlumX Metrics