Measuring the Direction of Innovation: Frontier Tools in Unassisted Machine Learning

44 Pages Posted: 4 Jun 2020

See all articles by Florenta Teodoridis

Florenta Teodoridis

University of Southern California - Marshall School of Business

Jino Lu

University of Southern California - Marshall School of Business

Jeffrey L. Furman

Boston University - Department of Strategy & Policy; National Bureau of Economic Research (NBER)

Date Written: May 8, 2020

Abstract

Understanding the factors affecting the direction of innovation is a central aim of research in the economics and strategic management of innovation. Progress on this topic has been inhibited by difficulties in measuring the location and movement of innovation in ideas space. To date, most efforts at measuring the direction of innovation rely on curated taxonomies, such as technology classes and keyword approaches, which either adapt slowly or are subject to gaming, and early generations of text analysis, which provide information on the similarity of sets of words, but not on the number of paths or direction of change. Relative to these, recent advances in machine learning offer promising paths forward. In this paper, we introduce and explore a particular approach based on an unassisted machine learning technique, Hierarchical Dirichlet Process (HDP), that flexibly generates categories from a corpus of text and enables calculations of the distance across knowledge categories and movement in ideas space. We apply our algorithm to the corpus of USPTO patent abstracts from the period 2000-2018 and demonstrate that, relative to the USPTO taxonomy of patent classes, our algorithm provides a leading indicator of shift in innovation topics and enables a more precise analysis of movement in ideas space. Working with such measures is important because it enables more accurate estimates of the direction of innovation and, hence, of economic actors’ responses to competitive environments and public policies. We share our algorithm, which can be applied to other innovation text corpora, as well as the patent data and measures we develop, with the aim of facilitating additional inquiries regarding the direction of innovation.

Keywords: knowledge production, innovation, research technology, rate and direction of innovation, technological change, topic modeling, machine learning, ideas space, research trajectories, knowledge trajectories, diversification, breadth and depth of knowledge

JEL Classification: O30, O31, O33, O39

Suggested Citation

Teodoridis, Florenta and Lu, Jino and Furman, Jeffrey L., Measuring the Direction of Innovation: Frontier Tools in Unassisted Machine Learning (May 8, 2020). Available at SSRN: https://ssrn.com/abstract=3596233 or http://dx.doi.org/10.2139/ssrn.3596233

Florenta Teodoridis (Contact Author)

University of Southern California - Marshall School of Business ( email )

701 Exposition Blvd
Los Angeles, CA 90089
United States

Jino Lu

University of Southern California - Marshall School of Business ( email )

701 Exposition Blvd, HOH 431
Los Angeles, CA California 90089-1424
United States

Jeffrey L. Furman

Boston University - Department of Strategy & Policy ( email )

595 Commonwealth Avenue
Boston, MA 02215
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
100
Abstract Views
361
rank
288,409
PlumX Metrics