Focused Concept Miner (FCM): An Interpretable Deep Learning for Text Exploration

31 Pages Posted: 5 Jan 2019

See all articles by Dokyun Lee

Dokyun Lee

Carnegie Mellon University - David A. Tepper School of Business

Emaad Manzoor

Carnegie Mellon University, Students

Zhaoqi Cheng

Carnegie Mellon University - David A. Tepper School of Business

Date Written: December 20, 2018

Abstract

We introduce the Focused Concept Miner (FCM), an interpretable deep learning text mining algorithm to (1) automatically extract interpretable “concepts” from text data, (2) “focus” the mined concepts to explain any existing user-specified business outcomes, such as purchase conversion (linked to reviews read) or crowdfunding success (linked to project descriptions), and (3) quantify the correlational relative importance of each mined concept for business outcomes, along with their relative importance to other user-specified explanatory variables. Compared to existing methods that partially achieve FCM’s goals, FCM achieves higher interpretability and predictive performance.

Relative importance of discovered concepts provide managers easy ways to gauge potential impact and to inform hypotheses development. We present FCM as a complimentary technique to explore and understand unstructured textual data before applying standard causal inference techniques.

Applications can be found in any settings with text and structured data tied to a business outcome. We evaluate FCM’s performance on a comprehensive dataset that tracks individual-level review reading, searching, and conversion in addition to another crowdfunding data. Furthermore, we run a series of experiments to investigate the accuracy-interpretability trade-off to provide empirical observations for interpretable machine learning literature. Paper concludes with ideas for future development, potential application scenarios, and managerial implications.

Keywords: Interpretable Machine Learning, Automatic Concept Extraction, Managerial Exploratory Tool, Unstructured Data, Deep Learning, Text Mining, Coherence

JEL Classification: C38, C39

Suggested Citation

Lee, Dokyun and Manzoor, Emaad and Cheng, Zhaoqi, Focused Concept Miner (FCM): An Interpretable Deep Learning for Text Exploration (December 20, 2018). Available at SSRN: https://ssrn.com/abstract=3304756

Dokyun Lee (Contact Author)

Carnegie Mellon University - David A. Tepper School of Business ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213-3890
United States

Emaad Manzoor

Carnegie Mellon University, Students ( email )

Pittsburgh, PA
United States

Zhaoqi Cheng

Carnegie Mellon University - David A. Tepper School of Business ( email )

Pittsburgh, PA
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
583
Abstract Views
1,106
PlumX Metrics