Identifying Customer Needs from User-Generated Content

50 Pages Posted: 14 Jun 2017 Last revised: 16 Jul 2018

Date Written: July 1, 2018


Firms traditionally rely on interviews and focus groups to identify customer needs for marketing strategy and product development. User-generated content (UGC) is a promising alternative source for identifying customer needs. However, established methods are neither efficient nor effective for large UGC corpora because much content is non-informative or repetitive. We propose a machine-learning approach to facilitate qualitative analysis by selecting content for efficient review. We use a convolutional neural network to filter out non-informative content and cluster dense sentence embeddings to avoid sampling repetitive content. We further address two key questions: Are UGC-based customer needs comparable to interview-based customer needs? Do the machine-learning methods improve customer-need identification? These comparisons are enabled by a custom dataset of customer needs for oral care products identified by professional analysts using industry-standard experiential interviews. The analysts also coded 12,000 UGC sentences to identify which previously identified customer needs and/or new customer needs were articulated in each sentence. We show that (1) UGC is at least as valuable as a source of customer needs for product development, likely more-valuable, than conventional methods, and (2) machine-learning methods improve efficiency of identifying customer needs from UGC (unique customer needs per unit of professional services cost).

Keywords: Customer Needs, Online Reviews, Machine Learning, Voice of the Customer, User-generated Content, Market Research, Text Mining; Deep Learning, Natural Language Processing

Suggested Citation

Timoshenko, Artem and Hauser, John R., Identifying Customer Needs from User-Generated Content (July 1, 2018). Available at SSRN: or

Artem Timoshenko (Contact Author)

Kellogg School of Management, Northwestern University ( email )

2001 Sheridan Road
Evanston, IL 60208
United States

John R. Hauser

MIT Sloan School of Management ( email )

International Center for Research on the Mngmt Tech.
Cambridge, MA 02142
United States
617-253-2929 (Phone)
617-258-7597 (Fax)

Do you have negative results from your research you’d like to share?

Paper statistics

Abstract Views
PlumX Metrics