Using LLMs for Market Research

46 Pages Posted: 30 Mar 2023 Last revised: 8 Jul 2023

See all articles by James Brand

James Brand

Microsoft

Ayelet Israeli

Harvard Business School - Marketing Unit

Donald Ngwe

Microsoft Corporation

Date Written: March 21, 2023

Abstract

Large language models (LLMs) have rapidly gained popularity as labor-augmenting tools for programming, writing, and many other processes that benefit from quick text generation. In this paper we explore the uses and benefits of LLMs for researchers and practitioners who aim to understand consumer preferences. We focus on the distributional nature of LLM responses, and query the Generative Pre-trained Transformer 3.5 Turbo (GPT-3.5 Turbo) model to generate dozens of responses to each survey question. We offer two sets of results to illustrate and assess our approach. First, we show that estimates of willingness-to-pay for products and features derived from GPT responses are realistic and comparable to estimates from human studies. Second, we demonstrate a practical method for market researchers to enhance GPT's responses by incorporating previous survey data from similar contexts via fine-tuning. This method improves the alignment of GPT's responses with human responses for existing and, importantly, new product features. We do not find similar improvements in the alignment for new product categories or for differences between customer segments.

Keywords: Large language models (LLMs), Generative Pre-trained Transformer (GPT), Market research, Consumer preferences, AI, Conjoint

Suggested Citation

Brand, James and Israeli, Ayelet and Ngwe, Donald, Using LLMs for Market Research (March 21, 2023). Harvard Business School Marketing Unit Working Paper No. 23-062, Available at SSRN: https://ssrn.com/abstract=4395751 or http://dx.doi.org/10.2139/ssrn.4395751

James Brand

Microsoft ( email )

Redomond, WA 98052

Ayelet Israeli (Contact Author)

Harvard Business School - Marketing Unit ( email )

Soldiers Field
Boston, MA 02163
United States

Donald Ngwe

Microsoft Corporation ( email )

One Microsoft Way
Redmond, WA 98052
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
13,103
Abstract Views
34,561
Rank
755
PlumX Metrics