Generating “Accurate” Online Reviews: Augmenting a Transformer-Based Approach with Structured Predictions

50 Pages Posted: 13 Jan 2024

See all articles by Prasad Vana

Prasad Vana

Tuck School of Business at Dartmouth

Praveen K. Kopalle

Dartmouth College - Tuck School of Business

Pradeep Pachigolla

Cornell University - Samuel Curtis Johnson Graduate School of Management

Keith Carlson

Dartmouth College

Date Written: January 5, 2024

Abstract

A particular challenge with Generative Artificial Intelligence (GenAI) relates to the “hallucination” problem, wherein the generated content is factually incorrect. This is of particular concern for typical generative tasks in marketing. Here, we propose a two-step approach to address this issue. Our empirical context of an experience good (wines) where information about the taste of the product is important to the readers of the review but crucially, this data are unavailable a priori. Consequently, typical generative models may hallucinate this attribute in the generated review. Our approach of augmenting a transformer model with structured predictions results in a precision of .866 and a recall of .768 for the taste of wines, vastly outperforming popular benchmarks: transformer (precision .316, recall .250) and ChatGPT (precision .394, recall .243). We conduct an experimental study where respondents rated the similarity of reviews generated by our approach (versus those generated by ChatGPT) to those written by human wine experts. We find our reviews to be significantly more similar to human-expert reviews than those generated by ChatGPT. Apart from our app implementation, our main contribution in this work is to offer one approach towards more accurate GenAI, particularly towards marketing-related tasks.

Keywords: Artificial intelligence, Generative models, online reviews, machine learning, deep learning, neural networks

Suggested Citation

Vana, Prasad and Kopalle, Praveen K. and Pachigolla, Pradeep and Carlson, Keith, Generating “Accurate” Online Reviews: Augmenting a Transformer-Based Approach with Structured Predictions (January 5, 2024). Available at SSRN: https://ssrn.com/abstract=4692101 or http://dx.doi.org/10.2139/ssrn.4692101

Prasad Vana (Contact Author)

Tuck School of Business at Dartmouth ( email )

Hanover, NH 03755
United States

HOME PAGE: http://www.prasadvana.com

Praveen K. Kopalle

Dartmouth College - Tuck School of Business ( email )

100 Tuck Hall
Hanover, NH 03755
United States
603-646-3612 (Phone)
603-646-1308 (Fax)

Pradeep Pachigolla

Cornell University - Samuel Curtis Johnson Graduate School of Management ( email )

Ithaca, NY 14853
United States

Keith Carlson

Dartmouth College ( email )

Department of Sociology
Hanover, NH 03755
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
75
Abstract Views
253
Rank
577,266
PlumX Metrics