Generating “Accurate” Online Reviews: Augmenting a Transformer-Based Approach with Structured Predictions

Abstract

A particular challenge with Generative Artificial Intelligence (GenAI) relates to the “hallucination” problem, wherein the generated content is factually incorrect. This is of particular concern for typical generative tasks in marketing. Here, we propose a two-step approach to address this issue. Our empirical context of an experience good (wines) where information about the taste of the product is important to the readers of the review but crucially, this data are unavailable a priori. Consequently, typical generative models may hallucinate this attribute in the generated review. Our approach of augmenting a transformer model with structured predictions results in a precision of .866 and a recall of .768 for the taste of wines, vastly outperforming popular benchmarks: transformer (precision .316, recall .250) and ChatGPT (precision .394, recall .243). We conduct an experimental study where respondents rated the similarity of reviews generated by our approach (versus those generated by ChatGPT) to those written by human wine experts. We find our reviews to be significantly more similar to human-expert reviews than those generated by ChatGPT. Apart from our app implementation, our main contribution in this work is to offer one approach towards more accurate GenAI, particularly towards marketing-related tasks.

Keywords: Artificial intelligence, Generative models, online reviews, machine learning, deep learning, neural networks