Human Favoritism, Not AI Aversion: People’s Perceptions (and Bias) Toward Generative AI, Human Experts, and Human-GAI Collaboration in Persuasive Content Generation

Abstract

With the wide availability of Large Language Models and generative AI, there are four primary paradigms for Human-AI collaboration: human only, AI only (ChatGPT-4), augmented human (where a human making the final decision with AI output as a reference), or augmented AI (where the AI making the final decision with human output as a reference). In partnership with one of the world’s leading consulting firms, we enlist professional content creators and ChatGPT-4 to create advertising content for products and persuasive content for campaigns following the aforementioned paradigms. First, we find that, contrary to the expectations of existing algorithm aversion literature on conventional predictive AI, content generated by generative AI and augmented AI is perceived as of higher quality than that produced by human experts and augmented human experts. Second, revealing the source of content production reduces – but does not reverse – the perceived quality gap between human- and AI-generated content. This bias in evaluation is predominantly driven by human favoritism rather than AI aversion: knowing the same content is created by a human expert increases its (reported) perceived quality, but knowing that AI is involved in the creation process does not affect its perceived quality. Further analysis suggests this bias is not due to a “quality prime” as knowing the content they are about to evaluate comes from competent creators (e.g., industry professionals and state-of-the-art AI) without knowing exactly the creator of each piece of content does not increase participants’ perceived quality.

Keywords: Large Language Models, Artificial Intelligence, Consumer Perception, Consumer Bias