GPT in the Loop: Evidence from the Field

Abstract

Generative Pre-trained Transformers (GPTs) are highly effective in generating content and increasing productivity, but companies have reservations about their use in a professional setting. OpenAI and policymakers suggest that disclosing the use of GPT is necessary, but there is little empirical evidence to understand its consequences. Our experiment found that managers from a leading consulting firm were unable to distinguish Human-GPT generated content when the content generation source was not disclosed and disclosing the use of GPT improved the content’s evaluation. We explored the effects of applying the GPT disclosure policy in the workplace. Managers prefer analysts to disclose their use of GPT, but their preferences regarding how junior analysts should use GPT may differ from those of the analysts, leading to potential conflicts over disclosure.

Keywords: Human-GPT collaboration, GPT disclosure, content evaluation, survey experiment