Beware of Botshit: How to Manage the Epistemic Risks of Generative Chatbots

28 Pages Posted: 18 Jan 2024

See all articles by Tim Hannigan

Tim Hannigan

University of Alberta - School of Business

Ian P. McCarthy

Simon Fraser University (SFU) - Beedie School of Business

Andre Spicer

City University London - The Business School

Date Written: December 28, 2023

Abstract

Advances in large language model (LLM) technology enable chatbots to generate and analyze content for our work. Generative chatbots do this work by ‘predicting’ responses rather than ‘knowing’ the meaning of their responses. This means chatbots can produce coherent sounding but inaccurate or fabricated content, referred to as ‘hallucinations’. When humans use this untruthful content for tasks, it becomes what we call ‘botshit’. This article focuses on how to use chatbots for content generation work while mitigating the epistemic (i.e., the process of producing knowledge) risks associated with botshit. Drawing on risk management research, we introduce a typology framework that orients how chatbots can be used based on two dimensions: response veracity verifiability, and response veracity importance. The framework identifies four modes of chatbot work (authenticated, autonomous, automated, and augmented) with a botshit related risk (ignorance, miscalibration, routinization, and black boxing). We describe and illustrate each mode and offer advice to help chatbot users guard against the botshit risks that come with each mode.

Keywords: chatbots, bullshit, botshit, hallucinations, large language models, artificial intelligence, epistemic risks

JEL Classification: M00

Suggested Citation

Hannigan, Timothy and McCarthy, Ian P. and Spicer, Andre, Beware of Botshit: How to Manage the Epistemic Risks of Generative Chatbots (December 28, 2023). Business Horizons, Forthcoming, Available at SSRN: https://ssrn.com/abstract=4678265 or http://dx.doi.org/10.2139/ssrn.4678265

Timothy Hannigan

University of Alberta - School of Business ( email )

2-43 Business Building
Edmonton, Alberta T6G 2C7
Canada

Ian P. McCarthy (Contact Author)

Simon Fraser University (SFU) - Beedie School of Business ( email )

8888 University Drive
Burnaby, British Columbia V5A 1S6
Canada

Andre Spicer

City University London - The Business School ( email )

106 Bunhill Row
London, EC1Y 8TZ
United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
3,382
Abstract Views
11,201
Rank
6,702
PlumX Metrics