What People Think of Machines as Doctors: Unveiling the Value of Gen-AI for e-Health

39 Pages Posted: 21 Mar 2024 Last revised: 14 Jan 2025

See all articles by Dicle Yagmur Ozdemir

Dicle Yagmur Ozdemir

Erasmus University Rotterdam (EUR) - Rotterdam School of Management (RSM)

Mehmet Ayvaci

Jindal School of Management - The University of Texas at Dallas

Alejandro Zentner

The University of Texas at Dallas - Naveen Jindal School of Management

Date Written: January 02, 2025

Abstract

Large language models (LLMs) generate human-like text from vast data, enabling natural language communication even for tasks demanding expert knowledge. As LLMs increasingly become an alternative for experts, understanding how non-experts perceive and respond to automated responses by machines (in this case, LLMs) is crucial. Framed within the context of patient-physician communication, we investigate how non-experts (typical patients) perceive LLM responses versus physician responses and explore the factors influencing their perception. In a survey-based experiment, we compare non-experts' (survey participants) evaluations of responses from physicians and ChatGPT, a Chat Generative Pretrained Transformer, to patient queries. Our findings reveal that non-experts overwhelmingly prefer ChatGPT responses over responses by physicians, even when machine responses are of low quality (as judged by a blinded panel of experts). Two key factors influencing this preference emerge from our study: longer prose from ChatGPT heightens non-experts' preference for machines, while disclosing the response source diminishes this preference, especially when the ChatGPT response quality is lower. Our study indicates the need for a careful use of LLMs when responding to laypersons, particularly patients, in their search for answers to health-related questions.

Keywords: LLMs, patient perception, healthcare quality, generative artificial intelligence

Suggested Citation

Ozdemir, Dicle Yagmur and Ayvaci, Mehmet and Zentner, Alejandro, What People Think of Machines as Doctors: Unveiling the Value of Gen-AI for e-Health (January 02, 2025). Available at SSRN: https://ssrn.com/abstract=4765222 or http://dx.doi.org/10.2139/ssrn.4765222

Dicle Yagmur Ozdemir (Contact Author)

Erasmus University Rotterdam (EUR) - Rotterdam School of Management (RSM) ( email )

P.O. Box 1738
Room T08-21
3000 DR Rotterdam, 3000 DR
Netherlands

Mehmet Ayvaci

Jindal School of Management - The University of Texas at Dallas ( email )

P.O. Box 830688
Richardson, TX 75083-0688
United States

Alejandro Zentner

The University of Texas at Dallas - Naveen Jindal School of Management ( email )

P.O. Box 830688
Richardson, TX 75083-0688
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
246
Abstract Views
837
Rank
256,314
PlumX Metrics