A Turing Test: Are Ai Chatbots Behaviorally Similar to Humans?

42 Pages Posted: 30 Nov 2023 Last revised: 3 Jan 2024

See all articles by Qiaozhu Mei

Qiaozhu Mei

University of Michigan at Ann Arbor

Yutong Xie

University of Michigan at Ann Arbor

Walter Yuan

MobLab

Matthew O. Jackson

Stanford University - Department of Economics; Santa Fe Institute

Date Written: November 19, 2023

Abstract

We administer a Turing Test to AI Chatbots. We examine how Chatbots behave in a suite of classic behavioral games that are designed to elicit characteristics such as trust, fairness, risk-aversion, cooperation, \textit{etc.}, as well as how they respond to a traditional Big-5 psychological survey that measures personality traits. ChatGPT-4 exhibits behavioral and personality traits that are statistically indistinguishable from a random human from tens of thousands of human subjects from more than 50 countries. Chatbots also modify their behavior based on previous experience and contexts ``as if'' they were learning from the interactions, and change their behavior in response to different framings of the same strategic situation. Their behaviors are often distinct from average and modal human behaviors, in which case they tend to behave on the more altruistic and cooperative end of the distribution. We estimate that they act as if they are maximizing an average of their own and partner's payoffs.

Keywords: Turing Test, AI, Chatbots, ChatGPT, Behavioral Games

JEL Classification: D9, C91, C88, C72

Suggested Citation

Mei, Qiaozhu and Xie, Yutong and Yuan, Walter and Jackson, Matthew O., A Turing Test: Are Ai Chatbots Behaviorally Similar to Humans? (November 19, 2023). Available at SSRN: https://ssrn.com/abstract=4637354 or http://dx.doi.org/10.2139/ssrn.4637354

Qiaozhu Mei

University of Michigan at Ann Arbor ( email )

Ann Arbor, MI
United States

Yutong Xie

University of Michigan at Ann Arbor ( email )

Ann Arbor, MI
United States

Walter Yuan

MobLab

Matthew O. Jackson (Contact Author)

Stanford University - Department of Economics ( email )

Landau Economics Building
579 Serra Mall
Stanford, CA 94305-6072
United States
1-650-723-3544 (Phone)

HOME PAGE: http://www.stanford.edu/~jacksonm

Santa Fe Institute

1399 Hyde Park Road
Santa Fe, NM 87501
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
970
Abstract Views
3,273
Rank
49,644
PlumX Metrics