Integrating Machine Behavior into Human Subject Experiments: A User-Friendly Toolkit and Illustrations

Abstract

Large Language Models (LLMs) have the potential to profoundly transform and enrich experimental economic research. We propose a new software framework, “alter_ego”, which makes it easy to design experiments between LLMs and to integrate LLMs into oTree-based experiments with human subjects. Our toolkit is freely available at github.com/mrpg/ego. To illustrate, we run differently framed prisoner’s dilemmas with interacting machines as well as with human-machine interaction. Framing effects in machine-only treatments are strong and similar to those expected from previous human-only experiments, yet less pronounced and qualitatively different if machines interact with human participants.

Keywords: Software for experiments, large language models, human-machine interaction, framing

JEL Classification: C91, C92, D91, O33, L86