Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing
25 Pages Posted: 14 Sep 2023 Last revised: 17 Sep 2023
Date Written: September 10, 2023
Abstract
This paper investigates the strategic behavior of Large Language Models (LLMs) across various game-theoretic settings, scrutinizing the interplay between game structure and contextual framing in decision-making. We focus our analysis on three advanced LLMs—GPT-3.5, GPT-4, and LLaMa-2—and how they negotiate both the intrinsic aspects of different games and the nuances of their surrounding contexts.
Our results highlight discernable patterns in each model's strategic approach. GPT-3.5 shows significant sensitivity to context but lags in its capacity for abstract strategic thinking. Conversely, both GPT-4 and LLaMa-2 demonstrate a more balanced sensitivity to game structures and contexts, albeit with distinct nuances. Specifically, GPT-4 prioritizes the internal mechanics of the game over its contextual backdrop but does so without nuanced differentiation between game types. In contrast, LLaMa-2 reflects a more granular understanding of individual game structures, while also giving due weight to contextual elements. This suggests that LLaMa-2 is better equipped to navigate the subtleties of different strategic scenarios while also incorporating context into its decision-making, whereas GPT-4 adopts a more generalized, structure-centric strategy.
Keywords: Large Language Models, Generative AI, Game Theory, Strategic Thinking, Social Dilemma, Context Effect
JEL Classification: C7, D9
Suggested Citation: Suggested Citation