Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing

Abstract

This paper investigates the strategic behavior of Large Language Models (LLMs) across various game-theoretic settings, scrutinizing the interplay between game structure and contextual framing in decision-making. We focus our analysis on three advanced LLMs—GPT-3.5, GPT-4, and LLaMa-2—and how they negotiate both the intrinsic aspects of different games and the nuances of their surrounding contexts.

Our results highlight discernable patterns in each model's strategic approach. GPT-3.5 shows significant sensitivity to context but lags in its capacity for abstract strategic thinking. Conversely, both GPT-4 and LLaMa-2 demonstrate a more balanced sensitivity to game structures and contexts, albeit with distinct nuances. Specifically, GPT-4 prioritizes the internal mechanics of the game over its contextual backdrop but does so without nuanced differentiation between game types. In contrast, LLaMa-2 reflects a more granular understanding of individual game structures, while also giving due weight to contextual elements. This suggests that LLaMa-2 is better equipped to navigate the subtleties of different strategic scenarios while also incorporating context into its decision-making, whereas GPT-4 adopts a more generalized, structure-centric strategy.

Keywords: Large Language Models, Generative AI, Game Theory, Strategic Thinking, Social Dilemma, Context Effect

JEL Classification: C7, D9