The Algorithmic Learning Equations: Evolving Strategies in Dynamic Games

Abstract

We introduce the algorithmic learning equations, a set of ordinary differential equations which characterizes the finite-time and asymptotic behavior of the stochastic interaction between state-dependent learning algorithms in dynamic games. Our framework allows for a variety of information and memory structures, including noisy, perfect, private, and public monitoring and for the possibility that players use distinct learning algorithms. We prove that play converges to a correlated equilibrium for a family of algorithms under correlated private signals. Finally, we apply our methodology in a repeated 2x2 prisoner's dilemma game with perfect monitoring. We show that algorithms can learn a reward-punishment mechanism to sustain tacit collusion. Additionally, we find that algorithms can also learn to coordinate in cycles of cooperation and defection.

Keywords: Stochastic Approximation, Learning in Games, Reinforcement Learning, Dynamic Games, Tacit Collusion, Artificial Intelligence

JEL Classification: C60, C72, C73, D83