A Theoretical Analysis of Cooperative Behavior in Multi-Agent Q-Learning
23 Pages Posted: 7 Feb 2006
Date Written: February 1, 2006
A number of experimental studies have investigated whether cooperative behavior may emerge in multi-agent Q-learning. In some studies cooperative behavior did emerge, in others it did not. This report provides a theoretical analysis of this issue. The analysis focuses on multi-agent Q-learning in iterated prisoner’s dilemmas. It is shown that under certain assumptions cooperative behavior may emerge when multi-agent Q-learning is applied in an iterated prisoner’s dilemma. An important consequence of the analysis is that multi-agent Q-learning may result in non-Nash behavior. It is found experimentally that the theoretical results derived in this report are quite robust to violations of the underlying assumptions.
Keywords: Cooperation, Multi-Agent Q-Learning, Multi-Agent Reinforcement Learning, Nash Equilibrium, Prisoner’s Dilemma
Suggested Citation: Suggested Citation