Autonomous Algorithmic Collusion: Q-Learning Under Sequential Pricing
41 Pages Posted: 14 Jun 2018 Last revised: 17 Nov 2019
Date Written: April 1, 2019
A recent and prominent concern is that intelligent, self-learning pricing algorithms may learn to tacitly collude. To date the debate is mostly based on intuition only. We show in a simulated environment of sequential competition that Q-learning (a simple and well-established reinforcement learning algorithm) can indeed collude on supra-competitive fixed-price equilibria -- at least when the number of discrete prices is limited. When the number of discrete prices increases, Q-learning increasingly converges to profitable asymmetric price cycles. We show that results are robust to various extensions and identify and discuss existing practical limitations.
Keywords: pricing algorithms, algorithmic collusion, machine learning, reinforcement learning, Q-learning, sequential pricing
JEL Classification: K21, L13, L49
Suggested Citation: Suggested Citation