Autonomous Algorithmic Collusion: Q-Learning Under Sequential Pricing
37 Pages Posted: 14 Jun 2018 Last revised: 12 Jul 2019
Date Written: July 2019
A recent and prominent concern within competition policy and regulation is whether autonomous machine learning algorithms may learn to collude on prices. We show in a simulated environment that when algorithmic competitors update prices sequentially, Q-learning (a simple but well-established self-learning algorithm) coordinates on high fixed-price equilibria or profitable asymmetric price cycles. This occurs even though the algorithm does not communicate and does not receive any instructions to collude. We show that results are robust to changes to the learning parameters and timing and discuss how more advanced algorithms could deal with practical limitations.
Keywords: artificial intelligence, machine learning, reinforcement learning, Q-learning, pricing algorithms, algorithmic collusion, sequential pricing
JEL Classification: L13, L41, D43, D83
Suggested Citation: Suggested Citation