Autonomous Algorithmic Collusion: Q-Learning Under Sequential Pricing

41 Pages Posted: 14 Jun 2018 Last revised: 17 Nov 2019

See all articles by Timo Klein

Timo Klein

Utrecht University School of Economics; Oxera Consulting LLP; University of Amsterdam - Amsterdam School of Economics (ASE)

Date Written: April 1, 2019

Abstract

A recent and prominent concern is that intelligent, self-learning pricing algorithms may learn to tacitly collude. To date the debate is mostly based on intuition only. We show in a simulated environment of sequential competition that Q-learning (a simple and well-established reinforcement learning algorithm) can indeed collude on supra-competitive fixed-price equilibria -- at least when the number of discrete prices is limited. When the number of discrete prices increases, Q-learning increasingly converges to profitable asymmetric price cycles. We show that results are robust to various extensions and identify and discuss existing practical limitations.

Keywords: pricing algorithms, algorithmic collusion, machine learning, reinforcement learning, Q-learning, sequential pricing

JEL Classification: K21, L13, L49

Suggested Citation

Klein, Timo, Autonomous Algorithmic Collusion: Q-Learning Under Sequential Pricing (April 1, 2019). Amsterdam Law School Research Paper No. 2018-15, Amsterdam Center for Law & Economics Working Paper No. 2018-05, Available at SSRN: https://ssrn.com/abstract=3195812 or http://dx.doi.org/10.2139/ssrn.3195812

Timo Klein (Contact Author)

Utrecht University School of Economics ( email )

Vredenburg 138
Utrecht, 3511 BG
Netherlands

Oxera Consulting LLP ( email )

Alfred Street
Oxford OX1 4EH
United States

University of Amsterdam - Amsterdam School of Economics (ASE) ( email )

Roetersstraat 11
Amsterdam, North Holland 1018 WB
Netherlands

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
460
Abstract Views
1,800
rank
67,721
PlumX Metrics