Autonomous Algorithmic Collusion: Q-Learning Under Sequential Pricing

42 Pages Posted: 14 Jun 2018 Last revised: 1 Oct 2021

See all articles by Timo Klein

Timo Klein

Utrecht University School of Economics; Oxera Consulting LLP

Date Written: April 1, 2019


Prices are increasingly set by algorithms. One concern is that intelligent algorithms may learn to collude on higher prices even in the absence of the kind of coordination necessary to establish an antitrust infringement. However, exactly how this may happen is an open question. I show how in simulated sequential competition, competing reinforcement learning algorithms can indeed learn to converge to collusive equilibria when the set of discrete prices is limited. When this set increases, the algorithm considered increasingly converges to supra-competitive asymmetric cycles. I show that results are robust to various extensions and discuss practical limitations and policy implications.

Keywords: pricing algorithms, algorithmic collusion, machine learning, reinforcement learning, Q-learning, sequential pricing

JEL Classification: K21, L13, L49

Suggested Citation

Klein, Timo and Klein, Timo, Autonomous Algorithmic Collusion: Q-Learning Under Sequential Pricing (April 1, 2019). RAND Journal of Economics, Forthcoming, Amsterdam Law School Research Paper No. 2018-15, Amsterdam Center for Law & Economics Working Paper No. 2018-05, Available at SSRN: or

Timo Klein (Contact Author)

Oxera Consulting LLP ( email )

Alfred Street
Oxford OX1 4EH
United States

Utrecht University School of Economics ( email )

Vredenburg 138
Utrecht, 3511 BG

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics