Artificial Collusion: Examining Supracompetitive Pricing by Q-Learning Algorithms

Abstract

We examine concerns that pricing algorithms used by competitors would autonomously and systematically learn to collude at supra-competitive prices. Findings of high prices with Q-learning have recently raised that alarm. A detailed analysis of the inner workings of this algorithm type reveals, however, that it does not constitute autonomous algorithmic collusion and is unlikely to be a risk in practice. The `collusive equilibria' only exist by the construction of the state space, a substantial fraction of supra-competitive prices is not sustained by a reward-punishment scheme, and observing reward-punishment patterns need not imply a scheme. If there is convergence on collusive equilibria, it is intrinsically slow and any benefits are obtained on timescales irrelevant to the firms' stated objectives. Moreover, Q-learning algorithms are outperformed by the first alternative pricing algorithm. Our analysis gives criteria for practically relevant colluding pricing algorithms that would constitute a threat to competition. They likely require malign programming, intent and explicit coordination, that would show from the codes.

Keywords: algorithmic collusion, multi-agent learning, Q-learning, pricing

JEL Classification: C63, L13, L44, K21