Reinforcement Learning Equilibrium in Limit Order Markets

Abstract

This paper introduces an information-based reinforcement learning to exploit information channels to traders’ trading behavior in an equilibrium limit order market. Anticipating that informed traders are more likely to submit market buy (sell) orders when asset is significantly under (over) valued, uninformed traders tend to chase market buy (sell) orders of the informed to buy (sell). To gain from the order chasing of the uninformed, informed traders strategically submit more market buy (sell) and limit sell (buy) orders. This amplifies the order chasing behaviour of the uninformed, generating predictable trading behaviours that can improve information efficiency but reduce market liquidity. Order book information and learning can have opposite effects on order choices and endogenous liquidity provision for the informed and uninformed. Furthermore, more informed trading is beneficial, but fast trading can be harmful for market quality.

Keywords: Reinforcement learning, strategic trading, limit order market, evolutionary equilibrium, herding, market liquidity, price discovery

JEL Classification: G14, C63, D82, D83