Modeling Stock Order Flows and Learning Market-Making from Data
Technical Report CBCL Paper No. 217 / AI Memo No. 2002-009, M.I.T., Cambridge, MA
8 Pages Posted: 28 Jul 2008 Last revised: 30 Jul 2008
Date Written: June 1, 2002
Abstract
Stock markets employ specialized traders, market-makers, designed to provide liquidity and volume to the market by constantly supplying both supply and demand. In this paper, we demonstrate a novel method for modeling the market as a dynamic system and a reinforcement learning algorithm that learns profitable market-making strategies when run on this model.
The sequence of buys and sells for a particular stock, the order flow, we model as an Input-Output Hidden Markov Model fit to historical data. When combined with the dynamics of the order book, this creates a highly non-linear and difficult dynamic system. Our reinforcement learning algorithm, based on likelihood ratios, is run on this partially-observable environment. We demonstrate learning results for two separate real stocks.
Keywords: reinforcement learning, market making, order flow, market simulation
JEL Classification: C11, C13, C15, C32, C51, C52, C53
Suggested Citation: Suggested Citation