Deep Hedging of Derivatives Using Reinforcement Learning

Posted: 27 Jan 2020 Last revised: 6 Feb 2021

See all articles by Jay Cao

Jay Cao

University of Toronto

Jacky Chen

University of Toronto

John C. Hull

University of Toronto - Rotman School of Management

Zissis Poulos

University of Toronto - Rotman School of Management

Date Written: December 20, 2019

Abstract

This paper shows how reinforcement learning can be used to derive optimal hedging strategies for derivatives when there are transaction costs. The paper illustrates the approach by showing the difference between using delta hedging and optimal hedging for a short position in a call option when the objective is to minimize a function equal to the mean hedging cost plus a constant times the standard deviation of the hedging cost. Two situations are considered. In the first, the asset price follows geometric Brownian motion. In the second, the asset price follows a stochastic volatility process. The paper extends the basic reinforcement learning approach in a number of ways. First, it uses two different Q-functions so that both the expected value of the cost and the expected value of the square of the cost are tracked for different state/action combinations. This approach increases the range of objective functions that can be used. Second, it uses a learning algorithm that allows for continuous state and action space. Third, it compares the accounting P&L approach (where the hedged position is valued at each step) and the cash flow approach (where cash inflows and outflows are used). We find that a hybrid approach involving the use of an accounting P&L approach that incorporates a relatively simple valuation model works well. The valuation model does not have to correspond to the process assumed for the underlying asset price.

Keywords: Deep hedging, derivatives, reinforcement learning

JEL Classification: C45, G10

Suggested Citation

Cao, Jay and Chen, Jacky and Hull, John C. and Poulos, Zissis, Deep Hedging of Derivatives Using Reinforcement Learning (December 20, 2019). Available at SSRN: https://ssrn.com/abstract=3514586 or http://dx.doi.org/10.2139/ssrn.3514586

Jay Cao

University of Toronto ( email )

Toronto, Ontario M5S 3G8
Canada

Jacky Chen

University of Toronto ( email )

Toronto, Ontario M5S 3G8
Canada

John C. Hull (Contact Author)

University of Toronto - Rotman School of Management ( email )

105 St. George Street
Toronto, Ontario M5S 3E6 M5S1S4
Canada
(416) 978-8615 (Phone)
416-971-3048 (Fax)

Zissis Poulos

University of Toronto - Rotman School of Management ( email )

105 St. George Street
Toronto, Ontario M5S 3E6 M5S1S4
Canada

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
2,047
PlumX Metrics