G-Learner and GIRL: Goal Based Wealth Management with Reinforcement Learning

20 Pages Posted: 20 Mar 2020 Last revised: 23 Mar 2020

See all articles by Matthew Francis Dixon

Matthew Francis Dixon

Illinois Institute of Technology

Igor Halperin

New York University (NYU) - NYU Tandon School of Engineering

Date Written: February 25, 2020

Abstract

We present a reinforcement learning approach to goal based wealth management problems such as optimization of retirement plans or target dated funds. In such problems, an investor seeks to achieve a financial goal by making periodic investments in the portfolio while being employed, and periodically draws from the account when in retirement, in addition to the ability to re-balance the portfolio by selling and buying different assets (e.g. stocks). Instead of relying on a utility of consumption, we present G-Learner: a reinforcement learning algorithm that operates with explicitly defined one-step rewards, does not assume a data generation process, and is suitable for noisy data. Our approach is based on G-learning (Fox et al., 2015) - a probabilistic extension of the Q-learning method of reinforcement learning. In this paper, we demonstrate how G-learning, when applied to a quadratic reward and Gaussian reference policy, gives an entropy-regulated Linear Quadratic Regulator (LQR). This critical insight provides a novel and computationally tractable tool for wealth management tasks which scales to high dimensional portfolios. In addition to the solution of the direct problem of G-learning, we also present a new algorithm, GIRL, that extends our goal-based G-learning approach to the setting of Inverse Reinforcement Learning (IRL) where rewards collected by the agent are not observed, and should instead be inferred. We demonstrate that GIRL can successfully learn the reward parameters of a G-Learner agent and thus imitate its behavior. Finally, we discuss potential applications of the G-Learner and GIRL algorithms for wealth management and robo-advising.

Keywords: G-learning, goal based wealth management, imitation learning, reinforcement learning

JEL Classification: G11, C61, C63

Suggested Citation

Dixon, Matthew Francis and Halperin, Igor, G-Learner and GIRL: Goal Based Wealth Management with Reinforcement Learning (February 25, 2020). Available at SSRN: https://ssrn.com/abstract=3543852 or http://dx.doi.org/10.2139/ssrn.3543852

Matthew Francis Dixon (Contact Author)

Illinois Institute of Technology ( email )

Department of Math
W 32nd St., E1 room 208, 10 S Wabash Ave, Chicago,
Chicago, IL 60616
United States

Igor Halperin

New York University (NYU) - NYU Tandon School of Engineering ( email )

6 MetroTech Center
Brooklyn, NY 11201
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
248
Abstract Views
1,156
rank
138,306
PlumX Metrics