Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework

39 Pages Posted: 29 May 2019

See all articles by Haoran Wang

Haoran Wang

Columbia University - Department of Industrial Engineering and Operations Research (IEOR)

Xun Yu Zhou

Columbia University - Department of Industrial Engineering and Operations Research (IEOR)

Date Written: May 5, 2019

Abstract

We approach the continuous-time mean-variance (MV) portfolio selection with reinforcement learning (RL). The problem is to achieve the best tradeoff between exploration and exploitation, and is formulated as an entropy-regularized, relaxed stochastic control problem. We prove that the optimal feedback policy for this problem must be Gaussian, with time-decaying variance. We then establish connections between the entropy-regularized MV and the classical MV, including the solvability equivalence and the convergence as exploration weighting parameter decays to zero. Finally, we prove a policy improvement theorem, based on which we devise an implementable RL algorithm. We find that our algorithm outperforms both an adaptive control based method and a deep neural networks based algorithm by a large margin in our simulations.

Keywords: Reinforcement learning, mean-variance portfolio selection, entropy regularization, stochastic control, value function, Gaussian distribution, policy improvement theorem

JEL Classification: C02, C61, C63, G11,

Suggested Citation

Wang, Haoran and Zhou, Xunyu, Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework (May 5, 2019). Available at SSRN: https://ssrn.com/abstract=3382932 or http://dx.doi.org/10.2139/ssrn.3382932

Haoran Wang (Contact Author)

Columbia University - Department of Industrial Engineering and Operations Research (IEOR) ( email )

331 S.W. Mudd Building
500 West 120th Street
New York, NY 10027
United States

Xunyu Zhou

Columbia University - Department of Industrial Engineering and Operations Research (IEOR) ( email )

331 S.W. Mudd Building
500 West 120th Street
New York, NY 10027
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
122
Abstract Views
492
rank
242,209
PlumX Metrics