Can Deep Reinforcement Learning Solve the Portfolio Allocation Problem? (PhD Manuscript)

Abstract

The promise of deep reinforcement learning (DRL) is to make no initial assumptions in terms of decisions or rules and let the machine find the best solution. In this thesis, we show that this type of machine learning method provides a new solution for portfolio allocation. In the first part, we present how to apply DRL to portfolio allocation and take into account the specific nature of financial time series. We introduce the concept of contextual variables allowing better learning. We change the cross-validation to a stepwise approach to ensure that the training data does not contain any future points compared to the validation and test data. We empirically show that DRL makes it possible to go beyond the state of the art of portfolio allocation methods by finding portfolios better adapted to market conditions, thanks to layers of convolutions allowing us to capture the dependence between market data and allocation decisions. We conclude this part with a model-based approach where DRL selects volatility-targeting models. In the second part, we present theoretical results justifying the DRL approach. We show how the DRL approach generalises classical portfolio theories. We study how DRL methods achieve variance reduction. We find similarities between reinforcement learning and supervised learning.

Keywords: Deep reinforcement learning, portfolio allocation, contextual data

JEL Classification: G11, G13