AlphaPortfolio: Direct Construction Through Deep Reinforcement Learning and Interpretable AI

Abstract

We directly optimize the objectives of portfolio management via deep reinforcement learning---an alternative to conventional supervised-learning paradigms that routinely entail first-step estimations of return distributions or risk premia. We develop multi-sequence, attention-based neural-network models tailored for the distinguishing features of financial big data, while allowing interactions with the market states and training without labels. Such AlphaPortfolio models yield stellar out-of-sample performances (e.g., Sharpe ratio above two and over 13% risk-adjusted alpha with monthly re-balancing) that are robust under various market conditions and economic restrictions (e.g., exclusion of small and illiquid stocks). We further demonstrate AlphaPortfolio's flexibility to incorporate transaction costs, state interactions, and alternative objectives, before applying polynomial-feature-sensitivity analysis to uncover key drivers of investment performance, including their rotation and nonlinearity. Overall, we highlight the utility of deep reinforcement learning in finance and "economic distillation" for model interpretation.

Keywords: Artificial Intelligence, Distillation, LSTM, Machine Learning, Portfolio Theory, Reinforcement Learning.