Big Data in Portfolio Allocation -- A New Approach to Successful Portfolio Optimization

Abstract

In the classic mean-variance portfolio theory as proposed by Harry Markowitz, the weights of the optimized portfolios are directly proportional to the inverse of the asset correlation matrix. However, most of contemporary portfolio optimization research focuses on optimizing the correlation matrix itself, and not its inverse. In this article, the author demonstrates that this is a mistake. Specifically, from the Big Data perspective, she proves that the inverse of the correlation matrix is much more unstable and sensitive to random perturbations than the correlation matrix itself. As such, optimization of the inverse of the correlation matrix adds more value to optimal portfolio selection than that of the correlation matrix. The author further shows the empirical results of portfolio reallocation under different common portfolio composition scenarios, and outperforms traditional portfolio allocation techniques out-of-sample, delivering nearly 400% improvement over the equally-weighted allocation over a 20-year investment period on the S&P 500 portfolio with monthly reallocation. In general, the author demonstrates that the correlation inverse optimization proposed in this article significantly outperforms the other core portfolio allocation strategies, such as equally-weighted portfolios, vanilla mean-variance optimization, and the techniques based on the spectral decomposition of the correlation matrix. The results presented in this article are novel in Data Science space, extend far beyond financial data, and are applicable to any data correlation matrices and their inverses, whether in advertising, healthcare or genomics.

Keywords: Portfolio optimization, big data, investment management, correlation, inverse, data science

JEL Classification: C02, C60