The Virtue of Complexity in Return Prediction
102 Pages Posted: 15 Dec 2021 Last revised: 26 May 2022
Date Written: December 13, 2021
Contrary to conventional wisdom in finance, return prediction R2 and optimal portfolio Sharpe ratio generally increase with model parameterization, even when minimal regularization is used. We theoretically characterize the behavior of return prediction models in the high complexity regime, i.e. when the number of parameters exceeds the number of observations. Empirically, we document this "virtue of complexity" in US equity market prediction. High complexity models deliver economically large and statistically significant out-of-sample portfolio gains relative to simpler models, due in large part to their remarkable ability to predict recessions.
Keywords: Portfolio choice, machine learning, random matrix theory, benign overfit, overparameterization
JEL Classification: C3, C58, C61, G11, G12, G14
Suggested Citation: Suggested Citation