Bias-Variance Trade-Off in Portfolio Optimization under Expected Shortfall with ℓ₂ Regularization

Abstract

The optimization of a large random portfolio under the Expected Shortfall risk measure with an ℓ₂ regularizer is carried out by analytical calculation. The regularizer reins in the large sample fluctuations and the concomitant divergent estimation error, and eliminates the phase transition where this error would otherwise blow up. In the data-dominated region, where the number of different assets in the portfolio is much less than the length of the available time series, the regularizer plays a negligible role, while in the opposite limit (which occurs much more frequently in practice), where the size of samples is comparable to, or even smaller than the number of assets, the optimum is almost entirely determined by the regularizer. Our results show that the transition region between these two extremes is relatively narrow, and it is only here that one can meaningfully speak of a trade-off between fluctuations and bias. The cause of both the anomalously large fluctuations and the limited usefulness of regularization is the unboundedness of Expected Shortfall as a loss function, a property it shares with all the other coherent measures, but also with downside risk measures in general, including Value at Risk.

Keywords: Portfolio Optimization, Bias-Variance Trade-Off, Expected Shortfall

JEL Classification: C02, C13, C61, G11, G32