Deep Learning under Model Uncertainty
Posted: 8 Jul 2021 Last revised: 1 Feb 2022
Date Written: June 28, 2021
Deep learning has proven to lead to very powerful predictive models, often outperforming classical regression models such as generalized linear models. Deep learning models perform representation learning, which means that they do covariate engineering themselves so that explanatory variables are optimally transformed for the predictive problem at hand. A crucial object in deep learning is the loss function (objective function) for model fitting which implicitly reflects the distributional properties of the observed samples. The purpose of this article is to discuss the choice of this loss function, in particular, we give a specific proposal of a loss function choice under model uncertainty. This proposal turns out to robustify representation learning and prediction.
Keywords: deep learning, neural network, representation learning, model uncertainty, exponential dispersion family, Tweedie's family, generalized linear model, regression
JEL Classification: G22, C10, C13
Suggested Citation: Suggested Citation