Singular Ridge Regression with Homoscedastic Residuals: Generalization Error with Estimated Parameters

24 Pages Posted: 13 Jul 2017

See all articles by Lyudmila Grigoryeva

Lyudmila Grigoryeva

University of Konstanz

Juan-Pablo Ortega

Centre National de la Recherche Scientifique (CNRS); Nanyang Technological University

Date Written: May 20, 2016

Abstract

This paper characterizes the conditional distribution properties of the finite sample ridge regression estimator and uses that result to evaluate total regression and generalization errors that incorporate the inaccuracies committed at the time of parameter estimation. The paper provides explicit formulas for those errors. Unlike other classical references in this setup, our results take place in a fully singular setup that does not assume the existence of a solution for the non-regularized regression problem. In exchange, we invoke a conditional homoscedasticity hypothesis on the regularized regression residuals that is crucial in our developments.

Keywords: ridge regression, singular regression, training error, testing error, generalization error, regularization methods, high-dimensional regression

JEL Classification: C35, C52

Suggested Citation

Grigoryeva, Lyudmila and Ortega, Juan-Pablo and Ortega, Juan-Pablo, Singular Ridge Regression with Homoscedastic Residuals: Generalization Error with Estimated Parameters (May 20, 2016). Available at SSRN: https://ssrn.com/abstract=2782670 or http://dx.doi.org/10.2139/ssrn.2782670

Lyudmila Grigoryeva

University of Konstanz ( email )

Fach D-144
Universitätsstraße 10
Konstanz, D-78457
Germany

Juan-Pablo Ortega (Contact Author)

Centre National de la Recherche Scientifique (CNRS) ( email )

16 route de Gray
Besançon, 25030
France

HOME PAGE: http://juan-pablo-ortega.com

Nanyang Technological University ( email )

21 Nanyang Link
Singapore, 637371
Singapore

HOME PAGE: http://https://juan-pablo-ortega.com

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
10
Abstract Views
189
PlumX Metrics