Forward Regression for Ultra-High Dimensional Variable Screening

33 Pages Posted: 10 Apr 2009

See all articles by Hansheng Wang

Hansheng Wang

Peking University - Guanghua School of Management

Date Written: April 9, 2009

Abstract

Motivated by the seminal theory of Sure Independence Screening (Fan and Lv, 2008, SIS), we investigate here another popular and classical variable screening method, namely, Forward Regression (FR). Our theoretical analysis reveals that FR can identify all relevant predictors consistently, even if the predictor dimension is substantially larger than the sample size. In particular, if the dimension of the true model is finite, FR can discover all relevant predictors within a finite number of steps. To practically select the "best" candidate from the models generated by FR, the recently proposed BIC criterion of Chen and Chen (2008) can be used. The resulting model can then serve as an excellent starting point, from where many existing variable selection methods (e.g., SCAD and Adaptive LASSO) can be applied directly. FR's outstanding ¿nite sample performances are con¿rmed by extensive numerical studies.

Keywords: Adaptive Lasso, BIC, Forward Regression, LASSO, SCAD, Screening Consistency

JEL Classification: C10, C13

Suggested Citation

Wang, Hansheng, Forward Regression for Ultra-High Dimensional Variable Screening (April 9, 2009). Available at SSRN: https://ssrn.com/abstract=1376127 or http://dx.doi.org/10.2139/ssrn.1376127

Hansheng Wang (Contact Author)

Peking University - Guanghua School of Management ( email )

Peking University
Beijing, Beijing 100871
China

HOME PAGE: http://hansheng.gsm.pku.edu.cn

Register to save articles to
your library

Register

Paper statistics

Downloads
953
Abstract Views
2,809
rank
23,422
PlumX Metrics