On General Adaptive Sparse Principal Component Analysis

Abstract

The method of sparse principal component analysis (S-PCA) proposed by Zou et al. (2006) is an attractive approach to obtain sparse loadings in principal component analysis (PCA). SPCA was motivated by reformulating PCA as a least squares problem so that a lasso penalty on the loading coefficients can be applied. In this article, we propose new estimates to improve S-PCA on the following two aspects. Firstly, we propose a method of simple adaptive sparse principal component analysis (SAS-PCA), which uses the adaptive lasso penalty (Zou, 2006; Wang et al., 2007) instead of the lasso penalty in S-PCA. Secondly, we replace the least squares objective function in S-PCA by a general least squares objective function. This formulation allows us to study many related sparse PCA estimators under a unified theoretical framework and leads to the method of general adaptive sparse principal component analysis (GAS-PCA). Compared with SAS-PCA, GAS-PCA enjoys much further improved finite sample performance. In addition to that, we show that when a BIC-type criterion is used for selecting the tuning parameters, the resulting estimates are consistent in variable selection. Numerical studies are conducted to compare the finite sample performance of various competing methods.

Keywords: Adaptive Lasso, BIC, GAS-PCA, LARS, Lasso, S-PCA, SAS-PCA

JEL Classification: C5, C52