Lprr: Locality Preserving Robust Regression Based Sparse Feature Extraction

Abstract

Jointly sparse projection learning attracts considerable attention due to its strong interpretability in feature extraction. To address the challenges related to weak discriminating representation in supervised feature extraction, we propose a more powerful regression framework. Based on the framework, we exhibit a new regression model called locality preserving robust regression (LPRR). In LPRR, we first combine the reconstruction error minimization and the projection variance maximization to explore the structured information of the data. Then, the label information is utilized and the low rank representation can be learned to explore the latent correlation structures among different classes. Furthermore, $L_{2,1}$-norm is applied to measure the loss function and regularization terms, enhancing the robustness of the model and ensuring the joint sparsity of the projection matrix. An iterative algorithm is elaborately designed to achieve the optimal solutions of LPRR, in which the subproblem of LPRR can be regarded as a general quadratic problem on the Stiefel manifold. The convergence and the computational complexity of LPRR are analyzed rigorously. Finally, comprehensive experiments demonstrate the competitive performance of the proposed algorithm.

Keywords: Locality preserving robust regression, Supervised learning, Feature extraction, Sparse projection learning