Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

IEEE Transactions on Signal Processing, Volume: 64, Issue: 23, Dec.1, 2016

16 Pages Posted: 24 Jan 2018

See all articles by Konstantinos Benidis

Konstantinos Benidis

Hong Kong University of Science & Technology (HKUST) - Department of Electronic and Computer Engineering

Ying Sun

Hong Kong University of Science & Technology (HKUST)

Prabhu Babu

Hong Kong University of Science & Technology (HKUST) - Department of Electronic and Computer Engineering

Daniel Palomar

Hong Kong University of Science and Technology (HKUST)

Date Written: January 29, 2016

Abstract

The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty term into the objective function that encourages a sparse solution. However, the vast majority of the resulting methods achieve sparsity at the expense of sacrificing the orthogonality property. In this paper, we develop a new method to estimate dominant sparse eigenvectors without trading off their orthogonality. The problem is highly non-convex and hard to handle. We apply the minorization-maximization framework where we iteratively maximize a tight lower bound (surrogate function) of the objective function over the Stiefel manifold. The inner maximization problem turns out to be a rectangular Procrustes problem, which has a closed form solution. In addition, we propose a method to improve the covariance estimation problem when its underlying eigenvectors are known to be sparse. We use the eigenvalue decomposition of the covariance matrix to formulate an optimization problem where we impose sparsity on the corresponding eigenvectors. Numerical experiments show that the proposed eigenvector extraction algorithm outperforms existing algorithms in terms of support recovery and explained variance, while the covariance estimation algorithms improve significantly the sample covariance estimator.

Keywords: Sparse PCA, Procrustes, Stiefel Manifold, Minorization-Maximization, Covariance Estimation

JEL Classification: C13, C21, C51

Suggested Citation

Benidis, Konstantinos and Sun, Ying and Babu, Prabhu and Palomar, Daniel, Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation (January 29, 2016). IEEE Transactions on Signal Processing, Volume: 64, Issue: 23, Dec.1, 2016, Available at SSRN: https://ssrn.com/abstract=3101689

Konstantinos Benidis

Hong Kong University of Science & Technology (HKUST) - Department of Electronic and Computer Engineering ( email )

Clearwater Bay
Kowloon
Hong Kong

Ying Sun

Hong Kong University of Science & Technology (HKUST) ( email )

Clear Water Bay
Hong Kong

Prabhu Babu

Hong Kong University of Science & Technology (HKUST) - Department of Electronic and Computer Engineering ( email )

Clear Water Bay
Hong Kong

Daniel Palomar (Contact Author)

Hong Kong University of Science and Technology (HKUST) ( email )

Clear Water Bay
Kowloon, 00000
Hong Kong

HOME PAGE: http://www.danielppalomar.com/

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
42
Abstract Views
457
PlumX Metrics