1,068 research outputs found
Sparse CCA: Adaptive Estimation and Computational Barriers
Canonical correlation analysis is a classical technique for exploring the
relationship between two sets of variables. It has important applications in
analyzing high dimensional datasets originated from genomics, imaging and other
fields. This paper considers adaptive minimax and computationally tractable
estimation of leading sparse canonical coefficient vectors in high dimensions.
First, we establish separate minimax estimation rates for canonical coefficient
vectors of each set of random variables under no structural assumption on
marginal covariance matrices. Second, we propose a computationally feasible
estimator to attain the optimal rates adaptively under an additional sample
size condition. Finally, we show that a sample size condition of this kind is
needed for any randomized polynomial-time estimator to be consistent, assuming
hardness of certain instances of the Planted Clique detection problem. The
result is faithful to the Gaussian models used in the paper. As a byproduct, we
obtain the first computational lower bounds for sparse PCA under the Gaussian
single spiked covariance model
Sparse PCA: Optimal rates and adaptive estimation
Principal component analysis (PCA) is one of the most commonly used
statistical procedures with a wide range of applications. This paper considers
both minimax and adaptive estimation of the principal subspace in the high
dimensional setting. Under mild technical conditions, we first establish the
optimal rates of convergence for estimating the principal subspace which are
sharp with respect to all the parameters, thus providing a complete
characterization of the difficulty of the estimation problem in term of the
convergence rate. The lower bound is obtained by calculating the local metric
entropy and an application of Fano's lemma. The rate optimal estimator is
constructed using aggregation, which, however, might not be computationally
feasible. We then introduce an adaptive procedure for estimating the principal
subspace which is fully data driven and can be computed efficiently. It is
shown that the estimator attains the optimal rates of convergence
simultaneously over a large collection of the parameter spaces. A key idea in
our construction is a reduction scheme which reduces the sparse PCA problem to
a high-dimensional multivariate regression problem. This method is potentially
also useful for other related problems.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1178 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Minimax bounds for sparse PCA with noisy high-dimensional data
We study the problem of estimating the leading eigenvectors of a
high-dimensional population covariance matrix based on independent Gaussian
observations. We establish a lower bound on the minimax risk of estimators
under the loss, in the joint limit as dimension and sample size increase
to infinity, under various models of sparsity for the population eigenvectors.
The lower bound on the risk points to the existence of different regimes of
sparsity of the eigenvectors. We also propose a new method for estimating the
eigenvectors by a two-stage coordinate selection scheme.Comment: 1 figur
Optimal Estimation and Rank Detection for Sparse Spiked Covariance Matrices
This paper considers sparse spiked covariance matrix models in the
high-dimensional setting and studies the minimax estimation of the covariance
matrix and the principal subspace as well as the minimax rank detection. The
optimal rate of convergence for estimating the spiked covariance matrix under
the spectral norm is established, which requires significantly different
techniques from those for estimating other structured covariance matrices such
as bandable or sparse covariance matrices. We also establish the minimax rate
under the spectral norm for estimating the principal subspace, the primary
object of interest in principal component analysis. In addition, the optimal
rate for the rank detection boundary is obtained. This result also resolves the
gap in a recent paper by Berthet and Rigollet [1] where the special case of
rank one is considered
- …