2,083 research outputs found
OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage
The truncated singular value decomposition (SVD) of the measurement matrix is
the optimal solution to the_representation_ problem of how to best approximate
a noisy measurement matrix using a low-rank matrix. Here, we consider the
(unobservable)_denoising_ problem of how to best approximate a low-rank signal
matrix buried in noise by optimal (re)weighting of the singular vectors of the
measurement matrix. We exploit recent results from random matrix theory to
exactly characterize the large matrix limit of the optimal weighting
coefficients and show that they can be computed directly from data for a large
class of noise models that includes the i.i.d. Gaussian noise case.
Our analysis brings into sharp focus the shrinkage-and-thresholding form of
the optimal weights, the non-convex nature of the associated shrinkage function
(on the singular values) and explains why matrix regularization via singular
value thresholding with convex penalty functions (such as the nuclear norm)
will always be suboptimal. We validate our theoretical predictions with
numerical simulations, develop an implementable algorithm (OptShrink) that
realizes the predicted performance gains and show how our methods can be used
to improve estimation in the setting where the measured matrix has missing
entries.Comment: Published version. The algorithm can be downloaded from
http://www.eecs.umich.edu/~rajnrao/optshrin
The singular values and vectors of low rank perturbations of large rectangular random matrices
In this paper, we consider the singular values and singular vectors of
finite, low rank perturbations of large rectangular random matrices.
Specifically, we prove almost sure convergence of the extreme singular values
and appropriate projections of the corresponding singular vectors of the
perturbed matrix. As in the prequel, where we considered the eigenvalue aspect
of the problem, the non-random limiting value is shown to depend explicitly on
the limiting singular value distribution of the unperturbed matrix via an
integral transforms that linearizes rectangular additive convolution in free
probability theory. The large matrix limit of the extreme singular values of
the perturbed matrix differs from that of the original matrix if and only if
the singular values of the perturbing matrix are above a certain critical
threshold which depends on this same aforementioned integral transform. We
examine the consequence of this singular value phase transition on the
associated left and right singular eigenvectors and discuss the finite
fluctuations above these non-random limits.Comment: 22 pages, presentation of the main results and of the hypotheses
slightly modifie
The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices
We consider the eigenvalues and eigenvectors of finite, low rank
perturbations of random matrices. Specifically, we prove almost sure
convergence of the extreme eigenvalues and appropriate projections of the
corresponding eigenvectors of the perturbed matrix for additive and
multiplicative perturbation models. The limiting non-random value is shown to
depend explicitly on the limiting eigenvalue distribution of the unperturbed
random matrix and the assumed perturbation model via integral transforms that
correspond to very well known objects in free probability theory that linearize
non-commutative free additive and multiplicative convolution. Furthermore, we
uncover a phase transition phenomenon whereby the large matrix limit of the
extreme eigenvalues of the perturbed matrix differs from that of the original
matrix if and only if the eigenvalues of the perturbing matrix are above a
certain critical threshold. Square root decay of the eigenvalue density at the
edge is sufficient to ensure that this threshold is finite. This critical
threshold is intimately related to the same aforementioned integral transforms
and our proof techniques bring this connection and the origin of the phase
transition into focus. Consequently, our results extend the class of `spiked'
random matrix models about which such predictions (called the BBP phase
transition) can be made well beyond the Wigner, Wishart and Jacobi random
ensembles found in the literature. We examine the impact of this eigenvalue
phase transition on the associated eigenvectors and observe an analogous phase
transition in the eigenvectors. Various extensions of our results to the
problem of non-extreme eigenvalues are discussed.Comment: 27 pages, 1 figure. The paragraph devoted to rectangular matrices has
been suppressed in this version (it will appear independently in a
forthcoming paper
- …