13,306 research outputs found
OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage
The truncated singular value decomposition (SVD) of the measurement matrix is
the optimal solution to the_representation_ problem of how to best approximate
a noisy measurement matrix using a low-rank matrix. Here, we consider the
(unobservable)_denoising_ problem of how to best approximate a low-rank signal
matrix buried in noise by optimal (re)weighting of the singular vectors of the
measurement matrix. We exploit recent results from random matrix theory to
exactly characterize the large matrix limit of the optimal weighting
coefficients and show that they can be computed directly from data for a large
class of noise models that includes the i.i.d. Gaussian noise case.
Our analysis brings into sharp focus the shrinkage-and-thresholding form of
the optimal weights, the non-convex nature of the associated shrinkage function
(on the singular values) and explains why matrix regularization via singular
value thresholding with convex penalty functions (such as the nuclear norm)
will always be suboptimal. We validate our theoretical predictions with
numerical simulations, develop an implementable algorithm (OptShrink) that
realizes the predicted performance gains and show how our methods can be used
to improve estimation in the setting where the measured matrix has missing
entries.Comment: Published version. The algorithm can be downloaded from
http://www.eecs.umich.edu/~rajnrao/optshrin
Data-Driven Optimal Shrinkage of Singular Values under High-Dimensional Noise with Separable Covariance Structure
We develop a data-driven optimal shrinkage algorithm for matrix denoising in
the presence of high-dimensional noise with separable covariance structure;
that is, the nose is colored and dependent. The algorithm, coined extended
OptShrink (eOptShrink), involves a new imputation and rank estimation and we do
not need to estimate the separable covariance structure of the noise. On the
theoretical side, we study the asymptotic behavior of singular values and
singular vectors of the random matrix associated with the noisy data, including
the sticking property of non-outlier singular values and delocalization of the
non-outlier singular vectors with a convergence rate. We apply these results to
establish the guarantee of the imputation, rank estimation and eOptShrink
algorithm with a convergence rate. On the application side, in addition to a
series of numerical simulations with a comparison with various state-of-the-art
optimal shrinkage algorithms, we apply eOptShrink to extract fetal
electrocardiogram from the single channel trans-abdominal maternal
electrocardiogram.Comment: arXiv admin note: text overlap with arXiv:1905.13060 by other author
- …