10,746 research outputs found
Stable low-rank matrix recovery via null space properties
The problem of recovering a matrix of low rank from an incomplete and
possibly noisy set of linear measurements arises in a number of areas. In order
to derive rigorous recovery results, the measurement map is usually modeled
probabilistically. We derive sufficient conditions on the minimal amount of
measurements ensuring recovery via convex optimization. We establish our
results via certain properties of the null space of the measurement map. In the
setting where the measurements are realized as Frobenius inner products with
independent standard Gaussian random matrices we show that
measurements are enough to uniformly and stably recover an
matrix of rank at most . We then significantly generalize this result by
only requiring independent mean-zero, variance one entries with four finite
moments at the cost of replacing by some universal constant. We also study
the case of recovering Hermitian rank- matrices from measurement matrices
proportional to rank-one projectors. For rank-one projective
measurements onto independent standard Gaussian vectors, we show that nuclear
norm minimization uniformly and stably reconstructs Hermitian rank- matrices
with high probability. Next, we partially de-randomize this by establishing an
analogous statement for projectors onto independent elements of a complex
projective 4-designs at the cost of a slightly higher sampling rate . Moreover, if the Hermitian matrix to be recovered is known to be
positive semidefinite, then we show that the nuclear norm minimization approach
may be replaced by minimizing the -norm of the residual subject to the
positive semidefinite constraint. Then no estimate of the noise level is
required a priori. We discuss applications in quantum physics and the phase
retrieval problem.Comment: 26 page
Non-convex Optimization for Machine Learning
A vast majority of machine learning algorithms train their models and perform
inference by solving optimization problems. In order to capture the learning
and prediction problems accurately, structural constraints such as sparsity or
low rank are frequently imposed or else the objective itself is designed to be
a non-convex function. This is especially true of algorithms that operate in
high-dimensional spaces or that train non-linear models such as tensor models
and deep networks.
The freedom to express the learning problem as a non-convex optimization
problem gives immense modeling power to the algorithm designer, but often such
problems are NP-hard to solve. A popular workaround to this has been to relax
non-convex problems to convex ones and use traditional methods to solve the
(convex) relaxed optimization problems. However this approach may be lossy and
nevertheless presents significant challenges for large scale optimization.
On the other hand, direct approaches to non-convex optimization have met with
resounding success in several domains and remain the methods of choice for the
practitioner, as they frequently outperform relaxation-based techniques -
popular heuristics include projected gradient descent and alternating
minimization. However, these are often poorly understood in terms of their
convergence and other properties.
This monograph presents a selection of recent advances that bridge a
long-standing gap in our understanding of these heuristics. The monograph will
lead the reader through several widely used non-convex optimization techniques,
as well as applications thereof. The goal of this monograph is to both,
introduce the rich literature in this area, as well as equip the reader with
the tools and techniques needed to analyze these simple procedures for
non-convex problems.Comment: The official publication is available from now publishers via
http://dx.doi.org/10.1561/220000005
Blind Demixing for Low-Latency Communication
In the next generation wireless networks, lowlatency communication is
critical to support emerging diversified applications, e.g., Tactile Internet
and Virtual Reality. In this paper, a novel blind demixing approach is
developed to reduce the channel signaling overhead, thereby supporting
low-latency communication. Specifically, we develop a low-rank approach to
recover the original information only based on a single observed vector without
any channel estimation. Unfortunately, this problem turns out to be a highly
intractable non-convex optimization problem due to the multiple non-convex
rankone constraints. To address the unique challenges, the quotient manifold
geometry of product of complex asymmetric rankone matrices is exploited by
equivalently reformulating original complex asymmetric matrices to the
Hermitian positive semidefinite matrices. We further generalize the geometric
concepts of the complex product manifolds via element-wise extension of the
geometric concepts of the individual manifolds. A scalable Riemannian
trust-region algorithm is then developed to solve the blind demixing problem
efficiently with fast convergence rates and low iteration cost. Numerical
results will demonstrate the algorithmic advantages and admirable performance
of the proposed algorithm compared with the state-of-art methods.Comment: 14 pages, accepted by IEEE Transaction on Wireless Communicatio
Informed Non-convex Robust Principal Component Analysis with Features
We revisit the problem of robust principal component analysis with features
acting as prior side information. To this aim, a novel, elegant, non-convex
optimization approach is proposed to decompose a given observation matrix into
a low-rank core and the corresponding sparse residual. Rigorous theoretical
analysis of the proposed algorithm results in exact recovery guarantees with
low computational complexity. Aptly designed synthetic experiments demonstrate
that our method is the first to wholly harness the power of non-convexity over
convexity in terms of both recoverability and speed. That is, the proposed
non-convex approach is more accurate and faster compared to the best available
algorithms for the problem under study. Two real-world applications, namely
image classification and face denoising further exemplify the practical
superiority of the proposed method
- …