126 research outputs found
Certifying the absence of spurious local minima at infinity
When searching for global optima of nonconvex unconstrained optimization
problems, it is desirable that every local minimum be a global minimum. This
property of having no spurious local minima is true in various problems of
interest nowadays, including principal component analysis, matrix sensing, and
linear neural networks. However, since these problems are non-coercive, they
may yet have spurious local minima at infinity. The classical tools used to
analyze the optimization landscape, namely the gradient and the Hessian, are
incapable of detecting spurious local minima at infinity. In this paper, we
identify conditions that certify the absence of spurious local minima at
infinity, one of which is having bounded subgradient trajectories. We check
that they hold in several applications of interest.Comment: 31 pages, 4 figure
From Symmetry to Geometry: Tractable Nonconvex Problems
As science and engineering have become increasingly data-driven, the role of
optimization has expanded to touch almost every stage of the data analysis
pipeline, from the signal and data acquisition to modeling and prediction. The
optimization problems encountered in practice are often nonconvex. While
challenges vary from problem to problem, one common source of nonconvexity is
nonlinearity in the data or measurement model. Nonlinear models often exhibit
symmetries, creating complicated, nonconvex objective landscapes, with multiple
equivalent solutions. Nevertheless, simple methods (e.g., gradient descent)
often perform surprisingly well in practice.
The goal of this survey is to highlight a class of tractable nonconvex
problems, which can be understood through the lens of symmetries. These
problems exhibit a characteristic geometric structure: local minimizers are
symmetric copies of a single "ground truth" solution, while other critical
points occur at balanced superpositions of symmetric copies of the ground
truth, and exhibit negative curvature in directions that break the symmetry.
This structure enables efficient methods to obtain global minimizers. We
discuss examples of this phenomenon arising from a wide range of problems in
imaging, signal processing, and data analysis. We highlight the key role of
symmetry in shaping the objective landscape and discuss the different roles of
rotational and discrete symmetries. This area is rich with observed phenomena
and open problems; we close by highlighting directions for future research.Comment: review paper submitted to SIAM Review, 34 pages, 10 figure
Nonsmooth rank-one matrix factorization landscape
We provide the first positive result on the nonsmooth optimization landscape
of robust principal component analysis, to the best of our knowledge. It is the
object of several conjectures and remains mostly uncharted territory. We
identify a necessary and sufficient condition for the absence of spurious local
minima in the rank-one case. Our proof exploits the subdifferential regularity
of the objective function in order to eliminate the existence quantifier from
the first-order optimality condition known as Fermat's rule.Comment: 23 pages, 5 figure
A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality
Symmetric nonnegative matrix factorization (SymNMF) has important
applications in data analytics problems such as document clustering, community
detection and image segmentation. In this paper, we propose a novel nonconvex
variable splitting method for solving SymNMF. The proposed algorithm is
guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points of the
nonconvex SymNMF problem. Furthermore, it achieves a global sublinear
convergence rate. We also show that the algorithm can be efficiently
implemented in parallel. Further, sufficient conditions are provided which
guarantee the global and local optimality of the obtained solutions. Extensive
numerical results performed on both synthetic and real data sets suggest that
the proposed algorithm converges quickly to a local minimum solution.Comment: IEEE Transactions on Signal Processing (to appear
- …