14 research outputs found
Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently
Multi-channel sparse blind deconvolution, or convolutional sparse coding,
refers to the problem of learning an unknown filter by observing its circulant
convolutions with multiple input signals that are sparse. This problem finds
numerous applications in signal processing, computer vision, and inverse
problems. However, it is challenging to learn the filter efficiently due to the
bilinear structure of the observations with the respect to the unknown filter
and inputs, as well as the sparsity constraint. In this paper, we propose a
novel approach based on nonconvex optimization over the sphere manifold by
minimizing a smooth surrogate of the sparsity-promoting loss function. It is
demonstrated that manifold gradient descent with random initializations will
provably recover the filter, up to scaling and shift ambiguity, as soon as the
number of observations is sufficiently large under an appropriate random data
model. Numerical experiments are provided to illustrate the performance of the
proposed method with comparisons to existing ones.Comment: accepted by IEEE Transactions on Information Theor
From Symmetry to Geometry: Tractable Nonconvex Problems
As science and engineering have become increasingly data-driven, the role of
optimization has expanded to touch almost every stage of the data analysis
pipeline, from the signal and data acquisition to modeling and prediction. The
optimization problems encountered in practice are often nonconvex. While
challenges vary from problem to problem, one common source of nonconvexity is
nonlinearity in the data or measurement model. Nonlinear models often exhibit
symmetries, creating complicated, nonconvex objective landscapes, with multiple
equivalent solutions. Nevertheless, simple methods (e.g., gradient descent)
often perform surprisingly well in practice.
The goal of this survey is to highlight a class of tractable nonconvex
problems, which can be understood through the lens of symmetries. These
problems exhibit a characteristic geometric structure: local minimizers are
symmetric copies of a single "ground truth" solution, while other critical
points occur at balanced superpositions of symmetric copies of the ground
truth, and exhibit negative curvature in directions that break the symmetry.
This structure enables efficient methods to obtain global minimizers. We
discuss examples of this phenomenon arising from a wide range of problems in
imaging, signal processing, and data analysis. We highlight the key role of
symmetry in shaping the objective landscape and discuss the different roles of
rotational and discrete symmetries. This area is rich with observed phenomena
and open problems; we close by highlighting directions for future research.Comment: review paper submitted to SIAM Review, 34 pages, 10 figure