4,061 research outputs found
Efficient Two-Dimensional Line Spectrum Estimation Based on Decoupled Atomic Norm Minimization
This paper presents an efficient optimization technique for gridless {2-D}
line spectrum estimation, named decoupled atomic norm minimization (D-ANM). The
framework of atomic norm minimization (ANM) is considered, which has been
successfully applied in 1-D problems to allow super-resolution frequency
estimation for correlated sources even when the number of snapshots is highly
limited. The state-of-the-art 2-D ANM approach vectorizes the 2-D measurements
to their 1-D equivalence, which incurs huge computational cost and may become
too costly for practical applications. We develop a novel decoupled approach of
2-D ANM via semi-definite programming (SDP), which introduces a new matrix-form
atom set to naturally decouple the joint observations in both dimensions
without loss of optimality. Accordingly, the original large-scale 2-D problem
is equivalently reformulated via two decoupled one-level Toeplitz matrices,
which can be solved by simple 1-D frequency estimation with pairing. Compared
with the conventional vectorized approach, the proposed D-ANM technique reduces
the computational complexity by several orders of magnitude with respect to the
problem size. It also retains the benefits of ANM in terms of precise signal
recovery, small number of required measurements, and robustness to source
correlation. The complexity benefits are particularly attractive for
large-scale antenna systems such as massive MIMO, radar signal processing and
radio astronomy
Vandermonde Decomposition of Multilevel Toeplitz Matrices with Application to Multidimensional Super-Resolution
The Vandermonde decomposition of Toeplitz matrices, discovered by
Carath\'{e}odory and Fej\'{e}r in the 1910s and rediscovered by Pisarenko in
the 1970s, forms the basis of modern subspace methods for 1D frequency
estimation. Many related numerical tools have also been developed for
multidimensional (MD), especially 2D, frequency estimation; however, a
fundamental question has remained unresolved as to whether an analog of the
Vandermonde decomposition holds for multilevel Toeplitz matrices in the MD
case. In this paper, an affirmative answer to this question and a constructive
method for finding the decomposition are provided when the matrix rank is lower
than the dimension of each Toeplitz block. A numerical method for searching for
a decomposition is also proposed when the matrix rank is higher. The new
results are applied to studying MD frequency estimation within the recent
super-resolution framework. A precise formulation of the atomic norm
is derived using the Vandermonde decomposition. Practical algorithms for
frequency estimation are proposed based on relaxation techniques. Extensive
numerical simulations are provided to demonstrate the effectiveness of these
algorithms compared to the existing atomic norm and subspace methods.Comment: 17 pages, double column, 5 figures, to appear in IEEE Transactions on
Information Theor
Harnessing Sparsity over the Continuum: Atomic Norm Minimization for Super Resolution
Convex optimization recently emerges as a compelling framework for performing
super resolution, garnering significant attention from multiple communities
spanning signal processing, applied mathematics, and optimization. This article
offers a friendly exposition to atomic norm minimization as a canonical convex
approach to solve super resolution problems. The mathematical foundations and
performances guarantees of this approach are presented, and its application in
super resolution image reconstruction for single-molecule fluorescence
microscopy are highlighted
A Geometric Approach to Low-Rank Matrix Completion
The low-rank matrix completion problem can be succinctly stated as follows:
given a subset of the entries of a matrix, find a low-rank matrix consistent
with the observations. While several low-complexity algorithms for matrix
completion have been proposed so far, it remains an open problem to devise
search procedures with provable performance guarantees for a broad class of
matrix models. The standard approach to the problem, which involves the
minimization of an objective function defined using the Frobenius metric, has
inherent difficulties: the objective function is not continuous and the
solution set is not closed. To address this problem, we consider an
optimization procedure that searches for a column (or row) space that is
geometrically consistent with the partial observations. The geometric objective
function is continuous everywhere and the solution set is the closure of the
solution set of the Frobenius metric. We also preclude the existence of local
minimizers, and hence establish strong performance guarantees, for special
completion scenarios, which do not require matrix incoherence or large matrix
size.Comment: 10 pages, 2 figure
Separation-Free Super-Resolution from Compressed Measurements is Possible: an Orthonormal Atomic Norm Minimization Approach
We consider the problem of recovering the superposition of distinct
complex exponential functions from compressed non-uniform time-domain samples.
Total Variation (TV) minimization or atomic norm minimization was proposed in
the literature to recover the frequencies or the missing data. However, it
is known that in order for TV minimization and atomic norm minimization to
recover the missing data or the frequencies, the underlying frequencies are
required to be well-separated, even when the measurements are noiseless. This
paper shows that the Hankel matrix recovery approach can super-resolve the
complex exponentials and their frequencies from compressed non-uniform
measurements, regardless of how close their frequencies are to each other. We
propose a new concept of orthonormal atomic norm minimization (OANM), and
demonstrate that the success of Hankel matrix recovery in separation-free
super-resolution comes from the fact that the nuclear norm of a Hankel matrix
is an orthonormal atomic norm. More specifically, we show that, in traditional
atomic norm minimization, the underlying parameter values be
well separated to achieve successful signal recovery, if the atoms are changing
continuously with respect to the continuously-valued parameter. In contrast,
for the OANM, it is possible the OANM is successful even though the original
atoms can be arbitrarily close.
As a byproduct of this research, we provide one matrix-theoretic inequality
of nuclear norm, and give its proof from the theory of compressed sensing.Comment: 39 page
Stable recovery of low-dimensional cones in Hilbert spaces: One RIP to rule them all
Many inverse problems in signal processing deal with the robust estimation of
unknown data from underdetermined linear observations. Low dimensional models,
when combined with appropriate regularizers, have been shown to be efficient at
performing this task. Sparse models with the 1-norm or low rank models with the
nuclear norm are examples of such successful combinations. Stable recovery
guarantees in these settings have been established using a common tool adapted
to each case: the notion of restricted isometry property (RIP). In this paper,
we establish generic RIP-based guarantees for the stable recovery of cones
(positively homogeneous model sets) with arbitrary regularizers. These
guarantees are illustrated on selected examples. For block structured sparsity
in the infinite dimensional setting, we use the guarantees for a family of
regularizers which efficiency in terms of RIP constant can be controlled,
leading to stronger and sharper guarantees than the state of the art.Comment: in Applied and Computational Harmonic Analysis, Elsevier, 201
A Super-Resolution Framework for Tensor Decomposition
This work considers a super-resolution framework for overcomplete tensor
decomposition. Specifically, we view tensor decomposition as a super-resolution
problem of recovering a sum of Dirac measures on the sphere and solve it by
minimizing a continuous analog of the norm on the space of measures.
The optimal value of this optimization defines the tensor nuclear norm. Similar
to the separation condition in the super-resolution problem, by explicitly
constructing a dual certificate, we develop incoherence conditions of the
tensor factors so that they form the unique optimal solution of the continuous
analog of norm minimization. Remarkably, the derived incoherence
conditions are satisfied with high probability by random tensor factors
uniformly distributed on the sphere, implying global identifiability of random
tensor factors
Spectral Compressed Sensing via CANDECOMP/PARAFAC Decomposition of Incomplete Tensors
We consider the line spectral estimation problem which aims to recover a
mixture of complex sinusoids from a small number of randomly observed time
domain samples. Compressed sensing methods formulates line spectral estimation
as a sparse signal recovery problem by discretizing the continuous frequency
parameter space into a finite set of grid points. Discretization, however,
inevitably incurs errors and leads to deteriorated estimation performance. In
this paper, we propose a new method which leverages recent advances in tensor
decomposition. Specifically, we organize the observed data into a structured
tensor and cast line spectral estimation as a CANDECOMP/PARAFAC (CP)
decomposition problem with missing entries. The uniqueness of the CP
decomposition allows the frequency components to be super-resolved with
infinite precision. Simulation results show that the proposed method provides a
competitive estimate accuracy compared with existing state-of-the-art
algorithms
Orthogonal Rank-One Matrix Pursuit for Low Rank Matrix Completion
In this paper, we propose an efficient and scalable low rank matrix
completion algorithm. The key idea is to extend orthogonal matching pursuit
method from the vector case to the matrix case. We further propose an economic
version of our algorithm by introducing a novel weight updating rule to reduce
the time and storage complexity. Both versions are computationally inexpensive
for each matrix pursuit iteration, and find satisfactory results in a few
iterations. Another advantage of our proposed algorithm is that it has only one
tunable parameter, which is the rank. It is easy to understand and to use by
the user. This becomes especially important in large-scale learning problems.
In addition, we rigorously show that both versions achieve a linear convergence
rate, which is significantly better than the previous known results. We also
empirically compare the proposed algorithms with several state-of-the-art
matrix completion algorithms on many real-world datasets, including the
large-scale recommendation dataset Netflix as well as the MovieLens datasets.
Numerical results show that our proposed algorithm is more efficient than
competing algorithms while achieving similar or better prediction performance
Forward - Backward Greedy Algorithms for Atomic Norm Regularization
In many signal processing applications, the aim is to reconstruct a signal
that has a simple representation with respect to a certain basis or frame.
Fundamental elements of the basis known as "atoms" allow us to define "atomic
norms" that can be used to formulate convex regularizations for the
reconstruction problem. Efficient algorithms are available to solve these
formulations in certain special cases, but an approach that works well for
general atomic norms, both in terms of speed and reconstruction accuracy,
remains to be found. This paper describes an optimization algorithm called
CoGEnT that produces solutions with succinct atomic representations for
reconstruction problems, generally formulated with atomic-norm constraints.
CoGEnT combines a greedy selection scheme based on the conditional gradient
approach with a backward (or "truncation") step that exploits the quadratic
nature of the objective to reduce the basis size. We establish convergence
properties and validate the algorithm via extensive numerical experiments on a
suite of signal processing applications. Our algorithm and analysis also allow
for inexact forward steps and for occasional enhancements of the current
representation to be performed. CoGEnT can outperform the basic conditional
gradient method, and indeed many methods that are tailored to specific
applications, when the enhancement and truncation steps are defined
appropriately. We also introduce several novel applications that are enabled by
the atomic-norm framework, including tensor completion, moment problems in
signal processing, and graph deconvolution.Comment: To appear in IEEE Transactions on Signal Processin
- …