5,299 research outputs found
Conjugate gradient acceleration of iteratively re-weighted least squares methods
Iteratively Re-weighted Least Squares (IRLS) is a method for solving
minimization problems involving non-quadratic cost functions, perhaps
non-convex and non-smooth, which however can be described as the infimum over a
family of quadratic functions. This transformation suggests an algorithmic
scheme that solves a sequence of quadratic problems to be tackled efficiently
by tools of numerical linear algebra. Its general scope and its usually simple
implementation, transforming the initial non-convex and non-smooth minimization
problem into a more familiar and easily solvable quadratic optimization
problem, make it a versatile algorithm. However, despite its simplicity,
versatility, and elegant analysis, the complexity of IRLS strongly depends on
the way the solution of the successive quadratic optimizations is addressed.
For the important special case of and sparse
recovery problems in signal processing, we investigate theoretically and
numerically how accurately one needs to solve the quadratic problems by means
of the (CG) method in each iteration in order to
guarantee convergence. The use of the CG method may significantly speed-up the
numerical solution of the quadratic subproblems, in particular, when fast
matrix-vector multiplication (exploiting for instance the FFT) is available for
the matrix involved. In addition, we study convergence rates. Our modified IRLS
method outperforms state of the art first order methods such as Iterative Hard
Thresholding (IHT) or Fast Iterative Soft-Thresholding Algorithm (FISTA) in
many situations, especially in large dimensions. Moreover, IRLS is often able
to recover sparse vectors from fewer measurements than required for IHT and
FISTA.Comment: 40 page
Proof of Convergence and Performance Analysis for Sparse Recovery via Zero-point Attracting Projection
A recursive algorithm named Zero-point Attracting Projection (ZAP) is
proposed recently for sparse signal reconstruction. Compared with the reference
algorithms, ZAP demonstrates rather good performance in recovery precision and
robustness. However, any theoretical analysis about the mentioned algorithm,
even a proof on its convergence, is not available. In this work, a strict proof
on the convergence of ZAP is provided and the condition of convergence is put
forward. Based on the theoretical analysis, it is further proved that ZAP is
non-biased and can approach the sparse solution to any extent, with the proper
choice of step-size. Furthermore, the case of inaccurate measurements in noisy
scenario is also discussed. It is proved that disturbance power linearly
reduces the recovery precision, which is predictable but not preventable. The
reconstruction deviation of -compressible signal is also provided. Finally,
numerical simulations are performed to verify the theoretical analysis.Comment: 29 pages, 6 figure
Relevance Singular Vector Machine for low-rank matrix sensing
In this paper we develop a new Bayesian inference method for low rank matrix
reconstruction. We call the new method the Relevance Singular Vector Machine
(RSVM) where appropriate priors are defined on the singular vectors of the
underlying matrix to promote low rank. To accelerate computations, a
numerically efficient approximation is developed. The proposed algorithms are
applied to matrix completion and matrix reconstruction problems and their
performance is studied numerically.Comment: International Conference on Signal Processing and Communications
(SPCOM), 5 page
- …