118 research outputs found
Dictionary optimization for representing sparse signals using Rank-One Atom Decomposition (ROAD)
Dictionary learning has attracted growing research interest during recent years. As it is a bilinear inverse problem, one typical way to address this problem is to iteratively alternate between two stages: sparse coding and dictionary update. The general principle of the alternating approach is to fix one variable and optimize the other one. Unfortunately, for the alternating method, an ill-conditioned dictionary in the training process may not only introduce numerical instability but also trap the overall training process towards a singular point. Moreover, it leads to difficulty in analyzing its convergence, and few dictionary learning algorithms have been proved to have global convergence. For the other bilinear inverse problems, such as short-and-sparse deconvolution (SaSD) and convolutional dictionary learning (CDL), the alternating method is still a popular choice. As these bilinear inverse problems are also ill-posed and complicated, they are tricky to handle. Additional inner iterative methods are usually required for both of the updating stages, which aggravates the difficulty of analyzing the convergence of the whole learning process. It is also challenging to determine the number of iterations for each stage, as over-tuning any stage will trap the whole process into a local minimum that is far from the ground truth.
To mitigate the issues resulting from the alternating method, this thesis proposes a novel algorithm termed rank-one atom decomposition (ROAD), which intends to recast a bilinear inverse problem into an optimization problem with respect to a single variable, that is, a set of rank-one matrices. Therefore, the resulting algorithm is one stage, which minimizes the sparsity of the coefficients while keeping the data consistency constraint throughout the whole learning process. Inspired by recent advances in applying the alternating direction method of multipliers (ADMM) to nonconvex nonsmooth problems, an ADMM solver is adopted to address ROAD problems, and a lower bound of the penalty parameter is derived to guarantee a convergence in the augmented Lagrangian despite nonconvexity of the optimization formulation. Compared to two-stage dictionary learning methods, ROAD simplifies the learning process, eases the difficulty of analyzing convergence, and avoids the singular point issue. From a practical point of view, ROAD reduces the number of tuning parameters required in other benchmark algorithms. Numerical tests reveal that ROAD outperforms other benchmark algorithms in both synthetic data tests and single image super-resolution applications. In addition to dictionary learning, the ROAD formulation can also be extended to solve the SaSD and CDL problems. ROAD can still be employed to recast these problems into a one-variable optimization problem. Numerical tests illustrate that ROAD has better performance in estimating convolutional kernels compared to the latest SaSD and CDL algorithms.Open Acces
Fast Image Recovery Using Variable Splitting and Constrained Optimization
We propose a new fast algorithm for solving one of the standard formulations
of image restoration and reconstruction which consists of an unconstrained
optimization problem where the objective includes an data-fidelity
term and a non-smooth regularizer. This formulation allows both wavelet-based
(with orthogonal or frame-based representations) regularization or
total-variation regularization. Our approach is based on a variable splitting
to obtain an equivalent constrained optimization formulation, which is then
addressed with an augmented Lagrangian method. The proposed algorithm is an
instance of the so-called "alternating direction method of multipliers", for
which convergence has been proved. Experiments on a set of image restoration
and reconstruction benchmark problems show that the proposed algorithm is
faster than the current state of the art methods.Comment: Submitted; 11 pages, 7 figures, 6 table
An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems
We propose a new fast algorithm for solving one of the standard approaches to
ill-posed linear inverse problems (IPLIP), where a (possibly non-smooth)
regularizer is minimized under the constraint that the solution explains the
observations sufficiently well. Although the regularizer and constraint are
usually convex, several particular features of these problems (huge
dimensionality, non-smoothness) preclude the use of off-the-shelf optimization
tools and have stimulated a considerable amount of research. In this paper, we
propose a new efficient algorithm to handle one class of constrained problems
(often known as basis pursuit denoising) tailored to image recovery
applications. The proposed algorithm, which belongs to the family of augmented
Lagrangian methods, can be used to deal with a variety of imaging IPLIP,
including deconvolution and reconstruction from compressive observations (such
as MRI), using either total-variation or wavelet-based (or, more generally,
frame-based) regularization. The proposed algorithm is an instance of the
so-called "alternating direction method of multipliers", for which convergence
sufficient conditions are known; we show that these conditions are satisfied by
the proposed algorithm. Experiments on a set of image restoration and
reconstruction benchmark problems show that the proposed algorithm is a strong
contender for the state-of-the-art.Comment: 13 pages, 8 figure, 8 tables. Submitted to the IEEE Transactions on
Image Processin
A Tensor-Based Dictionary Learning Approach to Tomographic Image Reconstruction
We consider tomographic reconstruction using priors in the form of a
dictionary learned from training images. The reconstruction has two stages:
first we construct a tensor dictionary prior from our training data, and then
we pose the reconstruction problem in terms of recovering the expansion
coefficients in that dictionary. Our approach differs from past approaches in
that a) we use a third-order tensor representation for our images and b) we
recast the reconstruction problem using the tensor formulation. The dictionary
learning problem is presented as a non-negative tensor factorization problem
with sparsity constraints. The reconstruction problem is formulated in a convex
optimization framework by looking for a solution with a sparse representation
in the tensor dictionary. Numerical results show that our tensor formulation
leads to very sparse representations of both the training images and the
reconstructions due to the ability of representing repeated features compactly
in the dictionary.Comment: 29 page
A fast patch-dictionary method for whole image recovery
Various algorithms have been proposed for dictionary learning. Among those
for image processing, many use image patches to form dictionaries. This paper
focuses on whole-image recovery from corrupted linear measurements. We address
the open issue of representing an image by overlapping patches: the overlapping
leads to an excessive number of dictionary coefficients to determine. With very
few exceptions, this issue has limited the applications of image-patch methods
to the local kind of tasks such as denoising, inpainting, cartoon-texture
decomposition, super-resolution, and image deblurring, for which one can
process a few patches at a time. Our focus is global imaging tasks such as
compressive sensing and medical image recovery, where the whole image is
encoded together, making it either impossible or very ineffective to update a
few patches at a time.
Our strategy is to divide the sparse recovery into multiple subproblems, each
of which handles a subset of non-overlapping patches, and then the results of
the subproblems are averaged to yield the final recovery. This simple strategy
is surprisingly effective in terms of both quality and speed. In addition, we
accelerate computation of the learned dictionary by applying a recent block
proximal-gradient method, which not only has a lower per-iteration complexity
but also takes fewer iterations to converge, compared to the current
state-of-the-art. We also establish that our algorithm globally converges to a
stationary point. Numerical results on synthetic data demonstrate that our
algorithm can recover a more faithful dictionary than two state-of-the-art
methods.
Combining our whole-image recovery and dictionary-learning methods, we
numerically simulate image inpainting, compressive sensing recovery, and
deblurring. Our recovery is more faithful than those of a total variation
method and a method based on overlapping patches
- …