6,601 research outputs found
Deep Learning Techniques for Inverse Problems in Imaging
Recent work in machine learning shows that deep neural networks can be used
to solve a wide variety of inverse problems arising in computational imaging.
We explore the central prevailing themes of this emerging area and present a
taxonomy that can be used to categorize different problems and reconstruction
methods. Our taxonomy is organized along two central axes: (1) whether or not a
forward model is known and to what extent it is used in training and testing,
and (2) whether or not the learning is supervised or unsupervised, i.e.,
whether or not the training relies on access to matched ground truth image and
measurement pairs. We also discuss the trade-offs associated with these
different reconstruction approaches, caveats and common failure modes, plus
open problems and avenues for future work
Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations
Inverse problems in imaging such as denoising, deblurring, superresolution
(SR) have been addressed for many decades. In recent years, convolutional
neural networks (CNNs) have been widely used for many inverse problem areas.
Although their indisputable success, CNNs are not mathematically validated as
to how and what they learn. In this paper, we prove that during training, CNN
elements solve for inverse problems which are optimum solutions stored as CNN
neuron filters. We discuss the necessity of mutual coherence between CNN layer
elements in order for a network to converge to the optimum solution. We prove
that required mutual coherence can be provided by the usage of residual
learning and skip connections. We have set rules over training sets and depth
of networks for better convergence, i.e. performance.Comment: PostPrint IET Signal Processing Journa
The Power of Complementary Regularizers: Image Recovery via Transform Learning and Low-Rank Modeling
Recent works on adaptive sparse and on low-rank signal modeling have
demonstrated their usefulness in various image / video processing applications.
Patch-based methods exploit local patch sparsity, whereas other works apply
low-rankness of grouped patches to exploit image non-local structures. However,
using either approach alone usually limits performance in image reconstruction
or recovery applications. In this work, we propose a simultaneous sparsity and
low-rank model, dubbed STROLLR, to better represent natural images. In order to
fully utilize both the local and non-local image properties, we develop an
image restoration framework using a transform learning scheme with joint
low-rank regularization. The approach owes some of its computational efficiency
and good performance to the use of transform learning for adaptive sparse
representation rather than the popular synthesis dictionary learning
algorithms, which involve approximation of NP-hard sparse coding and expensive
learning steps. We demonstrate the proposed framework in various applications
to image denoising, inpainting, and compressed sensing based magnetic resonance
imaging. Results show promising performance compared to state-of-the-art
competing methods.Comment: 13 pages, 7 figures, submitted to TI
Sparse synthesis regularization with deep neural networks
We propose a sparse reconstruction framework for solving inverse problems.
Opposed to existing sparse regularization techniques that are based on frame
representations, we train an encoder-decoder network by including an
-penalty. We demonstrate that the trained decoder network allows sparse
signal reconstruction using thresholded encoded coefficients without losing
much quality of the original image. Using the sparse synthesis prior, we
propose minimizing the -Tikhonov functional, which is the sum of a data
fitting term and the -norm of the synthesis coefficients, and show that
it provides a regularization method.Comment: Presented at the SAMPTA 2019 conferenc
Learning Personalized Representation for Inverse Problems in Medical Imaging Using Deep Neural Network
Recently deep neural networks have been widely and successfully applied in
computer vision tasks and attracted growing interests in medical imaging. One
barrier for the application of deep neural networks to medical imaging is the
need of large amounts of prior training pairs, which is not always feasible in
clinical practice. In this work we propose a personalized representation
learning framework where no prior training pairs are needed, but only the
patient's own prior images. The representation is expressed using a deep neural
network with the patient's prior images as network input. We then applied this
novel image representation to inverse problems in medical imaging in which the
original inverse problem was formulated as a constraint optimization problem
and solved using the alternating direction method of multipliers (ADMM)
algorithm. Anatomically guided brain positron emission tomography (PET) image
reconstruction and image denoising were employed as examples to demonstrate the
effectiveness of the proposed framework. Quantification results based on
simulation and real datasets show that the proposed personalized representation
framework outperform other widely adopted methods.Comment: 11 pages, 7 figure
prDeep: Robust Phase Retrieval with a Flexible Deep Network
Phase retrieval algorithms have become an important component in many modern
computational imaging systems. For instance, in the context of ptychography and
speckle correlation imaging, they enable imaging past the diffraction limit and
through scattering media, respectively. Unfortunately, traditional phase
retrieval algorithms struggle in the presence of noise. Progress has been made
recently on more robust algorithms using signal priors, but at the expense of
limiting the range of supported measurement models (e.g., to Gaussian or coded
diffraction patterns). In this work we leverage the regularization-by-denoising
framework and a convolutional neural network denoiser to create prDeep, a new
phase retrieval algorithm that is both robust and broadly applicable. We test
and validate prDeep in simulation to demonstrate that it is robust to noise and
can handle a variety of system models.
A MatConvNet implementation of prDeep is available at
https://github.com/ricedsp/prDeep
Sparse-View X-Ray CT Reconstruction Using Prior with Learned Transform
A major challenge in X-ray computed tomography (CT) is reducing radiation
dose while maintaining high quality of reconstructed images. To reduce the
radiation dose, one can reduce the number of projection views (sparse-view CT);
however, it becomes difficult to achieve high-quality image reconstruction as
the number of projection views decreases. Researchers have applied the concept
of learning sparse representations from (high-quality) CT image dataset to the
sparse-view CT reconstruction. We propose a new statistical CT reconstruction
model that combines penalized weighted-least squares (PWLS) and prior
with learned sparsifying transform (PWLS-ST-), and a corresponding
efficient algorithm based on Alternating Direction Method of Multipliers
(ADMM). To moderate the difficulty of tuning ADMM parameters, we propose a new
ADMM parameter selection scheme based on approximated condition numbers. We
interpret the proposed model by analyzing the minimum mean square error of its
(-norm relaxed) image update estimator. Our results with the extended
cardiac-torso (XCAT) phantom data and clinical chest data show that, for
sparse-view 2D fan-beam CT and 3D axial cone-beam CT, PWLS-ST- improves
the quality of reconstructed images compared to the CT reconstruction methods
using edge-preserving regularizer and prior with learned ST. These
results also show that, for sparse-view 2D fan-beam CT, PWLS-ST-
achieves comparable or better image quality and requires much shorter runtime
than PWLS-DL using a learned overcomplete dictionary. Our results with clinical
chest data show that, methods using the unsupervised learned prior generalize
better than a state-of-the-art deep "denoising" neural network that does not
use a physical imaging model.Comment: The first two authors contributed equally to this wor
Deep MR Fingerprinting with total-variation and low-rank subspace priors
Deep learning (DL) has recently emerged to address the heavy storage and
computation requirements of the baseline dictionary-matching (DM) for Magnetic
Resonance Fingerprinting (MRF) reconstruction. Fed with non-iterated
back-projected images, the network is unable to fully resolve
spatially-correlated corruptions caused from the undersampling artefacts. We
propose an accelerated iterative reconstruction to minimize these artefacts
before feeding into the network. This is done through a convex regularization
that jointly promotes spatio-temporal regularities of the MRF time-series.
Except for training, the rest of the parameter estimation pipeline is
dictionary-free. We validate the proposed approach on synthetic and in-vivo
datasets
Greedy Deep Dictionary Learning
In this work we propose a new deep learning tool called deep dictionary
learning. Multi-level dictionaries are learnt in a greedy fashion, one layer at
a time. This requires solving a simple (shallow) dictionary learning problem,
the solution to this is well known. We apply the proposed technique on some
benchmark deep learning datasets. We compare our results with other deep
learning tools like stacked autoencoder and deep belief network; and state of
the art supervised dictionary learning tools like discriminative KSVD and label
consistent KSVD. Our method yields better results than all
LSALSA: Accelerated Source Separation via Learned Sparse Coding
We propose an efficient algorithm for the generalized sparse coding (SC)
inference problem. The proposed framework applies to both the single dictionary
setting, where each data point is represented as a sparse combination of the
columns of one dictionary matrix, as well as the multiple dictionary setting as
given in morphological component analysis (MCA), where the goal is to separate
a signal into additive parts such that each part has distinct sparse
representation within a corresponding dictionary. Both the SC task and its
generalization via MCA have been cast as -regularized least-squares
optimization problems. To accelerate traditional acquisition of sparse codes,
we propose a deep learning architecture that constitutes a trainable
time-unfolded version of the Split Augmented Lagrangian Shrinkage Algorithm
(SALSA), a special case of the Alternating Direction Method of Multipliers
(ADMM). We empirically validate both variants of the algorithm, that we refer
to as LSALSA (learned-SALSA), on image vision tasks and demonstrate that at
inference our networks achieve vast improvements in terms of the running time,
the quality of estimated sparse codes, and visual clarity on both classic SC
and MCA problems. Finally, we present a theoretical framework for analyzing
LSALSA network: we show that the proposed approach exactly implements a
truncated ADMM applied to a new, learned cost function with curvature modified
by one of the learned parameterized matrices. We extend a very recent
Stochastic Alternating Optimization analysis framework to show that a gradient
descent step along this learned loss landscape is equivalent to a modified
gradient descent step along the original loss landscape. In this framework, the
acceleration achieved by LSALSA could potentially be explained by the network's
ability to learn a correction to the gradient direction of steeper descent.Comment: ECML-PKDD 2019 via journal track; Special Issue Mach Learn (2019
- …