534 research outputs found
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk
We examine the theoretical properties of enforcing priors provided by
generative deep neural networks via empirical risk minimization. In particular
we consider two models, one in which the task is to invert a generative neural
network given access to its last layer and another in which the task is to
invert a generative neural network given only compressive linear observations
of its last layer. We establish that in both cases, in suitable regimes of
network layer sizes and a randomness assumption on the network weights, that
the non-convex objective function given by empirical risk minimization does not
have any spurious stationary points. That is, we establish that with high
probability, at any point away from small neighborhoods around two scalar
multiples of the desired solution, there is a descent direction. Hence, there
are no local minima, saddle points, or other stationary points outside these
neighborhoods. These results constitute the first theoretical guarantees which
establish the favorable global geometry of these non-convex optimization
problems, and they bridge the gap between the empirical success of enforcing
deep generative priors and a rigorous understanding of non-linear inverse
problems.Comment: Accepted for presentation at Conference on Learning Theory (COLT)
201
Deep Learning Techniques for Inverse Problems in Imaging
Recent work in machine learning shows that deep neural networks can be used
to solve a wide variety of inverse problems arising in computational imaging.
We explore the central prevailing themes of this emerging area and present a
taxonomy that can be used to categorize different problems and reconstruction
methods. Our taxonomy is organized along two central axes: (1) whether or not a
forward model is known and to what extent it is used in training and testing,
and (2) whether or not the learning is supervised or unsupervised, i.e.,
whether or not the training relies on access to matched ground truth image and
measurement pairs. We also discuss the trade-offs associated with these
different reconstruction approaches, caveats and common failure modes, plus
open problems and avenues for future work
Deep De-Aliasing for Fast Compressive Sensing MRI
Fast Magnetic Resonance Imaging (MRI) is highly in demand for many clinical
applications in order to reduce the scanning cost and improve the patient
experience. This can also potentially increase the image quality by reducing
the motion artefacts and contrast washout. However, once an image field of view
and the desired resolution are chosen, the minimum scanning time is normally
determined by the requirement of acquiring sufficient raw data to meet the
Nyquist-Shannon sampling criteria. Compressive Sensing (CS) theory has been
perfectly matched to the MRI scanning sequence design with much less required
raw data for the image reconstruction. Inspired by recent advances in deep
learning for solving various inverse problems, we propose a conditional
Generative Adversarial Networks-based deep learning framework for de-aliasing
and reconstructing MRI images from highly undersampled data with great promise
to accelerate the data acquisition process. By coupling an innovative content
loss with the adversarial loss our de-aliasing results are more realistic.
Furthermore, we propose a refinement learning procedure for training the
generator network, which can stabilise the training with fast convergence and
less parameter tuning. We demonstrate that the proposed framework outperforms
state-of-the-art CS-MRI methods, in terms of reconstruction error and
perceptual image quality. In addition, our method can reconstruct each image in
0.22ms--0.37ms, which is promising for real-time applications.Comment: 15 pages, 5 figure
Compressed Sensing with Deep Image Prior and Learned Regularization
We propose a novel method for compressed sensing recovery using untrained
deep generative models. Our method is based on the recently proposed Deep Image
Prior (DIP), wherein the convolutional weights of the network are optimized to
match the observed measurements. We show that this approach can be applied to
solve any differentiable linear inverse problem, outperforming previous
unlearned methods. Unlike various learned approaches based on generative
models, our method does not require pre-training over large datasets. We
further introduce a novel learned regularization technique, which incorporates
prior information on the network weights. This reduces reconstruction error,
especially for noisy measurements. Finally, we prove that, using the DIP
optimization approach, moderately overparameterized single-layer networks can
perfectly fit any signal despite the non-convex nature of the fitting problem.
This theoretical result provides justification for early stopping
Deep Compressed Sensing
Compressed sensing (CS) provides an elegant framework for recovering sparse
signals from compressed measurements. For example, CS can exploit the structure
of natural images and recover an image from only a few random measurements. CS
is flexible and data efficient, but its application has been restricted by the
strong assumption of sparsity and costly reconstruction process. A recent
approach that combines CS with neural network generators has removed the
constraint of sparsity, but reconstruction remains slow. Here we propose a
novel framework that significantly improves both the performance and speed of
signal recovery by jointly training a generator and the optimisation process
for reconstruction via meta-learning. We explore training the measurements with
different objectives, and derive a family of models based on minimising
measurement errors. We show that Generative Adversarial Nets (GANs) can be
viewed as a special case in this family of models. Borrowing insights from the
CS perspective, we develop a novel way of improving GANs using gradient
information from the discriminator.Comment: ICML 201
A Transfer-Learning Approach for Accelerated MRI using Deep Neural Networks
Purpose: Neural networks have received recent interest for reconstruction of
undersampled MR acquisitions. Ideally network performance should be optimized
by drawing the training and testing data from the same domain. In practice,
however, large datasets comprising hundreds of subjects scanned under a common
protocol are rare. The goal of this study is to introduce a transfer-learning
approach to address the problem of data scarcity in training deep networks for
accelerated MRI.
Methods: Neural networks were trained on thousands of samples from public
datasets of either natural images or brain MR images. The networks were then
fine-tuned using only few tens of brain MR images in a distinct testing domain.
Domain-transferred networks were compared to networks trained directly in the
testing domain. Network performance was evaluated for varying acceleration
factors (2-10), number of training samples (0.5-4k) and number of fine-tuning
samples (0-100).
Results: The proposed approach achieves successful domain transfer between MR
images acquired with different contrasts (T1- and T2-weighted images), and
between natural and MR images (ImageNet and T1- or T2-weighted images).
Networks obtained via transfer-learning using only tens of images in the
testing domain achieve nearly identical performance to networks trained
directly in the testing domain using thousands of images.
Conclusion: The proposed approach might facilitate the use of neural networks
for MRI reconstruction without the need for collection of extensive imaging
datasets
Recurrent Generative Adversarial Networks for Proximal Learning and Automated Compressive Image Recovery
Recovering images from undersampled linear measurements typically leads to an
ill-posed linear inverse problem, that asks for proper statistical priors.
Building effective priors is however challenged by the low train and test
overhead dictated by real-time tasks; and the need for retrieving visually
"plausible" and physically "feasible" images with minimal hallucination. To
cope with these challenges, we design a cascaded network architecture that
unrolls the proximal gradient iterations by permeating benefits from generative
residual networks (ResNet) to modeling the proximal operator. A mixture of
pixel-wise and perceptual costs is then deployed to train proximals. The
overall architecture resembles back-and-forth projection onto the intersection
of feasible and plausible images. Extensive computational experiments are
examined for a global task of reconstructing MR images of pediatric patients,
and a more local task of superresolving CelebA faces, that are insightful to
design efficient architectures. Our observations indicate that for MRI
reconstruction, a recurrent ResNet with a single residual block effectively
learns the proximal. This simple architecture appears to significantly
outperform the alternative deep ResNet architecture by 2dB SNR, and the
conventional compressed-sensing MRI by 4dB SNR with 100x faster inference. For
image superresolution, our preliminary results indicate that modeling the
denoising proximal demands deep ResNets.Comment: 11 pages, 11 figure
Medical Image Generation using Generative Adversarial Networks
Generative adversarial networks (GANs) are unsupervised Deep Learning
approach in the computer vision community which has gained significant
attention from the last few years in identifying the internal structure of
multimodal medical imaging data. The adversarial network simultaneously
generates realistic medical images and corresponding annotations, which proven
to be useful in many cases such as image augmentation, image registration,
medical image generation, image reconstruction, and image-to-image translation.
These properties bring the attention of the researcher in the field of medical
image analysis and we are witness of rapid adaption in many novel and
traditional applications. This chapter provides state-of-the-art progress in
GANs-based clinical application in medical image generation, and cross-modality
synthesis. The various framework of GANs which gained popularity in the
interpretation of medical images, such as Deep Convolutional GAN (DCGAN),
Laplacian GAN (LAPGAN), pix2pix, CycleGAN, and unsupervised image-to-image
translation model (UNIT), continue to improve their performance by
incorporating additional hybrid architecture, has been discussed. Further, some
of the recent applications of these frameworks for image reconstruction, and
synthesis, and future research directions in the area have been covered.Comment: 19 pages, 3 figures, 5 table
Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction
Deep learning approaches have shown promising performance for compressed
sensing-based Magnetic Resonance Imaging. While deep neural networks trained
with mean squared error (MSE) loss functions can achieve high peak signal to
noise ratio, the reconstructed images are often blurry and lack sharp details,
especially for higher undersampling rates. Recently, adversarial and perceptual
loss functions have been shown to achieve more visually appealing results.
However, it remains an open question how to (1) optimally combine these loss
functions with the MSE loss function and (2) evaluate such a perceptual
enhancement. In this work, we propose a hybrid method, in which a visual
refinement component is learnt on top of an MSE loss-based reconstruction
network. In addition, we introduce a semantic interpretability score, measuring
the visibility of the region of interest in both ground truth and reconstructed
images, which allows us to objectively quantify the usefulness of the image
quality for image post-processing and analysis. Applied on a large cardiac MRI
dataset simulated with 8-fold undersampling, we demonstrate significant
improvements () over the state-of-the-art in both a human observer
study and the semantic interpretability score.Comment: To be published at MICCAI 201
Analysis of Deep Complex-Valued Convolutional Neural Networks for MRI Reconstruction
Many real-world signal sources are complex-valued, having real and imaginary
components. However, the vast majority of existing deep learning platforms and
network architectures do not support the use of complex-valued data. MRI data
is inherently complex-valued, so existing approaches discard the richer
algebraic structure of the complex data. In this work, we investigate
end-to-end complex-valued convolutional neural networks - specifically, for
image reconstruction in lieu of two-channel real-valued networks. We apply this
to magnetic resonance imaging reconstruction for the purpose of accelerating
scan times and determine the performance of various promising complex-valued
activation functions. We find that complex-valued CNNs with complex-valued
convolutions provide superior reconstructions compared to real-valued
convolutions with the same number of trainable parameters, over a variety of
network architectures and datasets
- …