59 research outputs found
Medical image reconstruction: a brief overview of past milestones and future directions
This paper briefly reviews past milestones in the field of medical image
reconstruction and describes some future directions. It is part of an overview
paper on "open problems in signal processing" that will appear in IEEE Signal
Processing Magazine, but presented here with citations and equations.Comment: Part of a submission to IEEE Signal Processing Magazin
Fidelity Imposed Network Edit (FINE) for Solving Ill-Posed Image Reconstruction
Deep learning (DL) is increasingly used to solve ill-posed inverse problems
in imaging, such as reconstruction from noisy or incomplete data, as DL offers
advantages over explicit image feature extractions in defining the needed
prior. However, DL typically does not incorporate the precise physics of data
generation or data fidelity. Instead, DL networks are trained to output some
average response to an input. Consequently, DL image reconstruction contains
errors, and may perform poorly when the test data deviates significantly from
the training data, such as having new pathological features. To address this
lack of data fidelity problem in DL image reconstruction, a novel approach,
which we call fidelity-imposed network edit (FINE), is proposed. In FINE, a
pre-trained prior network's weights are modified according to the physical
model, on a test case. Our experiments demonstrate that FINE can achieve
superior performance in two important inverse problems in neuroimaging:
quantitative susceptibility mapping (QSM) and under-sampled reconstruction in
MRI
Self-Supervised Deep Active Accelerated MRI
We propose to simultaneously learn to sample and reconstruct magnetic
resonance images (MRI) to maximize the reconstruction quality given a limited
sample budget, in a self-supervised setup. Unlike existing deep methods that
focus only on reconstructing given data, thus being passive, we go beyond the
current state of the art by considering both the data acquisition and the
reconstruction process within a single deep-learning framework. As our network
learns to acquire data, the network is active in nature. In order to do so, we
simultaneously train two neural networks, one dedicated to reconstruction and
the other to progressive sampling, each with an automatically generated
supervision signal that links them together. The two supervision signals are
created through Monte Carlo tree search (MCTS). MCTS returns a better sampling
pattern than what the current sampling network can give and, thus, a better
final reconstruction. The sampling network is trained to mimic the MCTS results
using the previous sampling network, thus being enhanced. The reconstruction
network is trained to give the highest reconstruction quality, given the MCTS
sampling pattern. Through this framework, we are able to train the two networks
without providing any direct supervision on sampling
RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
Regularization by denoising (RED) is an image reconstruction framework that
uses an image denoiser as a prior. Recent work has shown the state-of-the-art
performance of RED with learned denoisers corresponding to pre-trained
convolutional neural nets (CNNs). In this work, we propose to broaden the
current denoiser-centric view of RED by considering priors corresponding to
networks trained for more general artifact-removal. The key benefit of the
proposed family of algorithms, called regularization by artifact-removal
(RARE), is that it can leverage priors learned on datasets containing only
undersampled measurements. This makes RARE applicable to problems where it is
practically impossible to have fully-sampled groundtruth data for training. We
validate RARE on both simulated and experimentally collected data by
reconstructing a free-breathing whole-body 3D MRIs into ten respiratory phases
from heavily undersampled k-space measurements. Our results corroborate the
potential of learning regularizers for iterative inversion directly on
undersampled and noisy measurements.Comment: In press for IEEE Journal of Special Topics in Signal Processin
MoDL: Model Based Deep Learning Architecture for Inverse Problems
We introduce a model-based image reconstruction framework with a convolution
neural network (CNN) based regularization prior. The proposed formulation
provides a systematic approach for deriving deep architectures for inverse
problems with the arbitrary structure. Since the forward model is explicitly
accounted for, a smaller network with fewer parameters is sufficient to capture
the image information compared to black-box deep learning approaches, thus
reducing the demand for training data and training time. Since we rely on
end-to-end training, the CNN weights are customized to the forward model, thus
offering improved performance over approaches that rely on pre-trained
denoisers. The main difference of the framework from existing end-to-end
training strategies is the sharing of the network weights across iterations and
channels. Our experiments show that the decoupling of the number of iterations
from the network complexity offered by this approach provides benefits
including lower demand for training data, reduced risk of overfitting, and
implementations with significantly reduced memory footprint. We propose to
enforce data-consistency by using numerical optimization blocks such as
conjugate gradients algorithm within the network; this approach offers faster
convergence per iteration, compared to methods that rely on proximal gradients
steps to enforce data consistency. Our experiments show that the faster
convergence translates to improved performance, especially when the available
GPU memory restricts the number of iterations.Comment: published in IEEE Transaction on Medical Imagin
Optimal Sampling & Reconstruction: Theory and Applications
The optimization of MRI data sampling and image reconstruction methods has
been a priority for the MRI community since the very early days of the field.
Designing an "optimal" method requires the definition of an optimality metric
(i.e., a quantitative evaluation of the "goodness" of different competing
approaches that allows an objective comparison between them). However, a key
challenge is that there are many different possible ways of quantitatively
evaluating the "goodness" of a data sampling scheme or a reconstruction result,
and there are no acquisition or reconstruction methods that are known to be
universally optimal with respect to all of these possible metrics
simultaneously. Thus, optimization of MRI methods requires a subjective choice
about what aspects of quality matter most in the context of a given MRI
experiment, and subsequently the subjective choice of an optimality metric that
hopefully does a reasonable job of quantifying those aspects of quality. Once
these choices are made, the optimization problem becomes well-defined, and it
remains to choose an algorithm that can identify data sampling or image
reconstruction methods that are optimal with respect to the chosen metric. All
of these choices are generally nontrivial.
In this presentation, we will discuss optimal sampling and reconstruction
designs from multiple different perspectives, including ideas from information
and estimation theory and various practical perspectives.Comment: Syllabus material for an invited talk at the 2020 ISMRM Workshop on
Data Sampling & Image Reconstructio
Plug and play methods for magnetic resonance imaging (long version)
Magnetic Resonance Imaging (MRI) is a non-invasive diagnostic tool that
provides excellent soft-tissue contrast without the use of ionizing radiation.
Compared to other clinical imaging modalities (e.g., CT or ultrasound),
however, the data acquisition process for MRI is inherently slow, which
motivates undersampling and thus drives the need for accurate, efficient
reconstruction methods from undersampled datasets. In this article, we describe
the use of "plug-and-play" (PnP) algorithms for MRI image recovery. We first
describe the linearly approximated inverse problem encountered in MRI. Then we
review several PnP methods, where the unifying commonality is to iteratively
call a denoising subroutine as one step of a larger optimization-inspired
algorithm. Next, we describe how the result of the PnP method can be
interpreted as a solution to an equilibrium equation, allowing convergence
analysis from the equilibrium perspective. Finally, we present illustrative
examples of PnP methods applied to MRI image recovery
A Gaussian Mixture MRF for Model-Based Iterative Reconstruction with Applications to Low-Dose X-ray CT
Markov random fields (MRFs) have been widely used as prior models in various
inverse problems such as tomographic reconstruction. While MRFs provide a
simple and often effective way to model the spatial dependencies in images,
they suffer from the fact that parameter estimation is difficult. In practice,
this means that MRFs typically have very simple structure that cannot
completely capture the subtle characteristics of complex images.
In this paper, we present a novel Gaussian mixture Markov random field model
(GM-MRF) that can be used as a very expressive prior model for inverse problems
such as denoising and reconstruction. The GM-MRF forms a global image model by
merging together individual Gaussian-mixture models (GMMs) for image patches.
In addition, we present a novel analytical framework for computing MAP
estimates using the GM-MRF prior model through the construction of surrogate
functions that result in a sequence of quadratic optimizations. We also
introduce a simple but effective method to adjust the GM-MRF so as to control
the sharpness in low- and high-contrast regions of the reconstruction
separately. We demonstrate the value of the model with experiments including
image denoising and low-dose CT reconstruction.Comment: accepted by IEEE Transactions on Computed Imagin
Least Squares Optimal Density Compensation for the Gridding Non-uniform Discrete Fourier Transform
The Gridding algorithm has shown great utility for reconstructing images from
non-uniformly spaced samples in the Fourier domain in several imaging
modalities. Due to the non-uniform spacing, some correction for the variable
density of the samples must be made. Existing methods for generating density
compensation values are either sub-optimal or only consider a finite set of
points (a set of measure 0) in the optimization. This manuscript presents the
first density compensation algorithm for a general trajectory that takes into
account the point spread function over a set of non-zero measure. We show that
the images reconstructed with Gridding using the density compensation values of
this method are of superior quality when compared to density compensation
weights determined in other ways. Results are shown with a numerical phantom
and with magnetic resonance images of the abdomen and the knee
Deep Learning Techniques for Inverse Problems in Imaging
Recent work in machine learning shows that deep neural networks can be used
to solve a wide variety of inverse problems arising in computational imaging.
We explore the central prevailing themes of this emerging area and present a
taxonomy that can be used to categorize different problems and reconstruction
methods. Our taxonomy is organized along two central axes: (1) whether or not a
forward model is known and to what extent it is used in training and testing,
and (2) whether or not the learning is supervised or unsupervised, i.e.,
whether or not the training relies on access to matched ground truth image and
measurement pairs. We also discuss the trade-offs associated with these
different reconstruction approaches, caveats and common failure modes, plus
open problems and avenues for future work
- …