22 research outputs found
PND-Net: Physics based Non-local Dual-domain Network for Metal Artifact Reduction
Metal artifacts caused by the presence of metallic implants tremendously
degrade the reconstructed computed tomography (CT) image quality, affecting
clinical diagnosis or reducing the accuracy of organ delineation and dose
calculation in radiotherapy. Recently, deep learning methods in sinogram and
image domains have been rapidly applied on metal artifact reduction (MAR) task.
The supervised dual-domain methods perform well on synthesized data, while
unsupervised methods with unpaired data are more generalized on clinical data.
However, most existing methods intend to restore the corrupted sinogram within
metal trace, which essentially remove beam hardening artifacts but ignore other
components of metal artifacts, such as scatter, non-linear partial volume
effect and noise. In this paper, we mathematically derive a physical property
of metal artifacts which is verified via Monte Carlo (MC) simulation and
propose a novel physics based non-local dual-domain network (PND-Net) for MAR
in CT imaging. Specifically, we design a novel non-local sinogram decomposition
network (NSD-Net) to acquire the weighted artifact component, and an image
restoration network (IR-Net) is proposed to reduce the residual and secondary
artifacts in the image domain. To facilitate the generalization and robustness
of our method on clinical CT images, we employ a trainable fusion network
(F-Net) in the artifact synthesis path to achieve unpaired learning.
Furthermore, we design an internal consistency loss to ensure the integrity of
anatomical structures in the image domain, and introduce the linear
interpolation sinogram as prior knowledge to guide sinogram decomposition.
Extensive experiments on simulation and clinical data demonstrate that our
method outperforms the state-of-the-art MAR methods.Comment: 19 pages, 8 figure
Deep Learning with Constraints and Priors for Improved Subject Clustering, Medical Imaging, and Robust Inference
Deep neural networks (DNNs) have achieved significant success in several fields including computer vision, natural language processing, and robot control. The common philosophy behind these success is the use of large amount of annotated data and end-to-end networks with task-specific constraints and priors implicitly incorporated into the trained model without the need for careful feature engineering. However, DNNs are shown to be vulnerable to distribution shifts and adversarial perturbations, which indicates that such implicit priors and constraints are not sufficient for real world applications. In this dissertation, we target three applications and design task-specific constraints and priors for improved performance of deep neural networks.
We first study the problem of subject clustering, the task of grouping face images of the same person together. We propose to utilize the prior structure in the feature space of DNNs trained for face identification to design a novel clustering algorithm. Specifically, the clustering algorithm exploits the local neighborhood structure of deep representations by exemplar-based learning based on k-nearest neighbors (k-NN). Extensive experiments show promising results for grouping face images according to subject identity. As an example, we apply the proposed clustering algorithm to automatically curate a large-scale face dataset with noisy labels and show that the performance of face recognition DNNs can be significantly improved by training on the curated dataset. Furthermore, we empirically find that the k-NN rule does not capture proper local structures for deep representations when each subject has very few face images. We then propose to improve upon the exemplar-based approach by a density-aware similarity measure and theoretically show its asymptotic convergence to a density estimator. We conduct experiments on challenging face datasets that show promising results.
Second, we study the problem of metal artifact reduction in computed tomography (CT). Unlike typical image restoration tasks such as super-resolution and denoising, metal artifacts in CT images are structured and non-local. Conventional DNNs do not generalize well when metal implants with unseen shapes are presented. We find that the imaging process of CT induces a data consistency prior that can be exploited for image enhancement. Based on this observation, we propose a dual-domain learning approach to CT metal artifact reduction. We design and implement a novel Radon inversion layer that allows gradients in the image domain to be backpropagated to the projection domain. Experiments conducted on both simulated datasets and clinical datasets show promising results. Compared to conventional DNN-based models, the proposed dual-domain approach leads to impressive metal artifact reduction and has improved generalization capability.
Finally, we study the problem of robust classification. In the past few years, the vulnerability of DNNs to small imperceptible perturbations has been widely studied, which raises concerns about the security and robustness of DNNs against possible threat models. To defend against threat models, Samangoui et al. proposed DefenseGAN, a preprocessing approach which removes adversarial perturbations by projecting the input images onto the learned data prior. However, the projection operation in DefenseGAN is time-consuming and may not yield proper reconstruction when images have complicated textures. We propose an inversion network to constrain the initial estimates of the latent code for input images. With the proposed constraint, the number of optimization steps in DefenseGAN can be reduced while achieving improved accuracy and robustness. Furthermore, we conduct empirical studies on attack methods that have claimed to break DefenseGAN, which shows that on-manifold robustness might be the key factor for ensuring adversarial robustness
Improved compressed sensing algorithm for sparse-view CT
In computed tomography (CT) there are many situations where reconstruction may need to be performed with sparse-view data. In sparse-view CT imaging, strong streak artifacts may appear in conventionally reconstructed images due to the limited sampling rate, compromising image quality. Compressed sensing (CS) algorithm has shown potential to accurately recover images from highly undersampled data. In the past few years, total variation (TV)-base compressed sensing algorithms have been proposed to suppress the streak artifact in CT image reconstruction. In this paper, we formulate the problem of CT imaging under transform sparsity and sparse-view constraints, and propose a novel compressed sensing-based algorithm for CT image reconstruction from few-view data, in which we simultaneously minimize the â„“1 norm, total variation and a least square measure. The main feature of our algorithm is the use of two sparsity transforms: discrete wavelet transform and discrete gradient transform, both of which are proven to be powerful sparsity transforms. Experiments with simulated and real projections were performed to evaluate and validate the proposed algorithm. The reconstructions using the proposed approach have less streak artifacts and reconstruction errors than other conventional methods
A Review on Deep Learning in Medical Image Reconstruction
Medical imaging is crucial in modern clinics to guide the diagnosis and
treatment of diseases. Medical image reconstruction is one of the most
fundamental and important components of medical imaging, whose major objective
is to acquire high-quality medical images for clinical usage at the minimal
cost and risk to the patients. Mathematical models in medical image
reconstruction or, more generally, image restoration in computer vision, have
been playing a prominent role. Earlier mathematical models are mostly designed
by human knowledge or hypothesis on the image to be reconstructed, and we shall
call these models handcrafted models. Later, handcrafted plus data-driven
modeling started to emerge which still mostly relies on human designs, while
part of the model is learned from the observed data. More recently, as more
data and computation resources are made available, deep learning based models
(or deep models) pushed the data-driven modeling to the extreme where the
models are mostly based on learning with minimal human designs. Both
handcrafted and data-driven modeling have their own advantages and
disadvantages. One of the major research trends in medical imaging is to
combine handcrafted modeling with deep modeling so that we can enjoy benefits
from both approaches. The major part of this article is to provide a conceptual
review of some recent works on deep modeling from the unrolling dynamics
viewpoint. This viewpoint stimulates new designs of neural network
architectures with inspirations from optimization algorithms and numerical
differential equations. Given the popularity of deep modeling, there are still
vast remaining challenges in the field, as well as opportunities which we shall
discuss at the end of this article.Comment: 31 pages, 6 figures. Survey pape