Deep Learning for Task-Based Image Quality Assessment in Medical Imaging

It has been advocated to use objective measures of image quality (IQ) for assessing and optimizing medical imaging systems. Objective measures of IQ quantify the performance of an observer at a specific diagnostic task. Binary signal detection tasks and joint signal detection and localization (detection-localization) tasks are commonly considered in medical imaging. When optimizing imaging systems for binary signal detection tasks, the performance of the Bayesian Ideal Observer (IO) has been advocated for use as a figure-of-merit (FOM). The IO maximizes the observer performance that is summarized by the receiver operating characteristic (ROC) curve. When signal detection-localization tasks are considered, the IO that implements a modified generalized likelihood ratio test (MGLRT) maximizes the observer performance as measured by the localization ROC (LROC) curve. However, computation of the IO test statistic generally is analytically intractable. To address this difficulty, sampling-based methods that employ Markov-Chain Monte Carlo (MCMC) techniques have been proposed. However, current applications of MCMC methods have been limited to relatively simple stochastic object models (SOMs). When the IO is difficult or intractable to compute, the optimal linear observer, known as the Hotelling Observer (HO), can be employed to evaluate objective measures of IQ. Although computation of the HO is easier than that of the IO, it can still be challenging or even intractable because a potentially large covariance matrix needs to be estimated and subsequently inverted. In the first part of the dissertation, we introduce supervised learning-based methods for approximating the IO and the HO for binary signal detection tasks. The use of convolutional neural networks (CNNs) to approximate the IO and the use of single layer neural networks (SLNNs) to directly estimate the Hotelling template without computing and inverting covariance matrices are demonstrated. In the second part, a supervised learning method that employs CNNs to approximate the IO for signal detection-localization tasks is presented. This method represents a deep-learning-based implementation of a MGLRT that defines the IO decision strategy for signal detection-localization tasks. When evaluating observer performance for assessing and optimizing imaging systems by use of objective measures of IQ, all sources of variability in the measured image data should be accounted for. One important source of variability that can significantly affect observer performance is the variation in the ensemble of objects to-be-imaged. To describe this variability, a SOM can be established. A SOM is a generative model that can produce an ensemble of simulated objects with prescribed statistical properties. In order to establish a realistic SOM, it is desirable to use experimental data. Generative adversarial networks (GANs) hold great potential for establishing SOMs. However, images produced by imaging systems are affected by the measurement noise and a potential reconstruction process. Therefore, GANs that are trained by use of these images cannot represent SOMs because they are not established to learn object variability alone. An augmented GAN architecture named AmbientGAN that includes a measurement operator was proposed to address this issue. However, AmbientGANs cannot be immediately implemented with advanced GAN training strategies such as progressive growing of GANs (ProGANs). Therefore, the ability of AmbientGANs to establish realistic and sophisticated SOMs is limited. In the third part of this dissertation, we propose a novel deep learning method named progressively growing AmbientGANs (ProAmGANs) that incorporates the advanced progressive growing training procedure and therefore enables the AmbientGAN to be applied to realistically sized medical image data. Stylized numerical studies involving a variety of object ensembles with common medical imaging modalities are presented. Finally, a novel sampling-based method named MCMC-GAN is developed to approximate the IO. This method applies MCMC algorithms to SOMs that are established by use of GAN techniques. Because the implementation of GANs is general and not limited to specific images, our proposed method can be implemented with sophisticated object models and therefore extends the domain of applicability of the MCMC techniques. Numerical studies involving clinical brain positron emission tomography (PET) images and brain magnetic resonance (MR) images are presented

On the impact of incorporating task-information in learning-based image denoising

A variety of deep neural network (DNN)-based image denoising methods have been proposed for use with medical images. These methods are typically trained by minimizing loss functions that quantify a distance between the denoised image, or a transformed version of it, and the defined target image (e.g., a noise-free or low-noise image). They have demonstrated high performance in terms of traditional image quality metrics such as root mean square error (RMSE), structural similarity index measure (SSIM), or peak signal-to-noise ratio (PSNR). However, it has been reported recently that such denoising methods may not always improve objective measures of image quality. In this work, a task-informed DNN-based image denoising method was established and systematically evaluated. A transfer learning approach was employed, in which the DNN is first pre-trained by use of a conventional (non-task-informed) loss function and subsequently fine-tuned by use of the hybrid loss that includes a task-component. The task-component was designed to measure the performance of a numerical observer (NO) on a signal detection task. The impact of network depth and constraining the fine-tuning to specific layers of the DNN was explored. The task-informed training method was investigated in a stylized low-dose X-ray computed tomography (CT) denoising study for which binary signal detection tasks under signal-known-statistically (SKS) with background-known-statistically (BKS) conditions were considered. The impact of changing the specified task at inference time to be different from that employed for model training, a phenomenon we refer to as "task-shift", was also investigated. The presented results indicate that the task-informed training method can improve observer performance while providing control over the trade off between traditional and task-based measures of image quality

UNet and MobileNet CNN-based model observers for CT protocol optimization: comparative performance evaluation by means of phantom CT images

Purpose: The aim of this work is the development and characterization of a model observer (MO) based on convolutional neural networks (CNNs), trained to mimic human observers in image evaluation in terms of detection and localization of low-contrast objects in CT scans acquired on a reference phantom. The final goal is automatic image quality evaluation and CT protocol optimization to fulfill the ALARA principle. Approach: Preliminary work was carried out to collect localization confidence ratings of human observers for signal presence/absence from a dataset of 30,000 CT images acquired on a PolyMethyl MethAcrylate phantom containing inserts filled with iodinated contrast media at different concentrations. The collected data were used to generate the labels for the training of the artificial neural networks. We developed and compared two CNN architectures based respectively on Unet and MobileNetV2, specifically adapted to achieve the double tasks of classification and localization. The CNN evaluation was performed by computing the area under localization-ROC curve (LAUC) and accuracy metrics on the test dataset. Results: The mean of absolute percentage error between the LAUC of the human observer and MO was found to be below 5% for the most significative test data subsets. An elevated inter-rater agreement was achieved in terms of S-statistics and other common statistical indices. Conclusions: Very good agreement was measured between the human observer and MO, as well as between the performance of the two algorithms. Therefore, this work is highly supportive of the feasibility of employing CNN-MO combined with a specifically designed phantom for CT protocol optimization programs