Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a cross-entropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fully-automatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global- and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201

Domain Adaptation for Novel Imaging Modalities with Application to Prostate MRI

The need for training data can impede the adoption of novel imaging modalities for deep learning-based medical image analysis. Domain adaptation can mitigate this problem by exploiting training samples from an existing, densely-annotated source domain within a novel, sparsely-annotated target domain, by bridging the differences between the two domains. In this thesis we present methods for adapting between diffusion-weighed (DW)-MRI data from multiparametric (mp)-MRI acquisitions and VERDICT (Vascular, Extracellular and Restricted Diffusion for Cytometry in Tumors) MRI, a richer DW-MRI technique involving an optimized acquisition protocol for cancer characterization. We also show that the proposed methods are general and their applicability extends beyond medical imaging. First, we propose a semi-supervised domain adaptation method for prostate lesion segmentation on VERDICT MRI. Our approach relies on stochastic generative modelling to translate across two heterogeneous domains at pixel-space and exploits the inherent uncertainty in the cross-domain mapping to generate multiple outputs conditioned on a single input. We further extend this approach to the unsupervised scenario where there is no labeled data for the target domain. We rely on stochastic generative modelling to translate across the two domains at pixel space and introduce two loss functions that promote semantic consistency. Finally we demonstrate that the proposed approaches extend beyond medical image analysis and focus on unsupervised domain adaptation for semantic segmentation of urban scenes. We show that relying on stochastic generative modelling allows us to train more accurate target networks and achieve state-of-the-art performance on two challenging semantic segmentation benchmarks

Automatic prostate and prostate zones segmentation of magnetic resonance images using DenseNet-like U-net

Magnetic resonance imaging (MRI) provides detailed anatomical images of the prostate and its zones. It has a crucial role for many diagnostic applications. Automatic segmentation such as that of the prostate and prostate zones from MR images facilitates many diagnostic and therapeutic applications. However, the lack of a clear prostate boundary, prostate tissue heterogeneity, and the wide interindividual variety of prostate shapes make this a very challenging task. To address this problem, we propose a new neural network to automatically segment the prostate and its zones. We term this algorithm Dense U-net as it is inspired by the two existing state-of-the-art tools—DenseNet and U-net. We trained the algorithm on 141 patient datasets and tested it on 47 patient datasets using axial T2-weighted images in a four-fold cross-validation fashion. The networks were trained and tested on weakly and accurately annotated masks separately to test the hypothesis that the network can learn even when the labels are not accurate. The network successfully detects the prostate region and segments the gland and its zones. Compared with U-net, the second version of our algorithm, Dense-2 U-net, achieved an average Dice score for the whole prostate of 92.1± 0.8% vs. 90.7 ± 2%, for the central zone of 89.5±2% vs. 89.1±2.2 %, and for the peripheral zone of 78.1± 2.5% vs. 75±3%. Our initial results show Dense-2 U-net to be more accurate than state-of-the-art U-net for automatic segmentation of the prostate and prostate zones

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Prostate segmentation in MR images using ensemble deep convolutional neural networks

The automated segmentation of the prostate gland from MR images is increasingly used for clinical diagnosis. Since deep learning demonstrates superior performance in computer vision applications, we propose a coarse-to-fine segmentation strategy using ensemble deep convolutional neural networks (DCNNs) to address prostate segmentation in MR images. First, we use registration-based coarse segmentation on pre-processed prostate MR images to define the potential boundary region. We then train four DCNNs as voxel-based classifiers and classify the voxel in the potential region is a prostate voxel when at least three DCNNs made that decision. Finally, we use boundary refinement to eliminate the outliers and smooth the boundary. We evaluated our approach on the MICCAI PROMIS12 challenge dataset and our experimental results verify the effectiveness of the proposed algorithms

Deep learning assisted MRI guided attenuation correction in PET

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonPositron emission tomography (PET) is a unique imaging modality that provides physiological and functional details of the tissue at the molecular level. However, the acquired PET images have some limitations such as the attenuation. PET attenuation correction is an essential step to obtain the full potential of PET quantification. With the wide use of hybrid PET/MR scanners, magnetic resonance (MR) images are used to address the problem of PET attenuation correction. The MR images segmentation is one simple and robust approach to create pseudo computed tomography (CT) images, which are used to generate attenuation coefficient maps to correct the PET attenuation. Recently, deep learning has been proposed and used as a promising technique to efficiently perform MR and various medical images segmentation. In this research work, deep learning guided segmentation approaches have been proposed to enhance the bone class segmentation of MR brain images in order to generate accurate pseudo-CT images. The first approach has introduced the combination of handcrafted features with deep learning features to enrich the set of features. Multiresolution analysis techniques, which generate multiscale and multidirectional coefficients of an image such as contourlet and shearlet transforms, are applied and combined with deep convolutional neural network (CNN) features. Different experiments have been conducted to investigate the number of selected coefficients and the insertion location of the handcrafted features. The second approach aims at reducing the segmentation algorithm’s complexity while maintaining the segmentation performance. An attention based convolutional encode-decoder network has been proposed to adaptively recalibrate the deep network features. This attention based network consists of two different squeeze and excitation blocks that excite the features spatially and channel wise. The two blocks are combined sequentially to decrease the number of network’s parameters and reduces the model complexity. The third approach has been focuses on the application of transfer learning from different MR sequences such as T1 weighted (T1-w) and T2 weighted (T2-w) images. A pretrained model with T1-w MR sequences is fine tuned to perform the segmentation of T2-w images. Multiple fine tuning approaches and experiments have been conducted to study the best fine tuning mechanism that is able to build an efficient segmentation model for both T1-w and T2-w segmentation. Clinical datasets of fifty patients with different conditions and diagnosis have been used to carry an objective evaluation to measure the segmentation performance of the results obtained by the three proposed methods. The first and second approaches have been validated with other studies in the literature that applied deep network based segmentation technique to perform MR based attenuation correction for PET images. The proposed methods have shown an enhancement in the bone segmentation with an increase of dice similarity coefficient (DSC) from 0.6179 to 0.6567 using an ensemble of CNNs with an improvement percentage of 6.3%. The proposed excitation-based CNN has decreased the model complexity by decreasing the number of trainable parameters by more than 46% where less computing resources are required to train the model. The proposed hybrid transfer learning method has shown its superiority to build a multi-sequences (T1-w and T2-w) segmentation approach compared to other applied transfer learning methods especially with the bone class where the DSC is increased from 0.3841 to 0.5393. Moreover, the hybrid transfer learning approach requires less computing time than transfer learning using open and conservative fine tuning