Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation

We propose a dual pathway, 11-layers deep, three-dimensional Convolutional Neural Network for the challenging task of brain lesion segmentation. The devised architecture is the result of an in-depth analysis of the limitations of current networks proposed for similar applications. To overcome the computational burden of processing 3D medical scans, we have devised an efficient and effective dense training scheme which joins the processing of adjacent image patches into one pass through the network while automatically adapting to the inherent class imbalance present in the data. Further, we analyze the development of deeper, thus more discriminative 3D CNNs. In order to incorporate both local and larger contextual information, we employ a dual pathway architecture that processes the input images at multiple scales simultaneously. For post-processing of the network's soft segmentation, we use a 3D fully connected Conditional Random Field which effectively removes false positives. Our pipeline is extensively evaluated on three challenging tasks of lesion segmentation in multi-channel MRI patient data with traumatic brain injuries, brain tumours, and ischemic stroke. We improve on the state-of-the-art for all three applications, with top ranking performance on the public benchmarks BRATS 2015 and ISLES 2015. Our method is computationally efficient, which allows its adoption in a variety of research and clinical settings. The source code of our implementation is made publicly available.This work is supported by the EPSRC First Grant scheme (grant ref no. EP/N023668/1) and partially funded under the 7th Framework Programme by the European Commission (TBIcare: http: //www.tbicare.eu/ ; CENTER-TBI: https://www.center-tbi.eu/). This work was further supported by a Medical Research Council (UK) Program Grant (Acute brain injury: heterogeneity of mechanisms, therapeutic targets and outcome effects [G9439390 ID 65883]), the UK National Institute of Health Research Biomedical Research Centre at Cambridge and Technology Platform funding provided by the UK Department of Health. KK is supported by the Imperial College London PhD Scholarship Programme. VFJN is supported by a Health Foundation/Academy of Medical Sciences Clinician Scientist Fellowship. DKM is supported by an NIHR Senior Investigator Award. We gratefully acknowledge the support of NVIDIA Corporation with the donation of two Titan X GPUs for our research

Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation

We propose a dual pathway, 11-layers deep, three-dimensional Convolutional Neural Network for the challenging task of brain lesion segmentation. The devised architecture is the result of an in-depth analysis of the limitations of current networks proposed for similar applications. To overcome the computational burden of processing 3D medical scans, we have devised an efficient and effective dense training scheme which joins the processing of adjacent image patches into one pass through the network while automatically adapting to the inherent class imbalance present in the data. Further, we analyze the development of deeper, thus more discriminative 3D CNNs. In order to incorporate both local and larger contextual information, we employ a dual pathway architecture that processes the input images at multiple scales simultaneously. For post-processing of the network’s soft segmentation, we use a 3D fully connected Conditional Random Field which effectively removes false positives. Our pipeline is extensively evaluated on three challenging tasks of lesion segmentation in multi-channel MRI patient data with traumatic brain injuries, brain tumors, and ischemic stroke. We improve on the state-of-theart for all three applications, with top ranking performance on the public benchmarks BRATS 2015 and ISLES 2015. Our method is computationally efficient, which allows its adoption in a variety of research and clinical settings. The source code of our implementation is made publicly availabl

Multimodal image analysis of the human brain

Gedurende de laatste decennia heeft de snelle ontwikkeling van multi-modale en niet-invasieve hersenbeeldvorming technologieën een revolutie teweeg gebracht in de mogelijkheid om de structuur en functionaliteit van de hersens te bestuderen. Er is grote vooruitgang geboekt in het beoordelen van hersenschade door gebruik te maken van Magnetic Reconance Imaging (MRI), terwijl Elektroencefalografie (EEG) beschouwd wordt als de gouden standaard voor diagnose van neurologische afwijkingen. In deze thesis focussen we op de ontwikkeling van nieuwe technieken voor multi-modale beeldanalyse van het menselijke brein, waaronder MRI segmentatie en EEG bronlokalisatie. Hierdoor voegen we theorie en praktijk samen waarbij we focussen op twee medische applicaties: (1) automatische 3D MRI segmentatie van de volwassen hersens en (2) multi-modale EEG-MRI data analyse van de hersens van een pasgeborene met perinatale hersenschade. We besteden veel aandacht aan de verbetering en ontwikkeling van nieuwe methoden voor accurate en ruisrobuuste beeldsegmentatie, dewelke daarna succesvol gebruikt worden voor de segmentatie van hersens in MRI van zowel volwassen als pasgeborenen. Daarenboven ontwikkelden we een geïntegreerd multi-modaal methode voor de EEG bronlokalisatie in de hersenen van een pasgeborene. Deze lokalisatie wordt gebruikt voor de vergelijkende studie tussen een EEG aanval bij pasgeborenen en acute perinatale hersenletsels zichtbaar in MRI

Deep learning-based brain tumour image segmentation and its extension to stroke lesion segmentation

Medical imaging plays a very important role in clinical methods of treating cancer, as well as treatment selection, diagnosis, an evaluating the response to therapy. One of the best-known acquisition modalities is magnetic resonance imaging (MRI), which is used widely in the analysis of brain tumours by means of acquisition protocols (e.g. conventional and advanced). Due to the wide variation in the shape, location and appearance of tumours, automated segmentation in MRI is a difficult task. Although many studies have been conducted, automated segmentation is difficult and work to improve the accuracy of tumour segmentation is still ongoing. This research aims to develop fully automated methods for segmenting the abnormal tissues associated with brain tumours (i.e. those subject to oedema, necrosis and enhanced) from the multimodal MRI images that help radiologists to diagnose conditions and plan treatment. In this thesis the machine-learned features from the deep learning convolutional neural network (CIFAR) are investigated and joined with hand-crafted histogram texture features to encode global information and local dependencies in the representation of features. The combined features are then applied in a decision tree (DT) classifier to group individual pixels into normal brain tissues and the various parts of a tumour. These features are good point view for the clinicians to accurately visualize the texture tissue of tumour and sub-tumour regions. To further improve the segmentation of tumour and sub-tumour tissues, 3D datasets of the four MRI modalities (i.e. FLAIR, T1, T1ce and T2) are used and fully convolutional neural networks, called SegNet, are constructed for each of these four modalities of images. The outputs of these four SegNet models are then fused by choosing the one with the highest scores to construct feature maps, with the pixel intensities as an input to a DT classifier to further classify each pixel as either a normal brain tissue or the component parts of a tumour. To achieve a high-performance accuracy in the segmentation as a whole, deep learning (the IV SegNet network) and the hand-crafted features are combined, particularly in the grey-level co-occurrence matrix (GLCM) in the region of interest (ROI) that is initially detected from FLAIR modality images using the SegNet network. The methods that have been developed in this thesis (i.e. CIFAR _PI_HIS _DT, SegNet_Max_DT and SegNet_GLCM_DT) are evaluated on two datasets: the first is the publicly available Multimodal Brain Tumour Image Segmentation Benchmark (BRATS) 2017 dataset, and the second is a clinical dataset. In brain tumour segmentation methods, the F-measure performance of more than 0.83 is accepted, or at least useful from a clinical point of view, for segmenting the whole tumour structure which represents the brain tumour boundaries. Thanks to it, our proposed methods show promising results in the segmentation of brain tumour structures and they provide a close match to expert delineation across all grades of glioma. To further detect brain injury, these three methods were adopted and exploited for ischemic stroke lesion segmentation. In the steps of training and evaluation, the publicly available Ischemic Stroke Lesion (ISLES 2015) dataset and a clinical dataset were used. The performance accuracies of the three developed methods in ischemic stroke lesion segmentation were assessed. The third segmentation method (SegNet_GLCM_DT) was found to be more accurate than the other two methods (SegNet_Max_DT and SegNet_GLCM_DT) because it exploits GLCM as a set of hand-crafted features with machine features, which increases the accuracy of segmentation with ischemic stroke lesion

Advancing efficiency and robustness of neural networks for imaging

Enabling machines to see and analyze the world is a longstanding research objective. Advances in computer vision have the potential of influencing many aspects of our lives as they can enable machines to tackle a variety of tasks. Great progress in computer vision has been made, catalyzed by recent progress in machine learning and especially the breakthroughs achieved by deep artificial neural networks. Goal of this work is to alleviate limitations of deep neural networks that hinder their large-scale adoption for real-world applications. To this end, it investigates methodologies for constructing and training deep neural networks with low computational requirements. Moreover, it explores strategies for achieving robust performance on unseen data. Of particular interest is the application of segmenting volumetric medical scans because of the technical challenges it imposes, as well as its clinical importance. The developed methodologies are generic and of relevance to a broader computer vision and machine learning audience. More specifically, this work introduces an efficient 3D convolutional neural network architecture, which achieves high performance for segmentation of volumetric medical images, an application previously hindered by high computational requirements of 3D networks. It then investigates sensitivity of network performance on hyper-parameter configuration, which we interpret as overfitting the model configuration to the data available during development. It is shown that ensembling a set of models with diverse configurations mitigates this and improves generalization. The thesis then explores how to utilize unlabelled data for learning representations that generalize better. It investigates domain adaptation and introduces an architecture for adversarial networks tailored for adaptation of segmentation networks. Finally, a novel semi-supervised learning method is proposed that introduces a graph in the latent space of a neural network to capture relations between labelled and unlabelled samples. It then regularizes the embedding to form a compact cluster per class, which improves generalization.Open Acces

An insight into the brain of patients with type-2 diabetes mellitus and impaired glucose tolerance using multi-modal magnetic resonance image processing

The purpose of this thesis was to investigate brain anatomy and physiology of subjects with impaired glucose tolerance (IGT - 12 subjects), type-2 diabetes (T2DM - 17 subjects) and normoglycemia (16 subjects) using multi-modal magnetic resonance imaging (MRI) at 3T. Perfusion imaging using quantitative STAR labeling of arterial regions (QUASAR) arterial spin labeling (ASL) was the core dataset. Optimization of the post-processing methodology for this sequence was performed and the outcome was used for hemodynamic analysis of the cohort. Typical perfusion-related parameters, along with novel hemodynamic features were quantified. High-resolution structural, angiographic and carotid flow scans were also acquired and processed. Functional acquisitions were repeated following a vasodilating stimulus. Differences between the groups were examined using statistical analysis and a machine-learning framework. Hemodynamic parameters differing between the groups emerged from both baseline and post-stimulus scans for T2DM and mainly from the post-stimulus scan for IGT. It was demonstrated that quantification of not-typically determined hemodynamic features could lead to optimal group-separation. Such features captured the pattern of delayed delivery of the blood to the arterial and tissue compartments of the hyperglycemic groups. Alterations in gray and white matter, cerebral vasculature and carotid blood flow were detected for the T2DM group. The IGT cohort was structurally similar to the healthy cohort but demonstrated functional similarities to T2DM. When combining all extracted MRI metrics, features driving optimal separation between different glycemic conditions emerged mainly from the QUASAR scan. The only highly discriminant non-QUASAR feature, when comparing T2DM to healthy subjects, emerged from the cerebral angiogram. In this thesis, it was demonstrated that MRI-derived features could lead to potentially optimal differentiation between normoglycemia and hyperglycemia. More importantly, it was shown that an impaired cerebral hemodynamic pattern exists in both IGT and T2DM and that the IGT group exhibits functional alterations similar to the T2DM group