Automatic Semantic Segmentation of the Lumbar Spine: Clinical Applicability in a Multi-parametric and Multi-centre Study on Magnetic Resonance Images

One of the major difficulties in medical image segmentation is the high variability of these images, which is caused by their origin (multi-centre), the acquisition protocols (multi-parametric), as well as the variability of human anatomy, the severity of the illness, the effect of age and gender, among others. The problem addressed in this work is the automatic semantic segmentation of lumbar spine Magnetic Resonance images using convolutional neural networks. The purpose is to assign a class label to each pixel of an image. Classes were defined by radiologists and correspond to different structural elements like vertebrae, intervertebral discs, nerves, blood vessels, and other tissues. The proposed network topologies are variants of the U-Net architecture. Several complementary blocks were used to define the variants: Three types of convolutional blocks, spatial attention models, deep supervision and multilevel feature extractor. This document describes the topologies and analyses the results of the neural network designs that obtained the most accurate segmentations. Several of the proposed designs outperform the standard U-Net used as baseline, especially when used in ensembles where the output of multiple neural networks is combined according to different strategies.Comment: 19 pages, 9 Figures, 8 Tables; Supplementary Material: 6 pages, 8 Table

Saliency-based approaches for multidimensional explainability of deep networks

In deep learning, visualization techniques extract the salient patterns exploited by deep networks to perform a task (e.g. image classification) focusing on single images. These methods allow a better understanding of these complex models, empowering the identification of the most informative parts of the input data. Beyond the deep network understanding, visual saliency is useful for many quantitative reasons and applications, both in the 2D and 3D domains, such as the analysis of the generalization capabilities of a classifier and autonomous navigation. In this thesis, we describe an approach to cope with the interpretability problem of a convolutional neural network and propose our ideas on how to exploit the visualization for applications like image classification and active object recognition. After a brief overview on common visualization methods producing attention/saliency maps, we will address two separate points: firstly, we will describe how visual saliency can be effectively used in the 2D domain (e.g. RGB images) to boost image classification performances: as a matter of fact, visual summaries, i.e. a compact representation of an ensemble of saliency maps, can be used to improve the classification accuracy of a network through summary-driven specializations. Then, we will present a 3D active recognition system that allows to consider different views of a target object, overcoming the single-view hypothesis of classical object recognition, making the classification problem much easier in principle. Here we adopt such attention maps in a quantitative fashion, by building a 3D dense saliency volume which fuses together saliency maps obtained from different viewpoints, obtaining a continuous proxy on which parts of an object are more discriminative for a given classifier. Finally, we will show how to inject this representations in a real world application, so that an agent (e.g. robot) can move knowing the capabilities of its classifier

Automated quality control by application of machine learning techniques for quantitative liver MRI

Quantitative magnetic resonance imaging (qMRI) and multi-parametric MRI are being increasingly used to diagnose and monitor liver diseases such as non-alcoholic fatty liver disease (NAFLD). These acquisitions are comparably more complicated than traditional T1-weighted and T2-weighted MRI scans and are also more prone to image quality is- sues and artefacts. In order for the output of the qMRI scans to be useable, they must undergo a rigorous and often lengthy quality control (QC). This manual QC is prone to human error and subjective. Additionally, with the development of new qMRI tech- niques, this leads to the manifestation of new quality issues. This thesis focuses on the development and implementation of automated QC processes for liver qMRI scans, that is where possible tag-free such that the process can be adapted to different imag- ing techniques. These automated QC processes were implemented using a variety of machine learning (ML) and deep learning (DL) approaches. These methods, developed on T1 mapping in UKBiobank, were designed to output metrics from the MRI scans that could be used to identify a specific quality issue, such as in chapter 3, or give a more general indication of the image quality in chapter 4. Furthermore, it was hypothe- sised that the introduction of associated meta-data, such as patient factors and scanning parameters, into these deep learning models would increase overall performance. This was explored in chapter 5. Finally, in order to assess the utility of our developed al- gorithms in a wider setting except for T1 mapping in UKBiobank, we tested it in two settings. Pilot study one assessed the utility of the model in T1 mapping in a separate study (CoverScan). Pilot study two assessed the utility of the model in a different qMRI acquisition; proton density fat fraction (PDFF) acquisitions from UKBiobank

Detection of organs in CT images using Neural Networks

Táto práca sa zaoberá výskumom zobrazovacích metód v medicíne, klasických prístupov k segmentácii obrázkov, CT a konvolučným neuronovým sietiam. Praktickou časťou je implementácia architektúry 3D UNet pre segmentáciu chrbtice a jednotlivých stavcov z CT obrázkov a jej porovnanie s jej 2D verziou.This thesis contains research of the field of medical imaging, classical methods of image segmentation, computed tomography and convolutional neural networks. The practical part involves implementation of an architecture of 3D UNet for segmentation of the spine and specific vertebrae from CT scans. Furthermore, this architecture is compared to its 2D counterpart

Quantitative Magnetic Resonance Imaging and Analysis of Articular Cartilage and Osteoarthritis

MRI plays an important role in the continuing search for a sensitive osteoarthritis (OA) imaging biomarker able to detect early, pre-morphological alterations in cartilage composition. Determining the compositional recovery pattern of cartilage following acute joint loading could potentially present a more sensitive biomarker for defining cartilage health [1]. However, only a limited amount of studies have assessed both the immediate effect of joint loading on cartilage, as well as its post-loading recovery. In addition, when assessing the compositional responses of cartilage to joint loading, previous studies usually did not incorporate the measurement error of the used quantitative MRI technique into their analysis. Therefore, an uncertainty persists whether or not compositional MRI techniques are sensitive enough to measure changes in water and macromolecular content of cartilage, or if previous studies were merely measuring noise. Consequently, an objective of this thesis is to increase our understanding of and reliability in quantitative T2 and T1ρ relaxation time mapping to detect compositional responses of cartilage following a joint loading activity. Furthermore, to obtain the quantitative morphological and compositional measures of cartilage, detailed region-specific delineation of cartilage is required. This delineation (or segmentation) of cartilage is laborious and time-consuming as it is usually performed manually by an expert observer. Many new advances in image analysis, particularly those in convolutional neural networks (CNNs) and deep learning, have enabled a time-efficient semi- or fully-automated alternative to this process [2, 3]. This thesis explores the utility of deep CNNs generated segmentations for accurate surface-based analysis of cartilage morphology and composition from knee MRIs as well as of cortical bone thickness from knee CTs. Chapter 1 will provide an introduction into the structure and biomechanics of articular cartilage and the role of MRI in imaging the degenerative joint disorder, osteoarthritis as well as the effects of different joint loading activities on cartilage morphology and composition. Chapter 2 explains the principle of MRI and the pulse sequences used in the following chapter for the morphometric and compositional assessment of articular cartilage. Chapter 3 describes the use of 3D Cartilage Surface Mapping (3D-CaSM) [3] to assess variations in cartilage T1ρ and T2 relaxation times of young, healthy participants following a mild, unilateral stepping activity. By evaluating and incorporating the intrasessional repeatability of the T1ρ and T2 mapping techniques, I aim to highlight those cartilage areas experiencing exercise-induced compositional changes greater than measurement error. A significant amount of time is needed to manually segment the regions-of-interest required to perform the 3D-CaSM used in Chapter 3. Therefore, in Chapter 4, I assessed the use of deep convolutional neural networks for automating the segmentation process for multiple knee joint tissues simultaneous and increase the time-efficiency for evaluating knee MR datasets. I evaluated the use of a conditional Generative Adversarial Network (cGAN) as a potentially improved method for automated segmentation compared to the widely used convolutional neural network, U-Net. In Chapter 5 I combined the 3D-CaSM and automated segmentation methods presented in Chapters 3 and 4, respectively to assess the use of fully automatic segmentations of femoral and tibial bone-cartilage structures for accurate surface-based analysis of cartilage morphology and composition on knee MR images. This was performed on publicly available data from the Osteoarthritis Initiative, a multicentre observational study with expert manual segmentations provided by the Zuse Institute in Berlin. Chapter 6 describes an automated pipeline for subchondral cortical bone thickness mapping from knee CT data. I developed a method of using automated segmentations of articular cartilage and bone from knee MRI data to determine the periarticular bone surface which is covered by cartilage. This surface was then used to perform cortical bone thickness measurements on corresponding CT data. I validated this pipeline using data from the EU-funded, multi-centre observational study called Applied Private-Public partneRship enabling OsteoArthritis Clinical Headway (APPROACH). Chapter 7 summarises the main conclusions and contributions of the works presented in this thesis as well as providing directions for future work.PhD Studentship funded by GlaxoSmithKlin

Inter-comparison of medical image segmentation algorithms

Segmentation of images is a vital part of medical image processing, and MRI (Magnetic Resonance Imaging) is already recognized as a very important tool for clinical diagnosis. In this thesis, comparisons between different segmentation algorithms are carried out, specifically on brain MRI images. Initial parts of the thesis provide the background to the project, and an introduction to the basic principles of MRI, respectively, followed by parameter definitions and MRI image artifacts. The next part briefly covers various image pre-processing techniques which are required, and this is followed with a review of the major segmentation techniques which are available, including thresholding, region growing, clustering, and K-Means clustering. The concept of fuzzy logic is also introduced here, and the chapter concludes with a discussion of fuzzy logic based segmentation algorithms such as Fuzzy C-Means (FCM) and Improved Fuzzy C-Means (IFCM) clustering algorithms. The following part provides details concerning the source, type and parameters of the data (images) used for this thesis. Evaluation and inter-comparisons between a number of different segmentation algorithms are given in near concluding part, finally, conclusions and suggestions for future research are provided in last part. Qualitative comparisons on real images and quantitative comparisons on simulated images were performed. Both qualitative and quantitative comparisons demonstrated that fuzzy logic based segmentation algorithms are superior in comparison with classical segmentation algorithms. Edge-based segmentation algorithms demonstrated the poorest performance of all; K-means and IFCM clustering algorithms performed better, and FCM demonstrated the best performance of all. However, it should be noted that IFCM was not properly evaluated due to time restrictions in code generation, testing and evaluation.Segmentation of images is a vital part of medical image processing, and MRI (Magnetic Resonance Imaging) is already recognized as a very important tool for clinical diagnosis. In this thesis, comparisons between different segmentation algorithms are carried out, specifically on brain MRI images. Initial parts of the thesis provide the background to the project, and an introduction to the basic principles of MRI, respectively, followed by parameter definitions and MRI image artifacts. The next part briefly covers various image pre-processing techniques which are required, and this is followed with a review of the major segmentation techniques which are available, including thresholding, region growing, clustering, and K-Means clustering. The concept of fuzzy logic is also introduced here, and the chapter concludes with a discussion of fuzzy logic based segmentation algorithms such as Fuzzy C-Means (FCM) and Improved Fuzzy C-Means (IFCM) clustering algorithms. The following part provides details concerning the source, type and parameters of the data (images) used for this thesis. Evaluation and inter-comparisons between a number of different segmentation algorithms are given in near concluding part, finally, conclusions and suggestions for future research are provided in last part. Qualitative comparisons on real images and quantitative comparisons on simulated images were performed. Both qualitative and quantitative comparisons demonstrated that fuzzy logic based segmentation algorithms are superior in comparison with classical segmentation algorithms. Edge-based segmentation algorithms demonstrated the poorest performance of all; K-means and IFCM clustering algorithms performed better, and FCM demonstrated the best performance of all. However, it should be noted that IFCM was not properly evaluated due to time restrictions in code generation, testing and evaluation

Finding structure in language

Since the Chomskian revolution, it has become apparent that natural language is richly structured, being naturally represented hierarchically, and requiring complex context sensitive rules to define regularities over these representations. It is widely assumed that the richness of the posited structure has strong nativist implications for mechanisms which might learn natural language, since it seemed unlikely that such structures could be derived directly from the observation of linguistic data (Chomsky 1965).This thesis investigates the hypothesis that simple statistics of a large, noisy, unlabelled corpus of natural language can be exploited to discover some of the structure which exists in natural language automatically. The strategy is to initially assume no knowledge of the structures present in natural language, save that they might be found by analysing statistical regularities which pertain between a word and the words which typically surround it in the corpus.To achieve this, various statistical methods are applied to define similarity between statistical distributions, and to infer a structure for a domain given knowledge of the similarities which pertain within it. Using these tools, it is shown that it is possible to form a hierarchical classification of many domains, including words in natural language. When this is done, it is shown that all the major syntactic categories can be obtained, and the classification is both relatively complete, and very much in accord with a standard linguistic conception of how words are classified in natural language.Once this has been done, the categorisation derived is used as the basis of a similar classification of short sequences of words. If these are analysed in a similar way, then several syntactic categories can be derived. These include simple noun phrases, various tensed forms of verbs, and simple prepositional phrases. Once this has been done, the same technique can be applied one level higher, and at this level simple sentences and verb phrases, as well as more complicated noun phrases and prepositional phrases, are shown to be derivable

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task