Reasoning with Uncertainty in Deep Learning for Safer Medical Image Computing

Deep learning is now ubiquitous in the research field of medical image computing. As such technologies progress towards clinical translation, the question of safety becomes critical. Once deployed, machine learning systems unavoidably face situations where the correct decision or prediction is ambiguous. However, the current methods disproportionately rely on deterministic algorithms, lacking a mechanism to represent and manipulate uncertainty. In safety-critical applications such as medical imaging, reasoning under uncertainty is crucial for developing a reliable decision making system. Probabilistic machine learning provides a natural framework to quantify the degree of uncertainty over different variables of interest, be it the prediction, the model parameters and structures, or the underlying data (images and labels). Probability distributions are used to represent all the uncertain unobserved quantities in a model and how they relate to the data, and probability theory is used as a language to compute and manipulate these distributions. In this thesis, we explore probabilistic modelling as a framework to integrate uncertainty information into deep learning models, and demonstrate its utility in various high-dimensional medical imaging applications. In the process, we make several fundamental enhancements to current methods. We categorise our contributions into three groups according to the types of uncertainties being modelled: (i) predictive; (ii) structural and (iii) human uncertainty. Firstly, we discuss the importance of quantifying predictive uncertainty and understanding its sources for developing a risk-averse and transparent medical image enhancement application. We demonstrate how a measure of predictive uncertainty can be used as a proxy for the predictive accuracy in the absence of ground-truths. Furthermore, assuming the structure of the model is flexible enough for the task, we introduce a way to decompose the predictive uncertainty into its orthogonal sources i.e. aleatoric and parameter uncertainty. We show the potential utility of such decoupling in providing a quantitative “explanations” into the model performance. Secondly, we introduce our recent attempts at learning model structures directly from data. One work proposes a method based on variational inference to learn a posterior distribution over connectivity structures within a neural network architecture for multi-task learning, and share some preliminary results in the MR-only radiotherapy planning application. Another work explores how the training algorithm of decision trees could be extended to grow the architecture of a neural network to adapt to the given availability of data and the complexity of the task. Lastly, we develop methods to model the “measurement noise” (e.g., biases and skill levels) of human annotators, and integrate this information into the learning process of the neural network classifier. In particular, we show that explicitly modelling the uncertainty involved in the annotation process not only leads to an improvement in robustness to label noise, but also yields useful insights into the patterns of errors that characterise individual experts

Multi-atlas segmentation using clustering, local non-linear manifold embeddings and target-specific templates

Multi-atlas segmentation (MAS) has become an established technique for the automated delineation of anatomical structures. The often manually annotated labels from each of multiple pre-segmented images (atlases) are typically transferred to a target through the spatial mapping of corresponding structures of interest. The mapping can be estimated by pairwise registration between each atlas and the target or by creating an intermediate population template for spatial normalisation of atlases and targets. The former is done at runtime which is computationally expensive but provides high accuracy. In the latter approach the template can be constructed from the atlases offline requiring only one registration to the target at runtime. Although this is computationally more efficient, the composition of deformation fields can lead to decreased accuracy. Our goal was to develop a MAS method which was both efficient and accurate. In our approach we create a target-specific template (TST) which has a high similarity to the target and serves as intermediate step to increase registration accuracy. The TST is constructed from the atlas images that are most similar to the target. These images are determined in low-dimensional manifold spaces on the basis of deformation fields in local regions of interest. We also introduce a clustering approach to divide atlas labels into meaningful sub-regions of interest and increase local specificity for TST construction and label fusion. Our approach was tested on a variety of MR brain datasets and applied to an in-house dataset. We achieve state-of-the-art accuracy while being computationally much more efficient than competing methods. This efficiency opens the door to the use of larger sets of atlases which could lead to further improvement in segmentation accuracy

Patch-based segmentation with spatial context for medical image analysis

Accurate segmentations in medical imaging form a crucial role in many applications from pa- tient diagnosis to population studies. As the amount of data generated from medical images increases, the ability to perform this task without human intervention becomes ever more de- sirable. One approach, known broadly as atlas-based segmentation, is to propagate labels from images which have already been manually labelled by clinical experts. Methods using this ap- proach have been shown to be e ective in many applications, demonstrating great potential for automatic labelling of large datasets. However, these methods usually require the use of image registration and are dependent on the outcome of the registration. Any registrations errors that occur are also propagated to the segmentation process and are likely to have an adverse e ect on segmentation accuracy. Recently, patch-based methods have been shown to allow a relaxation of the required image alignment, whilst achieving similar results. In general, these methods label each voxel of a target image by comparing the image patch centred on the voxel with neighbouring patches from an atlas library and assigning the most likely label according to the closest matches. The main contributions of this thesis focuses around this approach in providing accurate segmentation results whilst minimising the dependency on registration quality. In particular, this thesis proposes a novel kNN patch-based segmentation framework, which utilises both intensity and spatial information, and explore the use of spatial context in a diverse range of applications. The proposed methods extend the potential for patch-based segmentation to tolerate registration errors by rede ning the \locality" for patch selection and comparison, whilst also allowing similar looking patches from di erent anatomical structures to be di erentiated. The methods are evaluated on a wide variety of image datasets, ranging from the brain to the knees, demonstrating its potential with results which are competitive to state-of-the-art techniques.Open Acces

Multi-atlas segmentation using clustering, local non-linear manifold embeddings and target-specific templates

Multi-atlas segmentation (MAS) has become an established technique for the automated delineation of anatomical structures. The often manually annotated labels from each of multiple pre-segmented images (atlases) are typically transferred to a target through the spatial mapping of corresponding structures of interest. The mapping can be estimated by pairwise registration between each atlas and the target or by creating an intermediate population template for spatial normalisation of atlases and targets. The former is done at runtime which is computationally expensive but provides high accuracy. In the latter approach the template can be constructed from the atlases offline requiring only one registration to the target at runtime. Although this is computationally more efficient, the composition of deformation fields can lead to decreased accuracy. Our goal was to develop a MAS method which was both efficient and accurate. In our approach we create a target-specific template (TST) which has a high similarity to the target and serves as intermediate step to increase registration accuracy. The TST is constructed from the atlas images that are most similar to the target. These images are determined in low-dimensional manifold spaces on the basis of deformation fields in local regions of interest. We also introduce a clustering approach to divide atlas labels into meaningful sub-regions of interest and increase local specificity for TST construction and label fusion. Our approach was tested on a variety of MR brain datasets and applied to an in-house dataset. We achieve state-of-the-art accuracy while being computationally much more efficient than competing methods. This efficiency opens the door to the use of larger sets of atlases which could lead to further improvement in segmentation accuracy

Applications

Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications

Compound Models for Vision-Based Pedestrian Recognition

This thesis addresses the problem of recognizing pedestrians in video images acquired from a moving camera in real-world cluttered environments. Instead of focusing on the development of novel feature primitives or pattern classifiers, we follow an orthogonal direction and develop feature- and classifier-independent compound techniques which integrate complementary information from multiple image-based sources with the objective of improved pedestrian classification performance. After establishing a performance baseline in terms of a thorough experimental study on monocular pedestrian recognition, we investigate the use of multiple cues on module-level. A motion-based focus of attention stage is proposed based on a learned probabilistic pedestrian-specific model of motion features. The model is used to generate pedestrian localization hypotheses for subsequent shape- and texture-based classification modules. In the remainder of this work, we focus on the integration of complementary information directly into the pattern classification step. We present a combination of shape and texture information by means of pose-specific generative shape and texture models. The generative models are integrated with discriminative classification models by utilizing synthesized virtual pedestrian training samples from the former to enhance the classification performance of the latter. Both models are linked using Active Learning to guide the training process towards informative samples. A multi-level mixture-of-experts classification framework is proposed which involves local pose-specific expert classifiers operating on multiple image modalities and features. In terms of image modalities, we consider gray-level intensity, depth cues derived from dense stereo vision and motion cues arising from dense optical flow. We furthermore employ shape-based, gradient-based and texture-based features. The mixture-of-experts formulation compares favorably to joint space approaches, in view of performance and practical feasibility. Finally, we extend this mixture-of-experts framework in terms of multi-cue partial occlusion handling and the estimation of pedestrian body orientation. Our occlusion model involves examining occlusion boundaries which manifest in discontinuities in depth and motion space. Occlusion-dependent weights which relate to the visibility of certain body parts focus the decision on unoccluded body components. We further apply the pose-specific nature of our mixture-of-experts framework towards estimating the density of pedestrian body orientation from single images, again integrating shape and texture information. Throughout this work, particular emphasis is laid on thorough performance evaluation both regarding methodology and competitive real-world datasets. Several datasets used in this thesis are made publicly available for benchmarking purposes. Our results indicate significant performance boosts over state-of-the-art for all aspects considered in this thesis, i.e. pedestrian recognition, partial occlusion handling and body orientation estimation. The pedestrian recognition performance in particular is considerably advanced; false detections at constant detection rates are reduced by significantly more than an order of magnitude