Visual Feature Attribution using Wasserstein GANs

Attributing the pixels of an input image to a certain category is an important and well-studied problem in computer vision, with applications ranging from weakly supervised localisation to understanding hidden effects in the data. In recent years, approaches based on interpreting a previously trained neural network classifier have become the de facto state-of-the-art and are commonly used on medical as well as natural image datasets. In this paper, we discuss a limitation of these approaches which may lead to only a subset of the category specific features being detected. To address this problem we develop a novel feature attribution technique based on Wasserstein Generative Adversarial Networks (WGAN), which does not suffer from this limitation. We show that our proposed method performs substantially better than the state-of-the-art for visual attribution on a synthetic dataset and on real 3D neuroimaging data from patients with mild cognitive impairment (MCI) and Alzheimer's disease (AD). For AD patients the method produces compellingly realistic disease effect maps which are very close to the observed effects.Comment: Accepted to CVPR 201

AI-driven spatio-temporal engine for finding gravitationally lensed supernovae

We present a spatio-temporal AI framework that concurrently exploits both the spatial and time-variable features of gravitationally lensed supernovae in optical images to ultimately aid in the discovery of such exotic transients in wide-field surveys. Our spatio-temporal engine is designed using recurrent convolutional layers, while drawing from recent advances in variational inference to quantify approximate Bayesian uncertainties via a confidence score. Using simulated Young Supernova Experiment (YSE) images as a showcase, we find that the use of time-series images yields a substantial gain of nearly 20 per cent in classification accuracy over single-epoch observations, with a preliminary application to mock observations from the Legacy Survey of Space and Time (LSST) yielding around 99 per cent accuracy. Our innovative deep learning machinery adds an extra dimension in the search for gravitationally lensed supernovae from current and future astrophysical transient surveys.Comment: 6+8 pages, 10 figures, 2 tables. For submission to a peer-reviewed journal. Comments welcom

Deep learning for fast and robust medical image reconstruction and analysis

Medical imaging is an indispensable component of modern medical research as well as clinical practice. Nevertheless, imaging techniques such as magnetic resonance imaging (MRI) and computational tomography (CT) are costly and are less accessible to the majority of the world. To make medical devices more accessible, affordable and efficient, it is crucial to re-calibrate our current imaging paradigm for smarter imaging. In particular, as medical imaging techniques have highly structured forms in the way they acquire data, they provide us with an opportunity to optimise the imaging techniques holistically by leveraging data. The central theme of this thesis is to explore different opportunities where we can exploit data and deep learning to improve the way we extract information for better, faster and smarter imaging. This thesis explores three distinct problems. The first problem is the time-consuming nature of dynamic MR data acquisition and reconstruction. We propose deep learning methods for accelerated dynamic MR image reconstruction, resulting in up to 10-fold reduction in imaging time. The second problem is the redundancy in our current imaging pipeline. Traditionally, imaging pipeline treated acquisition, reconstruction and analysis as separate steps. However, we argue that one can approach them holistically and optimise the entire pipeline jointly for a specific target goal. To this end, we propose deep learning approaches for obtaining high fidelity cardiac MR segmentation directly from significantly undersampled data, greatly exceeding the undersampling limit for image reconstruction. The final part of this thesis tackles the problem of interpretability of the deep learning algorithms. We propose attention-models that can implicitly focus on salient regions in an image to improve accuracy for ultrasound scan plane detection and CT segmentation. More crucially, these models can provide explainability, which is a crucial stepping stone for the harmonisation of smart imaging and current clinical practice.Open Acces

Segmentation of Medical Images with Adaptable Multifunctional Discretization Bayesian Neural Networks and Gaussian Operations

Bayesian statistics is incorporated into a neural network to create a Bayesian neural network (BNN) that adds posterior inference aims at preventing overfitting. BNNs are frequently used in medical image segmentation because they provide a stochastic viewpoint of segmentation approaches by producing a posterior probability with conventional limitations and allowing the depiction of uncertainty over following distributions. However, the actual efficacy of BNNs is constrained by the difficulty in selecting expressive discretization and accepting suitable following disseminations in a higher-order domain. Functional discretization BNN using Gaussian processes (GPs) that analyze medical image segmentation is proposed in this paper. Here, a discretization inference has been assumed in the functional domain by considering the former and dynamic consequent distributions to be GPs. An upsampling operator that utilizes a content-based feature extraction has been proposed. This is an adaptive method for extracting features after feature mapping is used in conjunction with the functional evidence lower bound and weights. This results in a loss-aware segmentation network that achieves an F1-score of 91.54%, accuracy of 90.24%, specificity of 88.54%, and precision of 80.24%

Electrical Impedance Tomography: A Fair Comparative Study on Deep Learning and Analytic-based Approaches

Electrical Impedance Tomography (EIT) is a powerful imaging technique with diverse applications, e.g., medical diagnosis, industrial monitoring, and environmental studies. The EIT inverse problem is about inferring the internal conductivity distribution of an object from measurements taken on its boundary. It is severely ill-posed, necessitating advanced computational methods for accurate image reconstructions. Recent years have witnessed significant progress, driven by innovations in analytic-based approaches and deep learning. This review explores techniques for solving the EIT inverse problem, focusing on the interplay between contemporary deep learning-based strategies and classical analytic-based methods. Four state-of-the-art deep learning algorithms are rigorously examined, harnessing the representational capabilities of deep neural networks to reconstruct intricate conductivity distributions. In parallel, two analytic-based methods, rooted in mathematical formulations and regularisation techniques, are dissected for their strengths and limitations. These methodologies are evaluated through various numerical experiments, encompassing diverse scenarios that reflect real-world complexities. A suite of performance metrics is employed to assess the efficacy of these methods. These metrics collectively provide a nuanced understanding of the methods' ability to capture essential features and delineate complex conductivity patterns. One novel feature of the study is the incorporation of variable conductivity scenarios, introducing a level of heterogeneity that mimics textured inclusions. This departure from uniform conductivity assumptions mimics realistic scenarios where tissues or materials exhibit spatially varying electrical properties. Exploring how each method responds to such variable conductivity scenarios opens avenues for understanding their robustness and adaptability

On instabilities of deep learning in image reconstruction - Does AI come at a cost?

Deep learning, due to its unprecedented success in tasks such as image classification, has emerged as a new tool in image reconstruction with potential to change the field. In this paper we demonstrate a crucial phenomenon: deep learning typically yields unstablemethods for image reconstruction. The instabilities usually occur in several forms: (1) tiny, almost undetectable perturbations, both in the image and sampling domain, may result in severe artefacts in the reconstruction, (2) a small structural change, for example a tumour, may not be captured in the reconstructed image and (3) (a counterintuitive type of instability) more samples may yield poorer performance. Our new stability test with algorithms and easy to use software detects the instability phenomena. The test is aimed at researchers to test their networks for instabilities and for government agencies, such as the Food and Drug Administration (FDA), to secure safe use of deep learning methods

Uncertainty Quantifcation in Vision Based Classifcation

The past decade of artifcial intelligence and deep learning has made tremendous progress in highly perceptive tasks such as image recognition. Deep learning algorithms map high dimensional complex representations to low dimensional array mappings. However, these mappings are generally blindly assumed to be correct, further justifed with high accuracies on trending datasets. The challenge of creating a comprehensive, explainable and reasonable deep learning system is yet to be solved. One way to deal with this is by using uncertainty quantifcation, or uncertainty aware learning, with the help of Bayesian methods. This thesis contributes to the feld of uncertainty aware learning by demonstrating how uncertainty can be used to recover performance in case of a physical attack, how uncertainty can be used to improve sensitivity to noise and how it can be used to improve performance on dynamic datasets. The frst contribution involves learning from model uncertainty in the application of deep learning-based semantic segmentation. The second contribution deals with robustness and sensitivity analysis in image classifcation and fnally, the third contribution in continual learning by using variance to update the learning rate. The frst contribution proposes the architecture AdvSegNet which aims to improve the performance of Bayesian SegNet. In the second contribution, a combined architecture of convolutional network feature extractor and a Gaussian process (CNN-GP) is made to classify images under uncertain conditions including noise, blurring and adversarial attacks. Finally, in the continual learning subject area, the architecture CNN-GP is trained on datasets presented sequentially. Results show an improvement in performance and sensitivity to adversarial attack and noisy conditions as well as an improvement in dynamic datasets with a small number of tasks

Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation

We propose a dual pathway, 11-layers deep, three-dimensional Convolutional Neural Network for the challenging task of brain lesion segmentation. The devised architecture is the result of an in-depth analysis of the limitations of current networks proposed for similar applications. To overcome the computational burden of processing 3D medical scans, we have devised an efficient and effective dense training scheme which joins the processing of adjacent image patches into one pass through the network while automatically adapting to the inherent class imbalance present in the data. Further, we analyze the development of deeper, thus more discriminative 3D CNNs. In order to incorporate both local and larger contextual information, we employ a dual pathway architecture that processes the input images at multiple scales simultaneously. For post-processing of the networks soft segmentation, we use a 3D fully connected Conditional Random Field which effectively removes false positives. Our pipeline is extensively evaluated on three challenging tasks of lesion segmentation in multi-channel MRI patient data with traumatic brain injuries, brain tumors, and ischemic stroke. We improve on the state-of-the-art for all three applications, with top ranking performance on the public benchmarks BRATS 2015 and ISLES 2015. Our method is computationally efficient, which allows its adoption in a variety of research and clinical settings. The source code of our implementation is made publicly available

Understanding and mitigating universal adversarial perturbations for computer vision neural networks

Deep neural networks (DNNs) have become the algorithm of choice for many computer vision applications. They are able to achieve human level performance in many computer vision tasks, and enable the automation and large-scale deployment of applications such as object tracking, autonomous vehicles, and medical imaging. However, DNNs expose software applications to systemic vulnerabilities in the form of Universal Adversarial Perturbations (UAPs): input perturbation attacks that can cause DNNs to make classification errors on large sets of inputs. Our aim is to improve the robustness of computer vision DNNs to UAPs without sacrificing the models' predictive performance. To this end, we increase our understanding of these vulnerabilities by investigating the visual structures and patterns commonly appearing in UAPs. We demonstrate the efficacy and pervasiveness of UAPs by showing how Procedural Noise patterns can be used to generate efficient zero-knowledge attacks for different computer vision models and tasks at minimal cost to the attacker. We then evaluate the UAP robustness of various shape and texture-biased models, and found that applying them in ensembles provides marginal improvement to robustness. To mitigate UAP attacks, we develop two novel approaches. First, we propose the Jacobian of DNNs to measure the sensitivity of computer vision DNNs. We derive theoretical bounds and provide empirical evidence that shows how a combination of Jacobian regularisation and ensemble methods allow for increased model robustness against UAPs without degrading the predictive performance of computer vision DNNs. Our results evince a robustness-accuracy trade-off against UAPs that is better than those of models trained in conventional ways. Finally, we design a detection method that analyses the hidden layer activation values to identify a variety of UAP attacks in real-time with low-latency. We show that our work outperforms existing defences under realistic time and computation constraints.Open Acces