Level set segmentation using non-negative matrix factorization with application to brain MRI

We address the problem of image segmentation using a new deformable model based on the level set method (LSM) and non-negative matrix factorization (NMF). We describe the use of NMF to reduce the dimension of large images from thousands of pixels to a handful of metapixels or regions. In addition, the exact number of regions is discovered using the nuclear norm of the NMF factors. The proposed NMF-LSM characterizes the histogram of the image, calculated over the image blocks, as nonnegative combinations of basic histograms computed using NMF (V ~ W H). The matrix W represents the histograms of the image regions, whereas the matrix H provides the spatial clustering of the regions. NMF-LSM takes into account the bias field present particularly in medical images. We define two local clustering criteria in terms of the NMF factors. The first criterion defines a local intensity clustering property based on the matrix W by computing the average intensity and standard deviation of every region. The second criterion defines a local spatial clustering using the matrix H. The local clustering is then summed over all regions to give a global criterion of image segmentation. In LSM, these criteria define an energy minimized w.r.t. LSFs and the bias field to achieve the segmentation. The proposed method is validated on synthetic binary and gray-scale images, and then applied to real brain MRI images. NMF-LSM provides a general approach for robust region discovery and segmentation in heterogeneous images

Failure Detection in Deep Neural Networks for Medical Imaging

Deep neural networks (DNNs) have started to find their role in the modern healthcare system. DNNs are being developed for diagnosis, prognosis, treatment planning, and outcome prediction for various diseases. With the increasing number of applications of DNNs in modern healthcare, their trustworthiness and reliability are becoming increasingly important. An essential aspect of trustworthiness is detecting the performance degradation and failure of deployed DNNs in medical settings. The softmax output values produced by DNNs are not a calibrated measure of model confidence. Softmax probability numbers are generally higher than the actual model confidence. The model confidence-accuracy gap further increases for wrong predictions and noisy inputs. We employ recently proposed Bayesian deep neural networks (BDNNs) to learn uncertainty in the model parameters. These models simultaneously output the predictions and a measure of confidence in the predictions. By testing these models under various noisy conditions, we show that the (learned) predictive confidence is well calibrated. We use these reliable confidence values for monitoring performance degradation and failure detection in DNNs. We propose two different failure detection methods. In the first method, we define a fixed threshold value based on the behavior of the predictive confidence with changing signal-to-noise ratio (SNR) of the test dataset. The second method learns the threshold value with a neural network. The proposed failure detection mechanisms seamlessly abstain from making decisions when the confidence of the BDNN is below the defined threshold and hold the decision for manual review. Resultantly, the accuracy of the models improves on the unseen test samples. We tested our proposed approach on three medical imaging datasets: PathMNIST, DermaMNIST, and OrganAMNIST, under different levels and types of noise. An increase in the noise of the test images increases the number of abstained samples. BDNNs are inherently robust and show more than 10% accuracy improvement with the proposed failure detection methods. The increased number of abstained samples or an abrupt increase in the predictive variance indicates model performance degradation or possible failure. Our work has the potential to improve the trustworthiness of DNNs and enhance user confidence in the model predictions

Robust Learning via Ensemble Density Propagation in Deep Neural Networks

Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks

Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks

With the rise of deep neural networks, the challenge of explaining the predictions of these networks has become increasingly recognized. While many methods for explaining the decisions of deep neural networks exist, there is currently no consensus on how to evaluate them. On the other hand, robustness is a popular topic for deep learning research; however, it is hardly talked about in explainability until very recently. In this tutorial paper, we start by presenting gradient-based interpretability methods. These techniques use gradient signals to assign the burden of the decision on the input features. Later, we discuss how gradient-based methods can be evaluated for their robustness and the role that adversarial robustness plays in having meaningful explanations. We also discuss the limitations of gradient-based methods. Finally, we present the best practices and attributes that should be examined before choosing an explainability method. We conclude with the future directions for research in the area at the convergence of robustness and explainability

Trustworthy Medical Segmentation with Uncertainty Estimation

Deep Learning (DL) holds great promise in reshaping the healthcare systems given its precision, efficiency, and objectivity. However, the brittleness of DL models to noisy and out-of-distribution inputs is ailing their deployment in the clinic. Most systems produce point estimates without further information about model uncertainty or confidence. This paper introduces a new Bayesian deep learning framework for uncertainty quantification in segmentation neural networks, specifically encoder-decoder architectures. The proposed framework uses the first-order Taylor series approximation to propagate and learn the first two moments (mean and covariance) of the distribution of the model parameters given the training data by maximizing the evidence lower bound. The output consists of two maps: the segmented image and the uncertainty map of the segmentation. The uncertainty in the segmentation decisions is captured by the covariance matrix of the predictive distribution. We evaluate the proposed framework on medical image segmentation data from Magnetic Resonances Imaging and Computed Tomography scans. Our experiments on multiple benchmark datasets demonstrate that the proposed framework is more robust to noise and adversarial attacks as compared to state-of-the-art segmentation models. Moreover, the uncertainty map of the proposed framework associates low confidence (or equivalently high uncertainty) to patches in the test input images that are corrupted with noise, artifacts or adversarial attacks. Thus, the model can self-assess its segmentation decisions when it makes an erroneous prediction or misses part of the segmentation structures, e.g., tumor, by presenting higher values in the uncertainty map

PremiUm-CNN: Propagating Uncertainty Towards Robust Convolutional Neural Networks

Deep neural networks (DNNs) have surpassed human-level accuracy in various learning tasks. However, unlike humans who have a natural cognitive intuition for probabilities, DNNs cannot express their uncertainty in the output decisions. This limits the deployment of DNNs in mission-critical domains, such as warfighter decision-making or medical diagnosis. Bayesian inference provides a principled approach to reason about model\u27s uncertainty by estimating the posterior distribution of the unknown parameters. The challenge in DNNs remains the multi-layer stages of non-linearities, which make the propagation of high-dimensional distributions mathematically intractable. This paper establishes the theoretical and algorithmic foundations of uncertainty or belief propagation by developing new deep learning models named PremiUm-CNNs (Propagating Uncertainty in Convolutional Neural Networks). We introduce a tensor normal distribution as a prior over convolutional kernels and estimate the variational posterior by maximizing the evidence lower bound (ELBO). We start by deriving the first-order mean-covariance propagation framework. Later, we develop a framework based on the unscented transformation (correct at least up to the second-order) that propagates sigma points of the variational distribution through layers of a CNN. The propagated covariance of the predictive distribution captures uncertainty in the output decision. Comprehensive experiments conducted on diverse benchmark datasets demonstrate: 1) superior robustness against noise and adversarial attacks, 2) self-assessment through predictive uncertainty that increases quickly with increasing levels of noise or attacks, and 3) an ability to detect a targeted attack from ambient noise

SUPER-Net: Trustworthy Medical Image Segmentation with Uncertainty Propagation in Encoder-Decoder Networks

Deep Learning (DL) holds great promise in reshaping the healthcare industry owing to its precision, efficiency, and objectivity. However, the brittleness of DL models to noisy and out-of-distribution inputs is ailing their deployment in the clinic. Most models produce point estimates without further information about model uncertainty or confidence. This paper introduces a new Bayesian DL framework for uncertainty quantification in segmentation neural networks: SUPER-Net: trustworthy medical image Segmentation with Uncertainty Propagation in Encoder-decodeR Networks. SUPER-Net analytically propagates, using Taylor series approximations, the first two moments (mean and covariance) of the posterior distribution of the model parameters across the nonlinear layers. In particular, SUPER-Net simultaneously learns the mean and covariance without expensive post-hoc Monte Carlo sampling or model ensembling. The output consists of two simultaneous maps: the segmented image and its pixelwise uncertainty map, which corresponds to the covariance matrix of the predictive distribution. We conduct an extensive evaluation of SUPER-Net on medical image segmentation of Magnetic Resonances Imaging and Computed Tomography scans under various noisy and adversarial conditions. Our experiments on multiple benchmark datasets demonstrate that SUPER-Net is more robust to noise and adversarial attacks than state-of-the-art segmentation models. Moreover, the uncertainty map of the proposed SUPER-Net associates low confidence (or equivalently high uncertainty) to patches in the test input images that are corrupted with noise, artifacts, or adversarial attacks. Perhaps more importantly, the model exhibits the ability of self-assessment of its segmentation decisions, notably when making erroneous predictions due to noise or adversarial examples

Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks

With the rise of deep neural networks, the challenge of explaining the predictions of these networks has become increasingly recognized. While many methods for explaining the decisions of deep neural networks exist, there is currently no consensus on how to evaluate them. On the other hand, robustness is a popular topic for deep learning research; however, it is hardly talked about in explainability until very recently. In this tutorial paper, we start by presenting gradient-based interpretability methods. These techniques use gradient signals to assign the burden of the decision on the input features. Later, we discuss how gradient-based methods can be evaluated for their robustness and the role that adversarial robustness plays in having meaningful explanations. We also discuss the limitations of gradient-based methods. Finally, we present the best practices and attributes that should be examined before choosing an explainability method. We conclude with the future directions for research in the area at the convergence of robustness and explainability.Comment: 23 pages, 4 figure

Towards machine self-awareness - A Bayesian framework for uncertainty propagation in deep neural networks

Deep neural networks (DNNs) have surpassed human-level accuracy in various fields, including object recognition and classification. However, DNNs being inherently deterministic, are unable to evaluate their confidence in the decisions. Bayesian inference provides a principled approach to reason about model confidence or uncertainty by estimating the posterior distribution of the unknown parameters. The challenge in DNNs is the multi-layer stages of non-linearities, which makes propagation of high-dimensional distributions mathematically intractable. This dissertation establishes the theoretical and algorithmic foundations of uncertainty or belief propagation by developing new deep learning models that can quantify their uncertainty in the decision and self-assess their performance. We introduce Tensor Normal distributions as priors over the network parameters and derive a first-order Taylor series mean-covariance propagation framework. We subsequently extend the first-order approximation to an unscented framework that propagates sigma points through the model layers and is shown to be accurate to at least the second-order approximation. The uncertainty in the output decision is given by the propagated covariance of the predictive distribution. Experiments on benchmark datasets demonstrate: 1) superior robustness against Gaussian noise and adversarial attacks; 2) self-assessment through predictive confidence that decreases quickly with increasing levels of ambient noise or attack; and 3) an ability to detect a targeted attack from ambient noise

Robust Software Vulnerability Detection Using Bayesian Gated Recurrent Unit

Software systems are prone to source code vulnerabilities, resulting in deadlock, hacking, information leakage, and system failure. It is crucial to spot the security holes and identify the vulnerable software components prior to such encounters to prevent any security breach or system crash. While conventional machine learning and neural network-based vulnerability detection techniques have been reported in the literature, it is still challenging to determine the trustworthiness of these models. In this study, we develop a robust approach for detecting software vulnerabilities using a Bayesian gated recurrent unit. The proposed model identifies source code vulnerability while simultaneously learning the output prediction uncertainty. In the Bayesian framework, the Gaussian distribution is introduced as a prior distribution over the network parameters, which are then propagated through the network layers and activation functions. At the network output, the mean of the predictive distribution indicates the predicted vulnerability in source codes, while the covariance matrix reflects the uncertainty in predicted vulnerability. To evaluate the robustness of the model, we conducted rigorous experiments with 1.27 million source codes in C/C++. We reported the performance for five types of Common vulnerabilities encountered under different levels of Gaussian noise and adversarial attacks and compared the results with the state-of-the-art methods. The training and testing were done on the HPC resources in The Texas Advanced Computing Center (TACC), particularly on the lonestar6 supercomputer. This work is under the TACC project "TRUST- TRustworthy Uncertainty Propagation for Sequential Time-Series Analysis". By submitting multiple jobs to the queue, we could complete significant simulations within a reasonable period. The investigations have confirmed that the proposed model produces a significantly higher uncertainty when encountered with high levels of natural noise or adversarial attacks and could be used as a measure to alert human users in many high-stake applications.Texas Advanced Computing Center (TACC