3,982 research outputs found
Analyzing the Quality and Challenges of Uncertainty Estimations for Brain Tumor Segmentation
Automatic segmentation of brain tumors has the potential to enable volumetric measures and high-throughput analysis in the clinical setting. Reaching this potential seems almost achieved, considering the steady increase in segmentation accuracy. However, despite segmentation accuracy, the current methods still do not meet the robustness levels required for patient-centered clinical use. In this regard, uncertainty estimates are a promising direction to improve the robustness of automated segmentation systems. Different uncertainty estimation methods have been proposed, but little is known about their usefulness and limitations for brain tumor segmentation. In this study, we present an analysis of the most commonly used uncertainty estimation methods in regards to benefits and challenges for brain tumor segmentation. We evaluated their quality in terms of calibration, segmentation error localization, and segmentation failure detection. Our results show that the uncertainty methods are typically well-calibrated when evaluated at the dataset level. Evaluated at the subject level, we found notable miscalibrations and limited segmentation error localization (e.g., for correcting segmentations), which hinder the direct use of the voxel-wise uncertainties. Nevertheless, voxel-wise uncertainty showed value to detect failed segmentations when uncertainty estimates are aggregated at the subject level. Therefore, we suggest a careful usage of voxel-wise uncertainty measures and highlight the importance of developing solutions that address the subject-level requirements on calibration and segmentation error localization
Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis
The full acceptance of Deep Learning (DL) models in the clinical field is
rather low with respect to the quantity of high-performing solutions reported
in the literature. Particularly, end users are reluctant to rely on the rough
predictions of DL models. Uncertainty quantification methods have been proposed
in the literature as a potential response to reduce the rough decision provided
by the DL black box and thus increase the interpretability and the
acceptability of the result by the final user. In this review, we propose an
overview of the existing methods to quantify uncertainty associated to DL
predictions. We focus on applications to medical image analysis, which present
specific challenges due to the high dimensionality of images and their quality
variability, as well as constraints associated to real-life clinical routine.
We then discuss the evaluation protocols to validate the relevance of
uncertainty estimates. Finally, we highlight the open challenges of uncertainty
quantification in the medical field
Improving Calibration and Out-of-Distribution Detection in Medical Image Segmentation with Convolutional Neural Networks
Convolutional Neural Networks (CNNs) have shown to be powerful medical image
segmentation models. In this study, we address some of the main unresolved
issues regarding these models. Specifically, training of these models on small
medical image datasets is still challenging, with many studies promoting
techniques such as transfer learning. Moreover, these models are infamous for
producing over-confident predictions and for failing silently when presented
with out-of-distribution (OOD) data at test time. In this paper, we advocate
for multi-task learning, i.e., training a single model on several different
datasets, spanning several different organs of interest and different imaging
modalities. We show that not only a single CNN learns to automatically
recognize the context and accurately segment the organ of interest in each
context, but also that such a joint model often has more accurate and
better-calibrated predictions than dedicated models trained separately on each
dataset. Our experiments show that multi-task learning can outperform transfer
learning in medical image segmentation tasks. For detecting OOD data, we
propose a method based on spectral analysis of CNN feature maps. We show that
different datasets, representing different imaging modalities and/or different
organs of interest, have distinct spectral signatures, which can be used to
identify whether or not a test image is similar to the images used to train a
model. We show that this approach is far more accurate than OOD detection based
on prediction uncertainty. The methods proposed in this paper contribute
significantly to improving the accuracy and reliability of CNN-based medical
image segmentation models
- …