604 research outputs found
Style Data Augmentation for Robust Segmentation of Multi-Modality Cardiac MRI
International audienceWe propose a data augmentation method to improve thesegmentation accuracy of the convolutional neural network on multi-modality cardiac magnetic resonance (CMR) dataset. The strategy aims to reduce over-fitting of the network toward any specific intensity or contrast of the training images by introducing diversity in these two aspects. The style data augmentation (SDA) strategy increases the size of the training dataset by using multiple image processing functions including adaptive histogram equalisation, Laplacian transformation, Sobel edge detection, intensity inversion and histogram matching. For the segmentation task, we developed the thresholded connection layer network (TCL-Net), a minimalist rendition of the U-Net architecture, which is designed to reduce convergence and computation time. We integrate the dual U-Net strategy to increase the resolution of the 3D segmentation target. Utilising these approaches on a multi-modality dataset, with SSFP and T2 weighted images as training and LGE as validation, we achieve 90% and 96% validation Dice coefficient for endocardium and epicardium segmentations. This result can be interpreted as a proof of concept for a generalised segmentation network that is robust to the quality or modality of the input images. When testing with our mono-centric LGE image dataset, the SDA method also improves the performance of the epicardium segmentation, with an increase from 87% to 90% for the single network segmentation
Domain Generalization for Medical Image Analysis: A Survey
Medical Image Analysis (MedIA) has become an essential tool in medicine and
healthcare, aiding in disease diagnosis, prognosis, and treatment planning, and
recent successes in deep learning (DL) have made significant contributions to
its advances. However, DL models for MedIA remain challenging to deploy in
real-world situations, failing for generalization under the distributional gap
between training and testing samples, known as a distribution shift problem.
Researchers have dedicated their efforts to developing various DL methods to
adapt and perform robustly on unknown and out-of-distribution data
distributions. This paper comprehensively reviews domain generalization studies
specifically tailored for MedIA. We provide a holistic view of how domain
generalization techniques interact within the broader MedIA system, going
beyond methodologies to consider the operational implications on the entire
MedIA workflow. Specifically, we categorize domain generalization methods into
data-level, feature-level, model-level, and analysis-level methods. We show how
those methods can be used in various stages of the MedIA workflow with DL
equipped from data acquisition to model prediction and analysis. Furthermore,
we include benchmark datasets and applications used to evaluate these
approaches and analyze the strengths and weaknesses of various methods,
unveiling future research opportunities
Random Style Transfer based Domain Generalization Networks Integrating Shape and Spatial Information
Deep learning (DL)-based models have demonstrated good performance in medical
image segmentation. However, the models trained on a known dataset often fail
when performed on an unseen dataset collected from different centers, vendors
and disease populations. In this work, we present a random style transfer
network to tackle the domain generalization problem for multi-vendor and center
cardiac image segmentation. Style transfer is used to generate training data
with a wider distribution/ heterogeneity, namely domain augmentation. As the
target domain could be unknown, we randomly generate a modality vector for the
target modality in the style transfer stage, to simulate the domain shift for
unknown domains. The model can be trained in a semi-supervised manner by
simultaneously optimizing a supervised segmentation and an unsupervised style
translation objective. Besides, the framework incorporates the spatial
information and shape prior of the target by introducing two regularization
terms. We evaluated the proposed framework on 40 subjects from the M\&Ms
challenge2020, and obtained promising performance in the segmentation for data
from unknown vendors and centers.Comment: 11 page
Causality-inspired single-source domain generalization for medical image segmentation
Deep learning models usually suffer from the domain shift issue, where models trained on one source domain do not generalize well to other unseen domains. In this work, we investigate the single-source domain generalization problem: training a deep network that is robust to unseen domains, under the condition that training data are only available from one source domain, which is common in medical imaging applications. We tackle this problem in the context of cross-domain medical image segmentation. In this scenario, domain shifts are mainly caused by different acquisition processes. We propose a simple causality-inspired data augmentation approach to expose a segmentation model to synthesized domain-shifted training examples. Specifically, 1) to make the deep model robust to discrepancies in image intensities and textures, we employ a family of randomly-weighted shallow networks. They augment training images using diverse appearance transformations. 2) Further we show that spurious correlations among objects in an image are detrimental to domain robustness. These correlations might be taken by the network as domain-specific clues for making predictions, and they may break on unseen domains. We remove these spurious correlations via causal intervention. This is achieved by resampling the appearances of potentially correlated objects independently. The proposed approach is validated on three cross-domain segmentation scenarios: cross-modality (CT-MRI) abdominal image segmentation, cross-sequence (bSSFP-LGE) cardiac MRI segmentation, and cross-site prostate MRI segmentation. The proposed approach yields consistent performance gains compared with competitive methods when tested on unseen domains
Generalizable deep learning based medical image segmentation
Deep learning is revolutionizing medical image analysis and interpretation. However, its real-world deployment is often hindered by the poor generalization to unseen domains (new imaging modalities and protocols). This lack of generalization ability is further exacerbated by the scarcity of labeled datasets for training: Data collection and annotation can be prohibitively expensive in terms of labor and costs because label quality heavily dependents on the expertise of radiologists. Additionally, unreliable predictions caused by poor model generalization pose safety risks to clinical downstream applications.
To mitigate labeling requirements, we investigate and develop a series of techniques to strengthen the generalization ability and the data efficiency of deep medical image computing models. We further improve model accountability and identify unreliable predictions made on out-of-domain data, by designing probability calibration techniques.
In the first and the second part of thesis, we discuss two types of problems for handling unexpected domains: unsupervised domain adaptation and single-source domain generalization. For domain adaptation we present a data-efficient technique that adapts a segmentation model trained on a labeled source domain (e.g., MRI) to an unlabeled target domain (e.g., CT), using a small number of unlabeled training images from the target domain.
For domain generalization, we focus on both image reconstruction and segmentation. For image reconstruction, we design a simple and effective domain generalization technique for cross-domain MRI reconstruction, by reusing image representations learned from natural image datasets. For image segmentation, we perform causal analysis of the challenging cross-domain image segmentation problem. Guided by this causal analysis we propose an effective data-augmentation-based generalization technique for single-source domains. The proposed method outperforms existing approaches on a large variety of cross-domain image segmentation scenarios.
In the third part of the thesis, we present a novel self-supervised method for learning generic image representations that can be used to analyze unexpected objects of interest. The proposed method is designed together with a novel few-shot image segmentation framework that can segment unseen objects of interest by taking only a few labeled examples as references. Superior flexibility over conventional fully-supervised models is demonstrated by our few-shot framework: it does not require any fine-tuning on novel objects of interest. We further build a publicly available comprehensive evaluation environment for few-shot medical image segmentation.
In the fourth part of the thesis, we present a novel probability calibration model. To ensure safety in clinical settings, a deep model is expected to be able to alert human radiologists if it has low confidence, especially when confronted with out-of-domain data. To this end we present a plug-and-play model to calibrate prediction probabilities on out-of-domain data. It aligns the prediction probability in line with the actual accuracy on the test data. We evaluate our method on both artifact-corrupted images and images from an unforeseen MRI scanning protocol. Our method demonstrates improved calibration accuracy compared with the state-of-the-art method.
Finally, we summarize the major contributions and limitations of our works. We also suggest future research directions that will benefit from the works in this thesis.Open Acces
Causality-inspired Single-source Domain Generalization for Medical Image Segmentation
Deep learning models usually suffer from domain shift issues, where models
trained on one source domain do not generalize well to other unseen domains. In
this work, we investigate the single-source domain generalization problem:
training a deep network that is robust to unseen domains, under the condition
that training data is only available from one source domain, which is common in
medical imaging applications. We tackle this problem in the context of
cross-domain medical image segmentation. Under this scenario, domain shifts are
mainly caused by different acquisition processes. We propose a simple
causality-inspired data augmentation approach to expose a segmentation model to
synthesized domain-shifted training examples. Specifically, 1) to make the deep
model robust to discrepancies in image intensities and textures, we employ a
family of randomly-weighted shallow networks. They augment training images
using diverse appearance transformations. 2) Further we show that spurious
correlations among objects in an image are detrimental to domain robustness.
These correlations might be taken by the network as domain-specific clues for
making predictions, and they may break on unseen domains. We remove these
spurious correlations via causal intervention. This is achieved by resampling
the appearances of potentially correlated objects independently. The proposed
approach is validated on three cross-domain segmentation tasks: cross-modality
(CT-MRI) abdominal image segmentation, cross-sequence (bSSFP-LGE) cardiac MRI
segmentation, and cross-center prostate MRI segmentation. The proposed approach
yields consistent performance gains compared with competitive methods when
tested on unseen domains.Comment: Preprin
Disentangled Representations for Domain-generalized Cardiac Segmentation
Robust cardiac image segmentation is still an open challenge due to the
inability of the existing methods to achieve satisfactory performance on unseen
data of different domains. Since the acquisition and annotation of medical data
are costly and time-consuming, recent work focuses on domain adaptation and
generalization to bridge the gap between data from different populations and
scanners. In this paper, we propose two data augmentation methods that focus on
improving the domain adaptation and generalization abilities of
state-to-the-art cardiac segmentation models. In particular, our "Resolution
Augmentation" method generates more diverse data by rescaling images to
different resolutions within a range spanning different scanner protocols.
Subsequently, our "Factor-based Augmentation" method generates more diverse
data by projecting the original samples onto disentangled latent spaces, and
combining the learned anatomy and modality factors from different domains. Our
extensive experiments demonstrate the importance of efficient adaptation
between seen and unseen domains, as well as model generalization ability, to
robust cardiac image segmentation.Comment: Accepted by STACOM 202
XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms
Generative adversarial networks (GANs) have provided promising data
enrichment solutions by synthesizing high-fidelity images. However, generating
large sets of labeled images with new anatomical variations remains unexplored.
We propose a novel method for synthesizing cardiac magnetic resonance (CMR)
images on a population of virtual subjects with a large anatomical variation,
introduced using the 4D eXtended Cardiac and Torso (XCAT) computerized human
phantom. We investigate two conditional image synthesis approaches grounded on
a semantically-consistent mask-guided image generation technique: 4-class and
8-class XCAT-GANs. The 4-class technique relies on only the annotations of the
heart; while the 8-class technique employs a predicted multi-tissue label map
of the heart-surrounding organs and provides better guidance for our
conditional image synthesis. For both techniques, we train our conditional
XCAT-GAN with real images paired with corresponding labels and subsequently at
the inference time, we substitute the labels with the XCAT derived ones.
Therefore, the trained network accurately transfers the tissue-specific
textures to the new label maps. By creating 33 virtual subjects of synthetic
CMR images at the end-diastolic and end-systolic phases, we evaluate the
usefulness of such data in the downstream cardiac cavity segmentation task
under different augmentation strategies. Results demonstrate that even with
only 20% of real images (40 volumes) seen during training, segmentation
performance is retained with the addition of synthetic CMR images. Moreover,
the improvement in utilizing synthetic images for augmenting the real data is
evident through the reduction of Hausdorff distance up to 28% and an increase
in the Dice score up to 5%, indicating a higher similarity to the ground truth
in all dimensions.Comment: Accepted for MICCAI 202
- …