43 research outputs found
Learning Disentangled Representations in the Imaging Domain
Disentangled representation learning has been proposed as an approach to
learning general representations even in the absence of, or with limited,
supervision. A good general representation can be fine-tuned for new target
tasks using modest amounts of data, or used directly in unseen domains
achieving remarkable performance in the corresponding task. This alleviation of
the data and annotation requirements offers tantalising prospects for
applications in computer vision and healthcare. In this tutorial paper, we
motivate the need for disentangled representations, present key theory, and
detail practical building blocks and criteria for learning such
representations. We discuss applications in medical imaging and computer vision
emphasising choices made in exemplar key works. We conclude by presenting
remaining challenges and opportunities.Comment: Submitted. This paper follows a tutorial style but also surveys a
considerable (more than 200 citations) number of work
Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learning
In deep learning, auxiliary objectives are often used to facilitate learning
in situations where data is scarce, or the principal task is extremely complex.
This idea is primarily inspired by the improved generalization capability
induced by solving multiple tasks simultaneously, which leads to a more robust
shared representation. Nevertheless, finding optimal auxiliary tasks that give
rise to the desired improvement is a crucial problem that often requires
hand-crafted solutions or expensive meta-learning approaches. In this paper, we
propose a novel framework, dubbed Detaux, whereby a weakly supervised
disentanglement procedure is used to discover new unrelated classification
tasks and the associated labels that can be exploited with the principal task
in any Multi-Task Learning (MTL) model. The disentanglement procedure works at
a representation level, isolating a subspace related to the principal task,
plus an arbitrary number of orthogonal subspaces. In the most disentangled
subspaces, through a clustering procedure, we generate the additional
classification tasks, and the associated labels become their representatives.
Subsequently, the original data, the labels associated with the principal task,
and the newly discovered ones can be fed into any MTL framework. Extensive
validation on both synthetic and real data, along with various ablation
studies, demonstrate promising results, revealing the potential in what has
been, so far, an unexplored connection between learning disentangled
representations and MTL. The code will be made publicly available upon
acceptance.Comment: Under review in Pattern Recognition Letter
Domain Generalization for Medical Image Analysis: A Survey
Medical Image Analysis (MedIA) has become an essential tool in medicine and
healthcare, aiding in disease diagnosis, prognosis, and treatment planning, and
recent successes in deep learning (DL) have made significant contributions to
its advances. However, DL models for MedIA remain challenging to deploy in
real-world situations, failing for generalization under the distributional gap
between training and testing samples, known as a distribution shift problem.
Researchers have dedicated their efforts to developing various DL methods to
adapt and perform robustly on unknown and out-of-distribution data
distributions. This paper comprehensively reviews domain generalization studies
specifically tailored for MedIA. We provide a holistic view of how domain
generalization techniques interact within the broader MedIA system, going
beyond methodologies to consider the operational implications on the entire
MedIA workflow. Specifically, we categorize domain generalization methods into
data-level, feature-level, model-level, and analysis-level methods. We show how
those methods can be used in various stages of the MedIA workflow with DL
equipped from data acquisition to model prediction and analysis. Furthermore,
we include benchmark datasets and applications used to evaluate these
approaches and analyze the strengths and weaknesses of various methods,
unveiling future research opportunities
MI-SegNet: Mutual Information-Based US Segmentation for Unseen Domain Generalization
Generalization capabilities of learning-based medical image segmentation
across domains are currently limited by the performance degradation caused by
the domain shift, particularly for ultrasound (US) imaging. The quality of US
images heavily relies on carefully tuned acoustic parameters, which vary across
sonographers, machines, and settings. To improve the generalizability on US
images across domains, we propose MI-SegNet, a novel mutual information (MI)
based framework to explicitly disentangle the anatomical and domain feature
representations; therefore, robust domain-independent segmentation can be
expected. Two encoders are employed to extract the relevant features for the
disentanglement. The segmentation only uses the anatomical feature map for its
prediction. In order to force the encoders to learn meaningful feature
representations a cross-reconstruction method is used during training.
Transformations, specific to either domain or anatomy are applied to guide the
encoders in their respective feature extraction task. Additionally, any MI
present in both feature maps is punished to further promote separate feature
spaces. We validate the generalizability of the proposed domain-independent
segmentation approach on several datasets with varying parameters and machines.
Furthermore, we demonstrate the effectiveness of the proposed MI-SegNet serving
as a pre-trained model by comparing it with state-of-the-art networks.Comment: Accepted by MICCAI 202
CDDSA: Contrastive Domain Disentanglement and Style Augmentation for Generalizable Medical Image Segmentation
Generalization to previously unseen images with potential domain shifts and
different styles is essential for clinically applicable medical image
segmentation, and the ability to disentangle domain-specific and
domain-invariant features is key for achieving Domain Generalization (DG).
However, existing DG methods can hardly achieve effective disentanglement to
get high generalizability. To deal with this problem, we propose an efficient
Contrastive Domain Disentanglement and Style Augmentation (CDDSA) framework for
generalizable medical image segmentation. First, a disentangle network is
proposed to decompose an image into a domain-invariant anatomical
representation and a domain-specific style code, where the former is sent to a
segmentation model that is not affected by the domain shift, and the
disentangle network is regularized by a decoder that combines the anatomical
and style codes to reconstruct the input image. Second, to achieve better
disentanglement, a contrastive loss is proposed to encourage the style codes
from the same domain and different domains to be compact and divergent,
respectively. Thirdly, to further improve generalizability, we propose a style
augmentation method based on the disentanglement representation to synthesize
images in various unseen styles with shared anatomical structures. Our method
was validated on a public multi-site fundus image dataset for optic cup and
disc segmentation and an in-house multi-site Nasopharyngeal Carcinoma Magnetic
Resonance Image (NPC-MRI) dataset for nasopharynx Gross Tumor Volume (GTVnx)
segmentation. Experimental results showed that the proposed CDDSA achieved
remarkable generalizability across different domains, and it outperformed
several state-of-the-art methods in domain-generalizable segmentation.Comment: 14 pages, 8 figure
Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations
Ultrasound (US) imaging is widely used for biometric measurement and
diagnosis of internal organs due to the advantages of being real-time and
radiation-free. However, due to high inter-operator variability, resulting
images highly depend on operators' experience. In this work, an intelligent
robotic sonographer is proposed to autonomously "explore" target anatomies and
navigate a US probe to a relevant 2D plane by learning from expert. The
underlying high-level physiological knowledge from experts is inferred by a
neural reward function, using a ranked pairwise image comparisons approach in a
self-supervised fashion. This process can be referred to as understanding the
"language of sonography". Considering the generalization capability to overcome
inter-patient variations, mutual information is estimated by a network to
explicitly extract the task-related and domain features in latent space.
Besides, a Gaussian distribution-based filter is developed to automatically
evaluate and take the quality of the expert's demonstrations into account. The
robotic localization is carried out in coarse-to-fine mode based on the
predicted reward associated to B-mode images. To demonstrate the performance of
the proposed approach, representative experiments for the "line" target and
"point" target are performed on vascular phantom and two ex-vivo animal organ
phantoms (chicken heart and lamb kidney), respectively. The results
demonstrated that the proposed advanced framework can robustly work on
different kinds of known and unseen phantoms
AtrialGeneral: Domain Generalization for Left Atrial Segmentation of Multi-Center LGE MRIs
Left atrial (LA) segmentation from late gadolinium enhanced magnetic
resonance imaging (LGE MRI) is a crucial step needed for planning the treatment
of atrial fibrillation. However, automatic LA segmentation from LGE MRI is
still challenging, due to the poor image quality, high variability in LA
shapes, and unclear LA boundary. Though deep learning-based methods can provide
promising LA segmentation results, they often generalize poorly to unseen
domains, such as data from different scanners and/or sites. In this work, we
collect 210 LGE MRIs from different centers with different levels of image
quality. To evaluate the domain generalization ability of models on the LA
segmentation task, we employ four commonly used semantic segmentation networks
for the LA segmentation from multi-center LGE MRIs. Besides, we investigate
three domain generalization strategies, i.e., histogram matching, mutual
information based disentangled representation, and random style transfer, where
a simple histogram matching is proved to be most effective.Comment: 10 pages, 4 figures, MICCAI202
Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives
Deep learning has demonstrated remarkable performance across various tasks in
medical imaging. However, these approaches primarily focus on supervised
learning, assuming that the training and testing data are drawn from the same
distribution. Unfortunately, this assumption may not always hold true in
practice. To address these issues, unsupervised domain adaptation (UDA)
techniques have been developed to transfer knowledge from a labeled domain to a
related but unlabeled domain. In recent years, significant advancements have
been made in UDA, resulting in a wide range of methodologies, including feature
alignment, image translation, self-supervision, and disentangled representation
methods, among others. In this paper, we provide a comprehensive literature
review of recent deep UDA approaches in medical imaging from a technical
perspective. Specifically, we categorize current UDA research in medical
imaging into six groups and further divide them into finer subcategories based
on the different tasks they perform. We also discuss the respective datasets
used in the studies to assess the divergence between the different domains.
Finally, we discuss emerging areas and provide insights and discussions on
future research directions to conclude this survey.Comment: Under Revie