13,814 research outputs found
HeMIS: Hetero-Modal Image Segmentation
We introduce a deep learning image segmentation framework that is extremely
robust to missing imaging modalities. Instead of attempting to impute or
synthesize missing data, the proposed approach learns, for each modality, an
embedding of the input image into a single latent vector space for which
arithmetic operations (such as taking the mean) are well defined. Points in
that space, which are averaged over modalities available at inference time, can
then be further processed to yield the desired segmentation. As such, any
combinatorial subset of available modalities can be provided as input, without
having to learn a combinatorial number of imputation models. Evaluated on two
neurological MRI datasets (brain tumors and MS lesions), the approach yields
state-of-the-art segmentation results when provided with all modalities;
moreover, its performance degrades remarkably gracefully when modalities are
removed, significantly more so than alternative mean-filling or other synthesis
approaches.Comment: Accepted as an oral presentation at MICCAI 201
Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks
Recently, the cycle-consistent generative adversarial networks (CycleGAN) has
been widely used for synthesis of multi-domain medical images. The
domain-specific nonlinear deformations captured by CycleGAN make the
synthesized images difficult to be used for some applications, for example,
generating pseudo-CT for PET-MR attenuation correction. This paper presents a
deformation-invariant CycleGAN (DicycleGAN) method using deformable
convolutional layers and new cycle-consistency losses. Its robustness dealing
with data that suffer from domain-specific nonlinear deformations has been
evaluated through comparison experiments performed on a multi-sequence brain MR
dataset and a multi-modality abdominal dataset. Our method has displayed its
ability to generate synthesized data that is aligned with the source while
maintaining a proper quality of signal compared to CycleGAN-generated data. The
proposed model also obtained comparable performance with CycleGAN when data
from the source and target domains are alignable through simple affine
transformations
- …