30 research outputs found
Pathology Segmentation using Distributional Differences to Images of Healthy Origin
Fully supervised segmentation methods require a large training cohort of
already segmented images, providing information at the pixel level of each
image. We present a method to automatically segment and model pathologies in
medical images, trained solely on data labelled on the image level as either
healthy or containing a visual defect. We base our method on CycleGAN, an
image-to-image translation technique, to translate images between the domains
of healthy and pathological images. We extend the core idea with two key
contributions. Implementing the generators as residual generators allows us to
explicitly model the segmentation of the pathology. Realizing the translation
from the healthy to the pathological domain using a variational autoencoder
allows us to specify one representation of the pathology, as this
transformation is otherwise not unique. Our model hence not only allows us to
create pixelwise semantic segmentations, it is also able to create inpaintings
for the segmentations to render the pathological image healthy. Furthermore, we
can draw new unseen pathology samples from this model based on the distribution
in the data. We show quantitatively, that our method is able to segment
pathologies with a surprising accuracy being only slightly inferior to a
state-of-the-art fully supervised method, although the latter has per-pixel
rather than per-image training information. Moreover, we show qualitative
results of both the segmentations and inpaintings. Our findings motivate
further research into weakly-supervised segmentation using image level
annotations, allowing for faster and cheaper acquisition of training data
without a large sacrifice in segmentation accuracy
Pathology Synthesis of 3D Consistent Cardiac MR Images Using 2D VAEs and GANs
We propose a method for synthesizing cardiac MR images with plausible heart shapes and realistic appearances for the purpose of generating labeled data for deep-learning (DL) training. It breaks down the image synthesis into label deformation and label-to-image translation tasks. The former is achieved via latent space interpolation in a VAE model, while the latter is accomplished via a conditional GAN model. We devise an approach for label manipulation in the latent space of the trained VAE model, namely pathology synthesis, aiming to synthesize a series of pseudo-pathological synthetic subjects with characteristics of a desired heart disease. Furthermore, we propose to model the relationship between 2D slices in the latent space of the VAE via estimating the correlation coefficient matrix between the latent vectors and utilizing it to correlate elements of randomly drawn samples before decoding to image space. This simple yet effective approach results in generating 3D consistent subjects from 2D slice-by-slice generations. Such an approach could provide a solution to diversify and enrich the available database of cardiac MR images and to pave the way for the development of generalizable DL-based image analysis algorithms. The code will be available at https://github.com/sinaamirrajab/CardiacPathologySynthesis