424 research outputs found
Paired and Unpaired Deep Generative Models on Multimodal Retinal Image Reconstruction
[Abstract] This work explores the use of paired and unpaired data for training deep neural networks in the multimodal reconstruction of retinal images. Particularly, we focus on the reconstruction of fluorescein angiography from retinography, which are two complementary representations of the eye fundus. The performed experiments allow to compare the paired and unpaired alternatives.Instituto de Salud Carlos III; DTS18/00136Ministerio de Ciencia, Innovación y Universidades; DPI2015-69948-RMinisterio de Ciencia, Innovación y Universidades; RTI2018-095894-B-I00Xunta de Galicia; ED431G/01Xunta de Galicia; ED431C 2016-047Xunta de Galicia; ED481A-2017/328
Deep Learning Techniques for Automated Analysis and Processing of High Resolution Medical Imaging
Programa Oficial de Doutoramento en Computación . 5009V01[Abstract]
Medical imaging plays a prominent role in modern clinical practice for numerous
medical specialties. For instance, in ophthalmology, different imaging techniques are
commonly used to visualize and study the eye fundus. In this context, automated
image analysis methods are key towards facilitating the early diagnosis and adequate
treatment of several diseases. Nowadays, deep learning algorithms have already
demonstrated a remarkable performance for different image analysis tasks. However,
these approaches typically require large amounts of annotated data for the training
of deep neural networks. This complicates the adoption of deep learning approaches,
especially in areas where large scale annotated datasets are harder to obtain, such
as in medical imaging.
This thesis aims to explore novel approaches for the automated analysis of medical
images, particularly in ophthalmology. In this regard, the main focus is on
the development of novel deep learning-based approaches that do not require large
amounts of annotated training data and can be applied to high resolution images.
For that purpose, we have presented a novel paradigm that allows to take advantage
of unlabeled complementary image modalities for the training of deep neural
networks. Additionally, we have also developed novel approaches for the detailed
analysis of eye fundus images. In that regard, this thesis explores the analysis of
relevant retinal structures as well as the diagnosis of different retinal diseases. In
general, the developed algorithms provide satisfactory results for the analysis of the
eye fundus, even when limited annotated training data is available.[Resumen]
Las técnicas de imagen tienen un papel destacado en la práctica clínica moderna
de numerosas especialidades médicas. Por ejemplo, en oftalmología es común el uso
de diferentes técnicas de imagen para visualizar y estudiar el fondo de ojo. En este
contexto, los métodos automáticos de análisis de imagen son clave para facilitar
el diagnóstico precoz y el tratamiento adecuado de diversas enfermedades. En la
actualidad, los algoritmos de aprendizaje profundo ya han demostrado un notable
rendimiento en diferentes tareas de análisis de imagen. Sin embargo, estos métodos
suelen necesitar grandes cantidades de datos etiquetados para el entrenamiento de
las redes neuronales profundas. Esto complica la adopción de los métodos de aprendizaje
profundo, especialmente en áreas donde los conjuntos masivos de datos etiquetados
son más difíciles de obtener, como es el caso de la imagen médica.
Esta tesis tiene como objetivo explorar nuevos métodos para el análisis automático de imagen médica, concretamente en oftalmología. En este sentido, el foco
principal es el desarrollo de nuevos métodos basados en aprendizaje profundo que no
requieran grandes cantidades de datos etiquetados para el entrenamiento y puedan
aplicarse a imágenes de alta resolución. Para ello, hemos presentado un nuevo
paradigma que permite aprovechar modalidades de imagen complementarias no etiquetadas
para el entrenamiento de redes neuronales profundas. Además, también
hemos desarrollado nuevos métodos para el análisis en detalle de las imágenes del
fondo de ojo. En este sentido, esta tesis explora el análisis de estructuras retinianas
relevantes, así como el diagnóstico de diferentes enfermedades de la retina. En
general, los algoritmos desarrollados proporcionan resultados satisfactorios para el
análisis de las imágenes de fondo de ojo, incluso cuando la disponibilidad de datos
de entrenamiento etiquetados es limitada.[Resumo]
As técnicas de imaxe teñen un papel destacado na práctica clínica moderna de
numerosas especialidades médicas. Por exemplo, en oftalmoloxía é común o uso
de diferentes técnicas de imaxe para visualizar e estudar o fondo de ollo. Neste
contexto, os métodos automáticos de análises de imaxe son clave para facilitar o
diagn ostico precoz e o tratamento adecuado de diversas enfermidades. Na actualidade,
os algoritmos de aprendizaxe profunda xa demostraron un notable rendemento
en diferentes tarefas de análises de imaxe. Con todo, estes métodos adoitan necesitar
grandes cantidades de datos etiquetos para o adestramento das redes neuronais
profundas. Isto complica a adopción dos métodos de aprendizaxe profunda, especialmente
en áreas onde os conxuntos masivos de datos etiquetados son máis difíciles
de obter, como é o caso da imaxe médica.
Esta tese ten como obxectivo explorar novos métodos para a análise automática
de imaxe médica, concretamente en oftalmoloxía. Neste sentido, o foco principal
é o desenvolvemento de novos métodos baseados en aprendizaxe profunda que non
requiran grandes cantidades de datos etiquetados para o adestramento e poidan aplicarse
a imaxes de alta resolución. Para iso, presentamos un novo paradigma que
permite aproveitar modalidades de imaxe complementarias non etiquetadas para o
adestramento de redes neuronais profundas. Ademais, tamén desenvolvemos novos
métodos para a análise en detalle das imaxes do fondo de ollo. Neste sentido, esta
tese explora a análise de estruturas retinianas relevantes, así como o diagnóstico de
diferentes enfermidades da retina. En xeral, os algoritmos desenvolvidos proporcionan
resultados satisfactorios para a análise das imaxes de fondo de ollo, mesmo
cando a dispoñibilidade de datos de adestramento etiquetados é limitada
Bi-Modality Medical Image Synthesis Using Semi-Supervised Sequential Generative Adversarial Networks
In this paper, we propose a bi-modality medical image synthesis approach
based on sequential generative adversarial network (GAN) and semi-supervised
learning. Our approach consists of two generative modules that synthesize
images of the two modalities in a sequential order. A method for measuring the
synthesis complexity is proposed to automatically determine the synthesis order
in our sequential GAN. Images of the modality with a lower complexity are
synthesized first, and the counterparts with a higher complexity are generated
later. Our sequential GAN is trained end-to-end in a semi-supervised manner. In
supervised training, the joint distribution of bi-modality images are learned
from real paired images of the two modalities by explicitly minimizing the
reconstruction losses between the real and synthetic images. To avoid
overfitting limited training images, in unsupervised training, the marginal
distribution of each modality is learned based on unpaired images by minimizing
the Wasserstein distance between the distributions of real and fake images. We
comprehensively evaluate the proposed model using two synthesis tasks based on
three types of evaluate metrics and user studies. Visual and quantitative
results demonstrate the superiority of our method to the state-of-the-art
methods, and reasonable visual quality and clinical significance. Code is made
publicly available at
https://github.com/hustlinyi/Multimodal-Medical-Image-Synthesis
Improving foveal avascular zone segmentation in fluorescein angiograms by leveraging manual vessel labels from public color fundus pictures
In clinical routine, ophthalmologists frequently analyze the shape and size of the foveal avascular zone (FAZ) to detect and monitor retinal diseases. In order to extract those parameters, the contours of the FAZ need to be segmented, which is normally achieved by analyzing the retinal vasculature (RV) around the macula in fluorescein angiograms (FA). Computer-aided segmentation methods based on deep learning (DL) can automate this task. However, current approaches for segmenting the FAZ are often tailored to a specific dataset or require manual initialization. Furthermore, they do not take the variability and challenges of clinical FA into account, which are often of low quality and difficult to analyze. In this paper we propose a DL-based framework to automatically segment the FAZ in challenging FA scans from clinical routine. Our approach mimics the workflow of retinal experts by using additional RV labels as a guidance during training. Hence, our model is able to produce RV segmentations simultaneously. We minimize the annotation work by using a multi-modal approach that leverages already available public datasets of color fundus pictures (CFPs) and their respective manual RV labels. Our experimental evaluation on two datasets with FA from 1) clinical routine and 2) large multicenter clinical trials shows that the addition of weak RV labels as a guidance during training improves the FAZ segmentation significantly with respect to using only manual FAZ annotations.Fil: Hofer, Dominik. Medizinische Universität Wien; AustriaFil: Schmidt Erfurth, Ursula. Medizinische Universität Wien; AustriaFil: Orlando, José Ignacio. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ciencias Exactas. Grupo de Plasmas Densos Magnetizados. Provincia de Buenos Aires. Gobernación. Comision de Investigaciones Científicas. Grupo de Plasmas Densos Magnetizados; Argentina. Medizinische Universität Wien; AustriaFil: Goldbach, Felix. Medizinische Universität Wien; AustriaFil: Gerendas, Bianca S.. Medizinische Universität Wien; AustriaFil: Seeböck, Philipp. Medizinische Universität Wien; Austri
Anatomy-Aware Self-supervised Fetal MRI Synthesis from Unpaired Ultrasound Images
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the
developing brain but is not suitable for anomaly screening. For this ultrasound
(US) is employed. While expert sonographers are adept at reading US images, MR
images are much easier for non-experts to interpret. Hence in this paper we
seek to produce images with MRI-like appearance directly from clinical US
images. Our own clinical motivation is to seek a way to communicate US findings
to patients or clinical professionals unfamiliar with US, but in medical image
analysis such a capability is potentially useful, for instance, for US-MRI
registration or fusion. Our model is self-supervised and end-to-end trainable.
Specifically, based on an assumption that the US and MRI data share a similar
anatomical latent space, we first utilise an extractor to determine shared
latent features, which are then used for data synthesis. Since paired data was
unavailable for our study (and rare in practice), we propose to enforce the
distributions to be similar instead of employing pixel-wise constraints, by
adversarial learning in both the image domain and latent space. Furthermore, we
propose an adversarial structural constraint to regularise the anatomical
structures between the two modalities during the synthesis. A cross-modal
attention scheme is proposed to leverage non-local spatial correlations. The
feasibility of the approach to produce realistic looking MR images is
demonstrated quantitatively and with a qualitative evaluation compared to real
fetal MR images.Comment: MICCAI-MLMI 201
- …