108 research outputs found
MEMO: Dataset and Methods for Robust Multimodal Retinal Image Registration with Large or Small Vessel Density Differences
The measurement of retinal blood flow (RBF) in capillaries can provide a
powerful biomarker for the early diagnosis and treatment of ocular diseases.
However, no single modality can determine capillary flowrates with high
precision. Combining erythrocyte-mediated angiography (EMA) with optical
coherence tomography angiography (OCTA) has the potential to achieve this goal,
as EMA can measure the absolute 2D RBF of retinal microvasculature and OCTA can
provide the 3D structural images of capillaries. However, multimodal retinal
image registration between these two modalities remains largely unexplored. To
fill this gap, we establish MEMO, the first public multimodal EMA and OCTA
retinal image dataset. A unique challenge in multimodal retinal image
registration between these modalities is the relatively large difference in
vessel density (VD). To address this challenge, we propose a segmentation-based
deep-learning framework (VDD-Reg) and a new evaluation metric (MSD), which
provide robust results despite differences in vessel density. VDD-Reg consists
of a vessel segmentation module and a registration module. To train the vessel
segmentation module, we further designed a two-stage semi-supervised learning
framework (LVD-Seg) combining supervised and unsupervised losses. We
demonstrate that VDD-Reg outperforms baseline methods quantitatively and
qualitatively for cases of both small VD differences (using the CF-FA dataset)
and large VD differences (using our MEMO dataset). Moreover, VDD-Reg requires
as few as three annotated vessel segmentation masks to maintain its accuracy,
demonstrating its feasibility.Comment: Submitted to IEEE JBH
U-Net and its variants for medical image segmentation: theory and applications
U-net is an image segmentation technique developed primarily for medical
image analysis that can precisely segment images using a scarce amount of
training data. These traits provide U-net with a very high utility within the
medical imaging community and have resulted in extensive adoption of U-net as
the primary tool for segmentation tasks in medical imaging. The success of
U-net is evident in its widespread use in all major image modalities from CT
scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a
segmentation tool, there have been instances of the use of U-net in other
applications. As the potential of U-net is still increasing, in this review we
look at the various developments that have been made in the U-net architecture
and provide observations on recent trends. We examine the various innovations
that have been made in deep learning and discuss how these tools facilitate
U-net. Furthermore, we look at image modalities and application areas where
U-net has been applied.Comment: 42 pages, in IEEE Acces
Recent Progress in Transformer-based Medical Image Analysis
The transformer is primarily used in the field of natural language
processing. Recently, it has been adopted and shows promise in the computer
vision (CV) field. Medical image analysis (MIA), as a critical branch of CV,
also greatly benefits from this state-of-the-art technique. In this review, we
first recap the core component of the transformer, the attention mechanism, and
the detailed structures of the transformer. After that, we depict the recent
progress of the transformer in the field of MIA. We organize the applications
in a sequence of different tasks, including classification, segmentation,
captioning, registration, detection, enhancement, localization, and synthesis.
The mainstream classification and segmentation tasks are further divided into
eleven medical image modalities. A large number of experiments studied in this
review illustrate that the transformer-based method outperforms existing
methods through comparisons with multiple evaluation metrics. Finally, we
discuss the open challenges and future opportunities in this field. This
task-modality review with the latest contents, detailed information, and
comprehensive comparison may greatly benefit the broad MIA community.Comment: Computers in Biology and Medicine Accepte
Deep Learning Techniques for Automated Analysis and Processing of High Resolution Medical Imaging
Programa Oficial de Doutoramento en Computación . 5009V01[Abstract]
Medical imaging plays a prominent role in modern clinical practice for numerous
medical specialties. For instance, in ophthalmology, different imaging techniques are
commonly used to visualize and study the eye fundus. In this context, automated
image analysis methods are key towards facilitating the early diagnosis and adequate
treatment of several diseases. Nowadays, deep learning algorithms have already
demonstrated a remarkable performance for different image analysis tasks. However,
these approaches typically require large amounts of annotated data for the training
of deep neural networks. This complicates the adoption of deep learning approaches,
especially in areas where large scale annotated datasets are harder to obtain, such
as in medical imaging.
This thesis aims to explore novel approaches for the automated analysis of medical
images, particularly in ophthalmology. In this regard, the main focus is on
the development of novel deep learning-based approaches that do not require large
amounts of annotated training data and can be applied to high resolution images.
For that purpose, we have presented a novel paradigm that allows to take advantage
of unlabeled complementary image modalities for the training of deep neural
networks. Additionally, we have also developed novel approaches for the detailed
analysis of eye fundus images. In that regard, this thesis explores the analysis of
relevant retinal structures as well as the diagnosis of different retinal diseases. In
general, the developed algorithms provide satisfactory results for the analysis of the
eye fundus, even when limited annotated training data is available.[Resumen]
Las técnicas de imagen tienen un papel destacado en la práctica clínica moderna
de numerosas especialidades médicas. Por ejemplo, en oftalmología es común el uso
de diferentes técnicas de imagen para visualizar y estudiar el fondo de ojo. En este
contexto, los métodos automáticos de análisis de imagen son clave para facilitar
el diagnóstico precoz y el tratamiento adecuado de diversas enfermedades. En la
actualidad, los algoritmos de aprendizaje profundo ya han demostrado un notable
rendimiento en diferentes tareas de análisis de imagen. Sin embargo, estos métodos
suelen necesitar grandes cantidades de datos etiquetados para el entrenamiento de
las redes neuronales profundas. Esto complica la adopción de los métodos de aprendizaje
profundo, especialmente en áreas donde los conjuntos masivos de datos etiquetados
son más difíciles de obtener, como es el caso de la imagen médica.
Esta tesis tiene como objetivo explorar nuevos métodos para el análisis automático de imagen médica, concretamente en oftalmología. En este sentido, el foco
principal es el desarrollo de nuevos métodos basados en aprendizaje profundo que no
requieran grandes cantidades de datos etiquetados para el entrenamiento y puedan
aplicarse a imágenes de alta resolución. Para ello, hemos presentado un nuevo
paradigma que permite aprovechar modalidades de imagen complementarias no etiquetadas
para el entrenamiento de redes neuronales profundas. Además, también
hemos desarrollado nuevos métodos para el análisis en detalle de las imágenes del
fondo de ojo. En este sentido, esta tesis explora el análisis de estructuras retinianas
relevantes, así como el diagnóstico de diferentes enfermedades de la retina. En
general, los algoritmos desarrollados proporcionan resultados satisfactorios para el
análisis de las imágenes de fondo de ojo, incluso cuando la disponibilidad de datos
de entrenamiento etiquetados es limitada.[Resumo]
As técnicas de imaxe teñen un papel destacado na práctica clínica moderna de
numerosas especialidades médicas. Por exemplo, en oftalmoloxía é común o uso
de diferentes técnicas de imaxe para visualizar e estudar o fondo de ollo. Neste
contexto, os métodos automáticos de análises de imaxe son clave para facilitar o
diagn ostico precoz e o tratamento adecuado de diversas enfermidades. Na actualidade,
os algoritmos de aprendizaxe profunda xa demostraron un notable rendemento
en diferentes tarefas de análises de imaxe. Con todo, estes métodos adoitan necesitar
grandes cantidades de datos etiquetos para o adestramento das redes neuronais
profundas. Isto complica a adopción dos métodos de aprendizaxe profunda, especialmente
en áreas onde os conxuntos masivos de datos etiquetados son máis difíciles
de obter, como é o caso da imaxe médica.
Esta tese ten como obxectivo explorar novos métodos para a análise automática
de imaxe médica, concretamente en oftalmoloxía. Neste sentido, o foco principal
é o desenvolvemento de novos métodos baseados en aprendizaxe profunda que non
requiran grandes cantidades de datos etiquetados para o adestramento e poidan aplicarse
a imaxes de alta resolución. Para iso, presentamos un novo paradigma que
permite aproveitar modalidades de imaxe complementarias non etiquetadas para o
adestramento de redes neuronais profundas. Ademais, tamén desenvolvemos novos
métodos para a análise en detalle das imaxes do fondo de ollo. Neste sentido, esta
tese explora a análise de estruturas retinianas relevantes, así como o diagnóstico de
diferentes enfermidades da retina. En xeral, os algoritmos desenvolvidos proporcionan
resultados satisfactorios para a análise das imaxes de fondo de ollo, mesmo
cando a dispoñibilidade de datos de adestramento etiquetados é limitada
U-net and its variants for medical image segmentation: A review of theory and applications
U-net is an image segmentation technique developed primarily for image segmentation tasks. These traits provide U-net with a high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in nearly all major image modalities, from CT scans and MRI to Xrays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. Given that U-net’s potential is still increasing, this narrative literature review examines the numerous developments and breakthroughs in the U-net architecture and provides observations on recent trends. We also discuss the many innovations that have advanced in deep learning and discuss how these tools facilitate U-net. In addition, we review the different image modalities and application areas that have been enhanced by U-net
Deep learning for image-based liver analysis — A comprehensive review focusing on malignant lesions
Deep learning-based methods, in particular, convolutional neural networks and fully convolutional networks are now widely used in the medical image analysis domain. The scope of this review focuses on the analysis using deep learning of focal liver lesions, with a special interest in hepatocellular carcinoma and metastatic cancer; and structures like the parenchyma or the vascular system. Here, we address several neural network architectures used for analyzing the anatomical structures and lesions in the liver from various imaging modalities such as computed tomography, magnetic resonance imaging and ultrasound. Image analysis tasks like segmentation, object detection and classification for the liver, liver vessels and liver lesions are discussed. Based on the qualitative search, 91 papers were filtered out for the survey, including journal publications and conference proceedings. The papers reviewed in this work are grouped into eight categories based on the methodologies used. By comparing the evaluation metrics, hybrid models performed better for both the liver and the lesion segmentation tasks, ensemble classifiers performed better for the vessel segmentation tasks and combined approach performed better for both the lesion classification and detection tasks. The performance was measured based on the Dice score for the segmentation, and accuracy for the classification and detection tasks, which are the most commonly used metrics.publishedVersio
- …