Image Quality Improvement of Medical Images using Deep Learning for Computer-aided Diagnosis

Retina image analysis is an important screening tool for early detection of multiple dis eases such as diabetic retinopathy which greatly impairs visual function. Image analy sis and pathology detection can be accomplished both by ophthalmologists and by the use of computer-aided diagnosis systems. Advancements in hardware technology led to more portable and less expensive imaging devices for medical image acquisition. This promotes large scale remote diagnosis by clinicians as well as the implementation of computer-aided diagnosis systems for local routine disease screening. However, lower cost equipment generally results in inferior quality images. This may jeopardize the reliability of the acquired images and thus hinder the overall performance of the diagnos tic tool. To solve this open challenge, we carried out an in-depth study on using different deep learning-based frameworks for improving retina image quality while maintaining the underlying morphological information for the diagnosis. Our results demonstrate that using a Cycle Generative Adversarial Network for unpaired image-to-image trans lation leads to successful transformations of retina images from a low- to a high-quality domain. The visual evidence of this improvement was quantitatively affirmed by the two proposed validation methods. The first used a retina image quality classifier to confirm a significant prediction label shift towards a quality enhance. On average, a 50% increase of images being classified as high-quality was verified. The second analysed the perfor mance modifications of a diabetic retinopathy detection algorithm upon being trained with the quality-improved images. The latter led to strong evidence that the proposed solution satisfies the requirement of maintaining the images’ original information for diagnosis, and that it assures a pathology-assessment more sensitive to the presence of pathological signs. These experimental results confirm the potential effectiveness of our solution in improving retina image quality for diagnosis. Along with the addressed con tributions, we analysed how the construction of the data sets representing the low-quality domain impacts the quality translation efficiency. Our findings suggest that by tackling the problem more selectively, that is, constructing data sets more homogeneous in terms of their image defects, we can obtain more accentuated quality transformations

A Generic Fundus Image Enhancement Network Boosted by Frequency Self-supervised Representation Learning

Fundus photography is prone to suffer from image quality degradation that impacts clinical examination performed by ophthalmologists or intelligent systems. Though enhancement algorithms have been developed to promote fundus observation on degraded images, high data demands and limited applicability hinder their clinical deployment. To circumvent this bottleneck, a generic fundus image enhancement network (GFE-Net) is developed in this study to robustly correct unknown fundus images without supervised or extra data. Levering image frequency information, self-supervised representation learning is conducted to learn robust structure-aware representations from degraded images. Then with a seamless architecture that couples representation learning and image enhancement, GFE-Net can accurately correct fundus images and meanwhile preserve retinal structures. Comprehensive experiments are implemented to demonstrate the effectiveness and advantages of GFE-Net. Compared with state-of-the-art algorithms, GFE-Net achieves superior performance in data dependency, enhancement performance, deployment efficiency, and scale generalizability. Follow-up fundus image analysis is also facilitated by GFE-Net, whose modules are respectively verified to be effective for image enhancement.Comment: Accepted by Medical Image Analysis in Auguest, 202

An In-Depth Statistical Review of Retinal Image Processing Models from a Clinical Perspective

The burgeoning field of retinal image processing is critical in facilitating early diagnosis and treatment of retinal diseases, which are amongst the leading causes of vision impairment globally. Despite rapid advancements, existing machine learning models for retinal image processing are characterized by significant limitations, including disparities in pre-processing, segmentation, and classification methodologies, as well as inconsistencies in post-processing operations. These limitations hinder the realization of accurate, reliable, and clinically relevant outcomes. This paper provides an in-depth statistical review of extant machine learning models used in retinal image processing, meticulously comparing them based on their internal operating characteristics and performance levels. By adopting a robust analytical approach, our review delineates the strengths and weaknesses of current models, offering comprehensive insights that are instrumental in guiding future research and development in this domain. Furthermore, this review underscores the potential clinical impacts of these models, highlighting their pivotal role in enhancing diagnostic accuracy, prognostic assessments, and therapeutic interventions for retinal disorders. In conclusion, our work not only bridges the existing knowledge gap in the literature but also paves the way for the evolution of more sophisticated and clinically-aligned retinal image processing models, ultimately contributing to improved patient outcomes and advancements in ophthalmic care

OTRE: Where Optimal Transport Guided Unpaired Image-to-Image Translation Meets Regularization by Enhancing

Non-mydriatic retinal color fundus photography (CFP) is widely available due to the advantage of not requiring pupillary dilation, however, is prone to poor quality due to operators, systemic imperfections, or patient-related causes. Optimal retinal image quality is mandated for accurate medical diagnoses and automated analyses. Herein, we leveraged the Optimal Transport (OT) theory to propose an unpaired image-to-image translation scheme for mapping low-quality retinal CFPs to high-quality counterparts. Furthermore, to improve the flexibility, robustness, and applicability of our image enhancement pipeline in the clinical practice, we generalized a state-of-the-art model-based image reconstruction method, regularization by denoising, by plugging in priors learned by our OT-guided image-to-image translation network. We named it as regularization by enhancing (RE). We validated the integrated framework, OTRE, on three publicly available retinal image datasets by assessing the quality after enhancement and their performance on various downstream tasks, including diabetic retinopathy grading, vessel segmentation, and diabetic lesion segmentation. The experimental results demonstrated the superiority of our proposed framework over some state-of-the-art unsupervised competitors and a state-of-the-art supervised method.Comment: Accepted as a conference paper to The 28th biennial international conference on Information Processing in Medical Imaging (IPMI 2023

A deep learning model to assess and enhance eye fundus image quality

Engineering aims to design, build, and implement solutions that will increase and/or improve the life quality of human beings. Likewise, from medicine, solutions are generated for the same purposes, enabling these two knowledge areas to converge for a common goal. With the thesis work “A Deep Learning Model to Assess and Enhance Eye Fundus Image Quality", a model was proposed and implement a model that allows us to evaluate and enhance the quality of fundus images, which contributes to improving the efficiency and effectiveness of a subsequent diagnosis based on these images. On the one hand, for the evaluation of these images, a model based on a lightweight convolutional neural network architecture was developed, termed as Mobile Fundus Quality Network (MFQ-Net). This model has approximately 90% fewer parameters than those of the latest generation. For its evaluation, the Kaggle public data set was used with two sets of quality annotations, binary (good and bad) and three classes (good, usable and bad) obtaining an accuracy of 0.911 and 0.856 in the binary mode and three classes respectively in the classification of the fundus image quality. On the other hand, a method was developed for eye fundus quality enhancement termed as Pix2Pix Fundus Oculi Quality Enhancement (P2P-FOQE). This method is based on three stages which are; pre-enhancement: for color adjustment, enhancement: with a Pix2Pix network (which is a Conditional Generative Adversarial Network) as the core of the method and post-enhancement: which is a CLAHE adjustment for contrast and detail enhancement. This method was evaluated on a subset of quality annotations for the Kaggle public database which was re-classified for three categories (good, usable, and poor) by a specialist from the Fundación Oftalmolóica Nacional. With this method, the quality of these images for the good class was improved by 72.33%. Likewise, the image quality improved from the bad class to the usable class, and from the bad class to the good class by 56.21% and 29.49% respectively.La ingeniería busca diseñar, construir e implementar soluciones que permitan aumentar y/o mejorar la calidad de vida de los seres humanos. Igualmente, desde la medicina son generadas soluciones con los mismos fines, posibilitando que estas dos áreas del conocimiento convergan por un bien común. Con el trabajo de tesis “A Deep Learning Model to Assess and Enhance Eye Fundus Image Quality”, se propuso e implementó un modelo que permite evaluar y mejorar la calidad de las imágenes de fondo de ojo, lo cual contribuye a mejorar la eficiencia y eficacia de un posterior diagnóstico basado en estas imágenes. Para la evaluación de estás imágenes, se desarrolló un modelo basado en una arquitectura de red neuronal convolucional ligera, la cual fue llamada Mobile Fundus Quality Network (MFQ-Net). Este modelo posee aproximadamente 90% menos parámetros que aquellos de última generación. Para su evaluación se utilizó el conjunto de datos públicos de Kaggle con dos sets de anotaciones de calidad, binario (buena y mala) y tres clases (buena, usable y mala) obteniendo en la tareas de clasificación de la calidad de la imagen de fondo de ojo una exactitud de 0.911 y 0.856 en la modalidad binaria y tres clases respectivamente. Por otra parte, se desarrolló un método el cual realiza una mejora de la calidad de imágenes de fondo de ojo llamado Pix2Pix Fundus Oculi Quality Enhacement (P2P-FOQE). Este método está basado en tres etapas las cuales son; premejora: para ajuste de color, mejora: con una red Pix2Pix (la cual es una Conditional Generative Adversarial Network) como núcleo del método y postmejora: la cual es un ajuste CLAHE para contraste y realce de detalles. Este método fue evaluado en un subconjunto de anotaciones de calidad para la base de datos pública de Kaggle el cual fue re clasificado por un especialista de la Fundación Oftalmológica Nacional para tres categorías (buena, usable y mala). Con este método fue mejorada la calidad de estas imágenes para la clase buena en un 72,33%. Así mismo, la calidad de imagen mejoró de la clase mala a la clase utilizable, y de la clase mala a clase buena en 56.21% y 29.49% respectivamente.Línea de investigación: Visión por computadora para análisis de imágenes médicasMaestrí

MEMO: Dataset and Methods for Robust Multimodal Retinal Image Registration with Large or Small Vessel Density Differences

The measurement of retinal blood flow (RBF) in capillaries can provide a powerful biomarker for the early diagnosis and treatment of ocular diseases. However, no single modality can determine capillary flowrates with high precision. Combining erythrocyte-mediated angiography (EMA) with optical coherence tomography angiography (OCTA) has the potential to achieve this goal, as EMA can measure the absolute 2D RBF of retinal microvasculature and OCTA can provide the 3D structural images of capillaries. However, multimodal retinal image registration between these two modalities remains largely unexplored. To fill this gap, we establish MEMO, the first public multimodal EMA and OCTA retinal image dataset. A unique challenge in multimodal retinal image registration between these modalities is the relatively large difference in vessel density (VD). To address this challenge, we propose a segmentation-based deep-learning framework (VDD-Reg) and a new evaluation metric (MSD), which provide robust results despite differences in vessel density. VDD-Reg consists of a vessel segmentation module and a registration module. To train the vessel segmentation module, we further designed a two-stage semi-supervised learning framework (LVD-Seg) combining supervised and unsupervised losses. We demonstrate that VDD-Reg outperforms baseline methods quantitatively and qualitatively for cases of both small VD differences (using the CF-FA dataset) and large VD differences (using our MEMO dataset). Moreover, VDD-Reg requires as few as three annotated vessel segmentation masks to maintain its accuracy, demonstrating its feasibility.Comment: Submitted to IEEE JBH

Combining Multi-Sequence and Synthetic Images for Improved Segmentation of Late Gadolinium Enhancement Cardiac MRI

© Springer Nature Switzerland AG 2020. Accurate segmentation of the cardiac boundaries in late gadolinium enhancement magnetic resonance images (LGE-MRI) is a fundamental step for accurate quantification of scar tissue. However, while there are many solutions for automatic cardiac segmentation of cine images, the presence of scar tissue can make the correct delineation of the myocardium in LGE-MRI challenging even for human experts. As part of the Multi-Sequence Cardiac MR Segmentation Challenge, we propose a solution for LGE-MRI segmentation based on two components. First, a generative adversarial network is trained for the task of modality-to-modality translation between cine and LGE-MRI sequences to obtain extra synthetic images for both modalities. Second, a deep learning model is trained for segmentation with different combinations of original, augmented and synthetic sequences. Our results based on three magnetic resonance sequences (LGE, bSSFP and T2) from 45 different patients show that the multi-sequence model training integrating synthetic images and data augmentation improves in the segmentation over conventional training with real datasets. In conclusion, the accuracy of the segmentation of LGE-MRI images can be improved by using complementary information provided by non-contrast MRI sequences