Fundus-DeepNet: Multi-Label Deep Learning Classification System for Enhanced Detection of Multiple Ocular Diseases through Data Fusion of Fundus Images

YesDetecting multiple ocular diseases in fundus images is crucial in ophthalmic diagnosis. This study introduces the Fundus-DeepNet system, an automated multi-label deep learning classification system designed to identify multiple ocular diseases by integrating feature representations from pairs of fundus images (e.g., left and right eyes). The study initiates with a comprehensive image pre-processing procedure, including circular border cropping, image resizing, contrast enhancement, noise removal, and data augmentation. Subsequently, discriminative deep feature representations are extracted using multiple deep learning blocks, namely the High-Resolution Network (HRNet) and Attention Block, which serve as feature descriptors. The SENet Block is then applied to further enhance the quality and robustness of feature representations from a pair of fundus images, ultimately consolidating them into a single feature representation. Finally, a sophisticated classification model, known as a Discriminative Restricted Boltzmann Machine (DRBM), is employed. By incorporating a Softmax layer, this DRBM is adept at generating a probability distribution that specifically identifies eight different ocular diseases. Extensive experiments were conducted on the challenging Ophthalmic Image Analysis-Ocular Disease Intelligent Recognition (OIA-ODIR) dataset, comprising diverse fundus images depicting eight different ocular diseases. The Fundus-DeepNet system demonstrated F1-scores, Kappa scores, AUC, and final scores of 88.56%, 88.92%, 99.76%, and 92.41% in the off-site test set, and 89.13%, 88.98%, 99.86%, and 92.66% in the on-site test set.In summary, the Fundus-DeepNet system exhibits outstanding proficiency in accurately detecting multiple ocular diseases, offering a promising solution for early diagnosis and treatment in ophthalmology.European Union under the REFRESH – Research Excellence for Region Sustainability and High-tech Industries project number CZ.10.03.01/00/22_003/0000048 via the Operational Program Just Transition. The Ministry of Education, Youth, and Sports of the Czech Republic - Technical University of Ostrava, Czechia under Grants SP2023/039 and SP2023/042

Network-based features for retinal fundus vessel structure analysis

Retinal fundus imaging is a non-invasive method that allows visualizing the structure of the blood vessels in the retina whose features may indicate the presence of diseases such as diabetic retinopathy (DR) and glaucoma. Here we present a novel method to analyze and quantify changes in the retinal blood vessel structure in patients diagnosed with glaucoma or with DR. First, we use an automatic unsupervised segmentation algorithm to extract a tree-like graph from the retina blood vessel structure. The nodes of the graph represent branching (bifurcation) points and endpoints, while the links represent vessel segments that connect the nodes. Then, we quantify structural differences between the graphs extracted from the groups of healthy and non-healthy patients. We also use fractal analysis to characterize the extracted graphs. Applying these techniques to three retina fundus image databases we find significant differences between the healthy and non-healthy groups (p-values lower than 0.005 or 0.001 depending on the method and on the database). The results are sensitive to the segmentation method (manual or automatic) and to the resolution of the images.Peer ReviewedPostprint (published version

Qhaway: una herramienta de apoyo para el diagnóstico del glaucoma con aprendizaje profundo

Propone un método para el diagnóstico del glaucoma basado en un modelo híbrido de modelos DL, con el cual usando imágenes del fondo de ojo de un paciente se consigue hacer el diagnóstico con alta precisión. Se consideró la integración de los dataset públicos de glaucoma HRF, Drishti-GS1, sjchoi86-HRF, RIM-ONE y ACRIMA, con un total de 1707 imágenes (919 normal y 788 glaucoma) del fondo de ojo, un modelo híbrido de Voting sobre los modelos de DL ResNet50 con dos tipos de fine tuning y ResNet50V2, y la implementación usando Keras y Tensor Flow, con lo que se consiguió un diagnóstico con exactitud del 96.55%, sensibilidad del 98.54% y especificidad del 94.32%. Además, los experimentos numéricos muestran que el aprendizaje usando 5 bases de datos permite mejores resultados que por separado, incluso aplicando transfer learning, también muestran que el modelo híbrido voting genera una exactitud superior en 20.69% a la mejor exactitud obtenido por el mejor modelo de DL (DenseNet169) usando un dataset, 13.22% al mejor modelo (ResNet50V2) usando transfer learning con los 5 datasets, y 1.72% al mejor modelo (ResNet50) considerando los 5 dataset

Machine learning methods for the characterization and classification of complex data

This thesis work presents novel methods for the analysis and classification of medical images and, more generally, complex data. First, an unsupervised machine learning method is proposed to order anterior chamber OCT (Optical Coherence Tomography) images according to a patient's risk of developing angle-closure glaucoma. In a second study, two outlier finding techniques are proposed to improve the results of above mentioned machine learning algorithm, we also show that they are applicable to a wide variety of data, including fraud detection in credit card transactions. In a third study, the topology of the vascular network of the retina, considering it a complex tree-like network is analyzed and we show that structural differences reveal the presence of glaucoma and diabetic retinopathy. In a fourth study we use a model of a laser with optical injection that presents extreme events in its intensity time-series to evaluate machine learning methods to forecast such extreme events.El presente trabajo de tesis desarrolla nuevos métodos para el análisis y clasificación de imágenes médicas y datos complejos en general. Primero, proponemos un método de aprendizaje automático sin supervisión que ordena imágenes OCT (tomografía de coherencia óptica) de la cámara anterior del ojo en función del grado de riesgo del paciente de padecer glaucoma de ángulo cerrado. Luego, desarrollamos dos métodos de detección automática de anomalías que utilizamos para mejorar los resultados del algoritmo anterior, pero que su aplicabilidad va mucho más allá, siendo útil, incluso, para la detección automática de fraudes en transacciones de tarjetas de crédito. Mostramos también, cómo al analizar la topología de la red vascular de la retina considerándola una red compleja, podemos detectar la presencia de glaucoma y de retinopatía diabética a través de diferencias estructurales. Estudiamos también un modelo de un láser con inyección óptica que presenta eventos extremos en la serie temporal de intensidad para evaluar diferentes métodos de aprendizaje automático para predecir dichos eventos extremos.Aquesta tesi desenvolupa nous mètodes per a l’anàlisi i la classificació d’imatges mèdiques i dades complexes. Hem proposat, primer, un mètode d’aprenentatge automàtic sense supervisió que ordena imatges OCT (tomografia de coherència òptica) de la cambra anterior de l’ull en funció del grau de risc del pacient de patir glaucoma d’angle tancat. Després, hem desenvolupat dos mètodes de detecció automàtica d’anomalies que hem utilitzat per millorar els resultats de l’algoritme anterior, però que la seva aplicabilitat va molt més enllà, sent útil, fins i tot, per a la detecció automàtica de fraus en transaccions de targetes de crèdit. Mostrem també, com en analitzar la topologia de la xarxa vascular de la retina considerant-la una xarxa complexa, podem detectar la presència de glaucoma i de retinopatia diabètica a través de diferències estructurals. Finalment, hem estudiat un làser amb injecció òptica, el qual presenta esdeveniments extrems en la sèrie temporal d’intensitat. Hem avaluat diferents mètodes per tal de predir-los.Postprint (published version

Assessing neurodegeneration of the retina and brain with ultra-widefield retinal imaging

The eye is embryologically, physiologically and anatomically linked to the brain. Emerging evidence suggests that neurodegenerative diseases, such as Alzheimer’s disease (AD), manifest in the retina. Retinal imaging is a quick, non-invasive method to view the retina and its microvasculature. Features such as blood vessel calibre, tortuosity and complexity of the vascular structure (measured through fractal analysis) are thought to reflect microvascular health and have been found to associate with clinical signs of hypertension, diabetes, cardiovascular disease and cognitive decline. Small deposits of acellular debris called drusen in the peripheral retina have also been linked with AD where histological studies show they can contain amyloid beta, a hallmark of AD. Age-related macular degeneration (AMD) is a neurodegenerative disorder of the retina and a leading cause of irreversible vision loss in the ageing population. Increasing number and size of drusen is a characteristic of AMD disease progression. Ultra-widefield (UWF) retinal imaging with a scanning laser ophthalmoscope captures up to 80% of the retina in a single acquisition allowing a larger area of the retina to be assessed for signs of neurodegeneration than is possible with a conventional fundus camera, particularly the periphery. Quantification of changes to the microvasculature and drusen load could be used to derive early biomarkers of diseases that have vascular and neurodegenerative components such as AD and other forms of dementia.Manually grading drusen in UWF images is a difficult, subjective and a time-consuming process because the area imaged is large (around 700mm2) and drusen appear as small spots ( 0.8 and < 0.9), achieving AUC 0.55-0.59, 0.78-0.82 and 0.82-0.85 in the central, perimacular and peripheral zones, respectively. Measurements of the retinal vasculature appearing in UWF images of cognitively healthy (CH) individuals and patients diagnosed with mild cognitive impairment (MCI) and AD were obtained using a previously established pipeline. Following data cleaning, vascular measures were compared using multivariate generalised estimation equations (GEE), which accounts for the correlation between eyes of individuals with correction for confounders (e.g. age). The vascular measures were repeated for a subset of images and analysed using GEE to assess the repeatability of the results. When comparing AD with CH, the analysis showed a statistically significant difference between measurements of arterioles in the inferonasal quadrant, but fractal analysis produced inconsistent results due to differences in the area sampled in which the fractal dimension was calculated.When looking at drusen load, there was a higher abundance of drusen in the inferonasal region of the peripheral retina in the CH and AD compared to the MCI group. Using GEE analysis, there was evidence of a significant difference in drusen count when comparing MCI to CH (p = 0.02) and MCI to AD (p = 0.03), but no evidence of a difference when comparing AD to CH. However, given the low sensitivity of the system (partly the result of only moderate agreement between human observers), there will be a large proportion of drusen that are not detected giving an under estimation of the true amount of drusen present in an image. Overcoming this limitation will involve training the system using larger datasets and annotations from additional observers to create a more consistent reference standard. Further validation could then be performed in the future to determine if these promising pilot results persist, leading to candidate retinal biomarkers of AD

Detección automática de la presencia de patología ocular en retinografías empleando técnicas de procesado de imágenes

La vista es uno de los sentidos de mayor importancia para la vida humana. En los últimos años el número de enfermedades oculares ha aumentado y las predicciones de los científicos es que van a seguir aumentando en los próximos años. Existen enfermedades oculares que se han convertido en importantes causas de pérdida de visión a nivel mundial como la retinopatía diabética (RD), el glaucoma, la degeneración macular asociada a la edad (DMAE) y las cataratas. Estas enfermedades oculares suelen provocar alteraciones en el ojo humano, que pueden detectarse observando el ojo. Una de las técnicas más extendidas para observar el fondo del ojo es la retinografía, que es una imagen digital a color de la retina. Esta imagen es muy útil para el diagnóstico de enfermedades que afectan al ojo como RD y DMAE, entre otras. No obstante, la creciente incidencia de algunas enfermedades oculares y la escasez de oftalmólogos especialistas provoca que el análisis de las retinografías sea una tarea compleja y laboriosa. El objetivo de este Trabajo Fin de Grado (TFG) ha sido el diseño y desarrollo de un método automático para diferenciar entre retinografías patológicas y no patológicas. Este método permitiría ayudar en el diagnóstico y cribado de los pacientes con enfermedades oculares y reducir la carga de trabajo a los oftalmólogos. Para ello, se partió de una base de datos (BD) formada por 1044 imágenes de calidad adecuada para su procesado automático. De ellas, 326 pertenecían a sujetos sanos y a 819 pacientes con algún tipo de patología. Estas imágenes se dividieron en un conjunto de entrenamiento (559 imágenes) y un conjunto de test (585 imágenes). En todos los casos, un oftalmólogo especialista indicó si las imágenes eran normales o patológicas.Grado en Ingeniería de Tecnologías de Telecomunicació

Machine Learning Ensemble Methods for Classifying Multi-media Data

Multimedia data have, over recent years, been produced in many fields. They have important applications for such diverse areas as social media and healthcare, due to their capacity to capture rich information. However, their unstructured and separated nature gives rise to various problems. In particular, fusing and integrating multi-media datasets and finding effective ways to learn from them have proven to be major challenges for machine learning. In this thesis we investigated the development of the ensemble methods for classifying multi-media data in two key aspects: data fusion and model selection. For the data fusion, we devised two different strategies. The first one is the Feature Level Ensemble Method (FLEM) that aggregates all the features into a single dataset and then generates the models to build ensembles using this dataset. The second one is the Decision Level Ensemble Method (DLEM) that generates the models from each sub dataset individually and then aggregates their outputs with a decision fusion function. For the model selection we derived four different model selection rules. The first rule, R0, uses just the accuracy to select models. The rules R1 and R2 use firstly accuracy and then diversity to select models. In R3, we defined a generalised function that combines the accuracy and diversity with different weights to select models to build an ensemble. Our methods were compared with existing well known ensemble methods using the same dataset and another dataset that became available after our methods had been developed. The results were critically analysed and the statistical significance analyses of the results show that our methods had better performance in general and the generalised R3 is the most effective rule in building ensembles

Clasificación automática de la severidad de la retinopatía diabética mediante técnicas de Deep Learning

La retinopatía diabética (RD) es una de las principales enfermedades crónicas discapacitantes y una de las principales causas de ceguera y discapacidad visual en los países desarrollados. Los estudios indican que el 90% de los casos pueden prevenirse mediante la detección precoz y el tratamiento adecuado. Los médicos utilizan el cribado ocular mediante imágenes de la retina para detectar las lesiones relacionadas con esta enfermedad en exámenes oftalmológicos periódicos. Debido al creciente número de personas diabéticas, la cantidad de imágenes que los oftalmólogos especialistas han de analizar manualmente se está volviendo inasequible. Además, la formación de nuevo personal para este tipo de diagnóstico basado en imágenes es larga, ya que requiere adquirir experiencia mediante la práctica diaria. En este trabajo se propone el desarrollo y evaluación de un método automático para la clasificación del grado de severidad de la RD basado en el análisis de retinografías. Dicho método permitiría identificar las imágenes que muestren signos de RD y determinar qué imágenes pertenecen a pacientes que deban ser derivados a atención especializada de forma preferente u ordinaria, según el estadio de gravedad detectado por el sistema. Para este fin, se ha empleado como clasificador una red neuronal profunda combinada con diferentes técnicas de deep learning tales como data augmentation, dropout, transfer learning y fine tuning. El método se evaluó en la base de datos pública de retinografías APTOS-2019, en tres escenarios diferentes: clasificación de la RD en cinco grados de severidad (sano, RD no proliferativa leve, RD no proliferativa moderada, RD no proliferativa severa y RD proliferativa), detección de la presencia de la RD (clasificación en sano vs RD) y clasificación en casos derivables vs no derivables. Estos escenarios se abordan habitualmente en los trabajos existentes sobre el diagnóstico de la RD. En el escenario de clasificación multiclase se ha obtenido un coeficiente kappa de 0.919, una especificidad de 96.31%, un Area Under Curve (AUC) de 0.93 y una precisión de 94.10%. Para el caso de la detección de la presencia de la RD se ha obtenido un coeficiente kappa de 0.978, una sensibilidad de 97.99%, una especificidad de 100%, un AUC de 1 y una precisión de 98.91%. Finalmente, en la detección de los casos derivables y no derivables se ha conseguido un coeficiente kappa de 0.811, una sensibilidad de 85.15%, una especificidad del 100%, un AUC de 0.97 y una precisión de 90.71%. Además, como los resultados proporcionados por los modelos basados en deep learning son difíciles de interpretar, se ha incluido un análisis de los resultados utilizando técnicas de Explainable Artificial Intelligence (XAI). En concreto, se ha utilizado SHapley Additive exPlanations (SHAP). Los resultados obtenidos permiten comprobar que es posible tanto la detección y clasificación de la RD en casos derivables y no derivables como la clasificación automática de la severidad de la RD mediante el análisis de retinografías de pacientes diabéticos. El método propuesto, por tanto, permitiría acortar el tiempo de obtención de un diagnóstico, reducir la carga de trabajo de los expertos oftalmólogos y, como consecuencia, los costes económicos asociados al tratamiento de la RD. Además, el modelo está diseñado de tal forma que se puede generalizar su posible aplicación a otros tipos de imágenes médicas y dominios de clasificación.Diabetic retinopathy (DR) is a major chronic disabling disease and a leading cause of blindness and visual impairment in developed countries. Studies indicate that 90% of cases are preventable through early detection and appropriate treatment. Eye screening using retinal imaging is used by physicians to detect lesions related to DR. Due to the increasing number of diabetic patients, the amount of images that specialist ophthalmologists have to analyse manually is becoming unaffordable. Furthermore, the training of specialists for this type of image-based diagnosis is time-consuming, as it requires gaining experience through daily practice. This work proposes the development and evaluation of an automatic method for the classification of DR severity based on the analysis of fundus images. This method would make it possible to identify images showing signs of DR and to determine which images belong to patients who should be referred to specialised care on a preferential or ordinary basis, depending on the severity detected by the system. For this purpose, a deep neural network was used as a classifier, in combination with different deep learning techniques such as data augmentation, dropout, transfer learning and fine tuning. The method was evaluated on the APTOS-2019 public retinography database in three different scenarios: classification of DR into five severity grades (healthy, mild non-proliferative DR, moderate non-proliferative DR, severe non-proliferative DR and proliferative DR), detection of the presence of DR (classification into healthy vs. DR cases) and classification into referable vs. non-referable cases. These scenarios are commonly addressed in the existing literature related to the diagnosis of DR. In the multiclass classification scenario, a kappa coefficient of 0.919, a specificity of 96.31%, an Area Under Curve (AUC) of 0.93 and an accuracy of 94.10% were obtained. In the case of detecting the presence of DR, a kappa coefficient of 0.978, a sensitivity of 97.99%, a specificity of 100%, an AUC of 1 and an accuracy of 98.91% were obtained. Finally, in the detection of referable and non-referable cases, a kappa coefficient of 0.811, a sensitivity of 85.15%, a specificity of 100%, an AUC of 0.97 and an accuracy of 90.71% were achieved. In addition, as the results provided by deep learning-based models are difficult to interpret, an analysis of the results using Explainable Artificial Intelligence (XAI) techniques has been included. Specifically, SHapley Additive exPlanations (SHAP) have been used. The results obtained show that it is possible both to detect and classify DR in referable and non-referable cases and to automatically classify the severity of DR by analysing retinographs of diabetic patients. The proposed method would therefore shorten the time it takes to obtain a diagnosis, reduce the workload of ophthalmic experts and, as a consequence, the economic costs associated with the treatment of DR. Furthermore, the model is designed in such a way that it can be generalised to other types of medical imaging and classification domains.Departamento de Teoría de la Señal y Comunicaciones e Ingeniería TelemáticaMáster en Ingeniería de Telecomunicació