Deep learning analysis of eye fundus images to support medical diagnosis

Machine learning techniques have been successfully applied to support medical decision making of cancer, heart diseases and degenerative diseases of the brain. In particular, deep learning methods have been used for early detection of abnormalities in the eye that could improve the diagnosis of different ocular diseases, especially in developing countries, where there are major limitations to access to specialized medical treatment. However, the early detection of clinical signs such as blood vessel, optic disc alterations, exudates, hemorrhages, drusen, and microaneurysms presents three main challenges: the ocular images can be affected by noise artifact, the features of the clinical signs depend specifically on the acquisition source, and the combination of local signs and grading disease label is not an easy task. This research approaches the problem of combining local signs and global labels of different acquisition sources of medical information as a valuable tool to support medical decision making in ocular diseases. Different models for different eye diseases were developed. Four models were developed using eye fundus images: for DME, it was designed a two-stages model that uses a shallow model to predict an exudate binary mask. Then, the binary mask is stacked with the raw fundus image into a 4-channel array as an input of a deep convolutional neural network for diabetic macular edema diagnosis; for glaucoma, it was developed three deep learning models. First, it was defined a deep learning model based on three-stages that contains an initial stage for automatically segment two binary masks containing optic disc and physiological cup segmentation, followed by an automatic morphometric features extraction stage from previous segmentations, and a final classification stage that supports the glaucoma diagnosis with intermediate medical information. Two late-data-fusion methods that fused morphometric features from cartesian and polar segmentation of the optic disc and physiological cup with features extracted from raw eye fundus images. On the other hand, two models were defined using optical coherence tomography. First, a customized convolutional neural network termed as OCT-NET to extract features from OCT volumes to classify DME, DR-DME and AMD conditions. In addition, this model generates images with highlighted local information about the clinical signs, and it estimates the number of slides inside a volume with local abnormalities. Finally, a 3D-Deep learning model that uses OCT volumes as an input to estimate the retinal thickness map useful to grade AMD. The methods were systematically evaluated using ten free public datasets. The methods were compared and validated against other state-of-the-art algorithms and the results were also qualitatively evaluated by ophthalmology experts from Fundación Oftalmológica Nacional. In addition, the proposed methods were tested as a diagnosis support tool of diabetic macular edema, glaucoma, diabetic retinopathy and age-related macular degeneration using two different ocular imaging representations. Thus, we consider that this research could be potentially a big step in building telemedicine tools that could support medical personnel for detecting ocular diseases using eye fundus images and optical coherence tomography.Las técnicas de aprendizaje automático se han aplicado con éxito para apoyar la toma de decisiones médicas sobre el cáncer, las enfermedades cardíacas y las enfermedades degenerativas del cerebro. En particular, se han utilizado métodos de aprendizaje profundo para la detección temprana de anormalidades en el ojo que podrían mejorar el diagnóstico de diferentes enfermedades oculares, especialmente en países en desarrollo, donde existen grandes limitaciones para acceder a tratamiento médico especializado. Sin embargo, la detección temprana de signos clínicos como vasos sanguíneos, alteraciones del disco óptico, exudados, hemorragias, drusas y microaneurismas presenta tres desafíos principales: las imágenes oculares pueden verse afectadas por artefactos de ruido, las características de los signos clínicos dependen específicamente de fuente de adquisición, y la combinación de signos locales y clasificación de la enfermedad no es una tarea fácil. Esta investigación aborda el problema de combinar signos locales y etiquetas globales de diferentes fuentes de adquisición de información médica como una herramienta valiosa para apoyar la toma de decisiones médicas en enfermedades oculares. Se desarrollaron diferentes modelos para diferentes enfermedades oculares. Se desarrollaron cuatro modelos utilizando imágenes de fondo de ojo: para DME, se diseñó un modelo de dos etapas que utiliza un modelo superficial para predecir una máscara binaria de exudados. Luego, la máscara binaria se apila con la imagen de fondo de ojo original en una matriz de 4 canales como entrada de una red neuronal convolucional profunda para el diagnóstico de edema macular diabético; para el glaucoma, se desarrollaron tres modelos de aprendizaje profundo. Primero, se definió un modelo de aprendizaje profundo basado en tres etapas que contiene una etapa inicial para segmentar automáticamente dos máscaras binarias que contienen disco óptico y segmentación fisiológica de la copa, seguido de una etapa de extracción de características morfométricas automáticas de segmentaciones anteriores y una etapa de clasificación final que respalda el diagnóstico de glaucoma con información médica intermedia. Dos métodos de fusión de datos tardíos que fusionaron características morfométricas de la segmentación cartesiana y polar del disco óptico y la copa fisiológica con características extraídas de imágenes de fondo de ojo crudo. Por otro lado, se definieron dos modelos mediante tomografía de coherencia óptica. Primero, una red neuronal convolucional personalizada denominada OCT-NET para extraer características de los volúmenes OCT para clasificar las condiciones DME, DR-DME y AMD. Además, este modelo genera imágenes con información local resaltada sobre los signos clínicos, y estima el número de diapositivas dentro de un volumen con anomalías locales. Finalmente, un modelo de aprendizaje 3D-Deep que utiliza volúmenes OCT como entrada para estimar el mapa de espesor retiniano útil para calificar AMD. Los métodos se evaluaron sistemáticamente utilizando diez conjuntos de datos públicos gratuitos. Los métodos se compararon y validaron con otros algoritmos de vanguardia y los resultados también fueron evaluados cualitativamente por expertos en oftalmología de la Fundación Oftalmológica Nacional. Además, los métodos propuestos se probaron como una herramienta de diagnóstico de edema macular diabético, glaucoma, retinopatía diabética y degeneración macular relacionada con la edad utilizando dos representaciones de imágenes oculares diferentes. Por lo tanto, consideramos que esta investigación podría ser potencialmente un gran paso en la construcción de herramientas de telemedicina que podrían ayudar al personal médico a detectar enfermedades oculares utilizando imágenes de fondo de ojo y tomografía de coherencia óptica.Doctorad

Assessment of total retinal blood flow using Doppler Fourier Domain Optical Coherence Tomography during systemic hypercapnia and hypocapnia.

The purpose of this study was to investigate changes in total retinal blood flow (RBF) using Doppler Fourier Domain Optical Coherence Tomography (Doppler FD-OCT) in response to the manipulation of systemic partial pressure of CO2 (PETCO2). Double circular Doppler blood flow scans were captured in nine healthy individuals (mean age ± standard deviation: 27.1 ± 4.1, six males) using the RTVue(™) FD-OCT (Optovue). PETCO2 was manipulated using a custom-designed computer-controlled gas blender (RespirAct(™)) connected to a sequential gas delivery rebreathing circuit. Doppler FD-OCT measurements were captured at baseline, during stages of hypercapnia (+5/+10/+15 mmHg PETCO2), return to baseline and during stages of hypocapnia (-5/-10/-15 mmHg PETCO2). Repeated measures analysis of variance (reANOVA) and Tukeys post hoc analysis were used to compare Doppler FD-OCT measurements between the various PETCO2 levels relative to baseline. The effect of PETCO2 on TRBF was also investigated using linear regression models. The average RBF significantly increased by 15% (P < 0.0001) with an increase in PETCO2 and decreased significantly by 10% with a decrease in PETCO2 (P = 0.001). Venous velocity significantly increased by 3.11% from baseline to extreme hypercapnia (P < 0.001) and reduced significantly by 2.01% at extreme hypocapnia (P = 0.012). No significant changes were found in the average venous area measurements under hypercapnia (P = 0.36) or hypocapnia (P = 0.40). Overall, increased and decreased PETCO2 values had a significant effect on RBF outcomes (P < 0.002). In healthy individuals, altered end-tidal CO2 levels significantly changed RBF as measured by Doppler FD-OCT

Retinal Fundus Image Analysis for Diagnosis of Glaucoma: A Comprehensive Survey

© 2016 IEEE. The rapid development of digital imaging and computer vision has increased the potential of using the image processing technologies in ophthalmology. Image processing systems are used in standard clinical practices with the development of medical diagnostic systems. The retinal images provide vital information about the health of the sensory part of the visual system. Retinal diseases, such as glaucoma, diabetic retinopathy, age-related macular degeneration, Stargardt's disease, and retinopathy of prematurity, can lead to blindness manifest as artifacts in the retinal image. An automated system can be used for offering standardized large-scale screening at a lower cost, which may reduce human errors, provide services to remote areas, as well as free from observer bias and fatigue. Treatment for retinal diseases is available; the challenge lies in finding a cost-effective approach with high sensitivity and specificity that can be applied to large populations in a timely manner to identify those who are at risk at the early stages of the disease. The progress of the glaucoma disease is very often quiet in the early stages. The number of people affected has been increasing and patients are seldom aware of the disease, which can cause delay in the treatment. A review of how computer-aided approaches may be applied in the diagnosis and staging of glaucoma is discussed here. The current status of the computer technology is reviewed, covering localization and segmentation of the optic nerve head, pixel level glaucomatic changes, diagonosis using 3-D data sets, and artificial neural networks for detecting the progression of the glaucoma disease

Automated microaneurysm detection algorithms applied to diabetic retinopathy retinal images

Diabetic retinopathy is the commonest cause of blindness in working age people. It is characterised and graded by the development of retinal microaneurysms, haemorrhages and exudates. The damage caused by diabetic retinopathy can be prevented if it is treated in its early stages. Therefore, automated early detection can limit the severity of the disease, improve the follow-up management of diabetic patients and assist ophthalmologists in investigating and treating the disease more efficiently. This review focuses on microaneurysm detection as the earliest clinically localised characteristic of diabetic retinopathy, a frequently observed complication in both Type 1 and Type 2 diabetes. Algorithms used for microaneurysm detection from retinal images are reviewed. A number of features used to extract microaneurysm are summarised. Furthermore, a comparative analysis of reported methods used to automatically detect microaneurysms is presented and discussed. The performance of methods and their complexity are also discussed

Technological Advances in the Diagnosis and Management of Pigmented Fundus Tumours

Choroidal naevi are the most common intraocular tumour. They can be pigmented or non-pigmented and have a predilection for the posterior uvea. The majority remain undetected and cause no harm but are increasingly found on routine community optometry examinations. Rarely does a naevus demonstrate growth or the onset of suspicious features to fulfil the criteria for a malignant melanoma. Because of this very small risk, optometrists commonly refer these patients to hospital eye units for a second opinion, triggering specialist examination and investigation, causing significant anxiety to patients and stretching medical resources. This PhD thesis introduces the MOLES acronym and scoring system that has been devised to categorise the risk of malignancy in choroidal melanocytic tumours according to Mushroom tumour shape, Orange pigment, Large tumour size, Enlarging tumour and Subretinal fluid. This is a simplified system that can be used without sophisticated imaging, and hence its main utility lies in the screening of patients with choroidal pigmented lesions in the community and general ophthalmology clinics. Under this system, lesions were categorised by a scoring system as ‘common naevus’, ‘low-risk naevus’, ‘high-risk naevus’ and ‘probable melanoma.’ According to the sum total of the scores, the MOLES system correlates well with ocular oncologists’ final diagnosis. The PhD thesis also describes a model of managing such lesions in a virtual pathway, showing that images of choroidal naevi evaluated remotely using a decision-making algorithm by masked non-medical graders or masked ophthalmologists is safe. This work prospectively validates a virtual naevus clinic model focusing on patient safety as the primary consideration. The idea of a virtual naevus clinic as a fast, one-stop, streamlined and comprehensive service is attractive for patients and healthcare systems, including an optimised patient experience with reduced delays and inconvenience from repeated visits. A safe, standardised model ensures homogeneous management of cases, appropriate and prompt return of care closer to home to community-based optometrists. This research work and strategies, such as the MOLES scoring system for triage, could empower community-based providers to deliver management of benign choroidal naevi without referral to specialist units. Based on the positive outcome of this prospective study and the MOLES studies, a ‘Virtual Naevus Clinic’ has been designed and adapted at Moorfields Eye Hospital (MEH) to prove its feasibility as a response to the COVID-19 pandemic, and with the purpose of reducing in-hospital patient journey times and increasing the capacity of the naevus clinics, while providing safe and efficient clinical care for patients. This PhD chapter describes the design, pathways, and operating procedures for the digitally enabled naevus clinics in Moorfields Eye Hospital, including what this service provides and how it will be delivered and supported. The author will share the current experience and future plan. Finally, the PhD thesis will cover a chapter that discusses the potential role of artificial intelligence (AI) in differentiating benign choroidal naevus from choroidal melanoma. The published clinical and imaging risk factors for malignant transformation of choroidal naevus will be reviewed in the context of how AI applied to existing ophthalmic imaging systems might be able to determine features on medical images in an automated way. The thesis will include current knowledge to date and describe potential benefits, limitations and key issues that could arise with this technology in the ophthalmic field. Regulatory concerns will be addressed with possible solutions on how AI could be implemented in clinical practice and embedded into existing imaging technology with the potential to improve patient care and the diagnostic process. The PhD will also explore the feasibility of developed automated deep learning models and investigate the performance of these models in diagnosing choroidal naevomelanocytic lesions based on medical imaging, including colour fundus and autofluorescence fundus photographs. This research aimed to determine the sensitivity and specificity of an automated deep learning algorithm used for binary classification to differentiate choroidal melanomas from choroidal naevi and prove that a differentiation concept utilising a machine learning algorithm is feasible

Machine learning methods for the characterization and classification of complex data

This thesis work presents novel methods for the analysis and classification of medical images and, more generally, complex data. First, an unsupervised machine learning method is proposed to order anterior chamber OCT (Optical Coherence Tomography) images according to a patient's risk of developing angle-closure glaucoma. In a second study, two outlier finding techniques are proposed to improve the results of above mentioned machine learning algorithm, we also show that they are applicable to a wide variety of data, including fraud detection in credit card transactions. In a third study, the topology of the vascular network of the retina, considering it a complex tree-like network is analyzed and we show that structural differences reveal the presence of glaucoma and diabetic retinopathy. In a fourth study we use a model of a laser with optical injection that presents extreme events in its intensity time-series to evaluate machine learning methods to forecast such extreme events.El presente trabajo de tesis desarrolla nuevos métodos para el análisis y clasificación de imágenes médicas y datos complejos en general. Primero, proponemos un método de aprendizaje automático sin supervisión que ordena imágenes OCT (tomografía de coherencia óptica) de la cámara anterior del ojo en función del grado de riesgo del paciente de padecer glaucoma de ángulo cerrado. Luego, desarrollamos dos métodos de detección automática de anomalías que utilizamos para mejorar los resultados del algoritmo anterior, pero que su aplicabilidad va mucho más allá, siendo útil, incluso, para la detección automática de fraudes en transacciones de tarjetas de crédito. Mostramos también, cómo al analizar la topología de la red vascular de la retina considerándola una red compleja, podemos detectar la presencia de glaucoma y de retinopatía diabética a través de diferencias estructurales. Estudiamos también un modelo de un láser con inyección óptica que presenta eventos extremos en la serie temporal de intensidad para evaluar diferentes métodos de aprendizaje automático para predecir dichos eventos extremos.Aquesta tesi desenvolupa nous mètodes per a l’anàlisi i la classificació d’imatges mèdiques i dades complexes. Hem proposat, primer, un mètode d’aprenentatge automàtic sense supervisió que ordena imatges OCT (tomografia de coherència òptica) de la cambra anterior de l’ull en funció del grau de risc del pacient de patir glaucoma d’angle tancat. Després, hem desenvolupat dos mètodes de detecció automàtica d’anomalies que hem utilitzat per millorar els resultats de l’algoritme anterior, però que la seva aplicabilitat va molt més enllà, sent útil, fins i tot, per a la detecció automàtica de fraus en transaccions de targetes de crèdit. Mostrem també, com en analitzar la topologia de la xarxa vascular de la retina considerant-la una xarxa complexa, podem detectar la presència de glaucoma i de retinopatia diabètica a través de diferències estructurals. Finalment, hem estudiat un làser amb injecció òptica, el qual presenta esdeveniments extrems en la sèrie temporal d’intensitat. Hem avaluat diferents mètodes per tal de predir-los.Postprint (published version