Computational Analysis of Fundus Images: Rule-Based and Scale-Space Models

Fundus images are one of the most important imaging examinations in modern ophthalmology because they are simple, inexpensive and, above all, noninvasive. Nowadays, the acquisition and storage of highresolution fundus images is relatively easy and fast. Therefore, fundus imaging has become a fundamental investigation in retinal lesion detection, ocular health monitoring and screening programmes. Given the large volume and clinical complexity associated with these images, their analysis and interpretation by trained clinicians becomes a timeconsuming task and is prone to human error. Therefore, there is a growing interest in developing automated approaches that are affordable and have high sensitivity and specificity. These automated approaches need to be robust if they are to be used in the general population to diagnose and track retinal diseases. To be effective, the automated systems must be able to recognize normal structures and distinguish them from pathological clinical manifestations. The main objective of the research leading to this thesis was to develop automated systems capable of recognizing and segmenting retinal anatomical structures and retinal pathological clinical manifestations associated with the most common retinal diseases. In particular, these automated algorithms were developed on the premise of robustness and efficiency to deal with the difficulties and complexity inherent in these images. Four objectives were considered in the analysis of fundus images. Segmentation of exudates, localization of the optic disc, detection of the midline of blood vessels, segmentation of the vascular network and detection of microaneurysms. In addition, we also evaluated the detection of diabetic retinopathy on fundus images using the microaneurysm detection method. An overview of the state of the art is presented to compare the performance of the developed approaches with the main methods described in the literature for each of the previously described objectives. To facilitate the comparison of methods, the state of the art has been divided into rulebased methods and machine learningbased methods. In the research reported in this paper, rulebased methods based on image processing methods were preferred over machine learningbased methods. In particular, scalespace methods proved to be effective in achieving the set goals. Two different approaches to exudate segmentation were developed. The first approach is based on scalespace curvature in combination with the local maximum of a scalespace blob detector and dynamic thresholds. The second approach is based on the analysis of the distribution function of the maximum values of the noise map in combination with morphological operators and adaptive thresholds. Both approaches perform a correct segmentation of the exudates and cope well with the uneven illumination and contrast variations in the fundus images. Optic disc localization was achieved using a new technique called cumulative sum fields, which was combined with a vascular enhancement method. The algorithm proved to be reliable and efficient, especially for pathological images. The robustness of the method was tested on 8 datasets. The detection of the midline of the blood vessels was achieved using a modified corner detector in combination with binary philtres and dynamic thresholding. Segmentation of the vascular network was achieved using a new scalespace blood vessels enhancement method. The developed methods have proven effective in detecting the midline of blood vessels and segmenting vascular networks. The microaneurysm detection method relies on a scalespace microaneurysm detection and labelling system. A new approach based on the neighbourhood of the microaneurysms was used for labelling. Microaneurysm detection enabled the assessment of diabetic retinopathy detection. The microaneurysm detection method proved to be competitive with other methods, especially with highresolution images. Diabetic retinopathy detection with the developed microaneurysm detection method showed similar performance to other methods and human experts. The results of this work show that it is possible to develop reliable and robust scalespace methods that can detect various anatomical structures and pathological features of the retina. Furthermore, the results obtained in this work show that although recent research has focused on machine learning methods, scalespace methods can achieve very competitive results and typically have greater independence from image acquisition. The methods developed in this work may also be relevant for the future definition of new descriptors and features that can significantly improve the results of automated methods.As imagens do fundo do olho são hoje um dos principais exames imagiológicos da oftalmologia moderna, pela sua simplicidade, baixo custo e acima de tudo pelo seu carácter nãoinvasivo. A aquisição e armazenamento de imagens do fundo do olho com alta resolução é também relativamente simples e rápida. Desta forma, as imagens do fundo do olho são um exame fundamental na identificação de alterações retinianas, monitorização da saúde ocular, e em programas de rastreio. Considerando o elevado volume e complexidade clínica associada a estas imagens, a análise e interpretação das mesmas por clínicos treinados tornase uma tarefa morosa e propensa a erros humanos. Assim, há um interesse crescente no desenvolvimento de abordagens automatizadas, acessíveis em custo, e com uma alta sensibilidade e especificidade. Estas devem ser robustas para serem aplicadas à população em geral no diagnóstico e seguimento de doenças retinianas. Para serem eficazes, os sistemas de análise têm que conseguir detetar e distinguir estruturas normais de sinais patológicos. O objetivo principal da investigação que levou a esta tese de doutoramento é o desenvolvimento de sistemas automáticos capazes de detetar e segmentar as estruturas anatómicas da retina, e os sinais patológicos retinianos associados às doenças retinianas mais comuns. Em particular, estes algoritmos automatizados foram desenvolvidos segundo as premissas de robustez e eficácia para lidar com as dificuldades e complexidades inerentes a estas imagens. Foram considerados quatro objetivos de análise de imagens do fundo do olho. São estes, a segmentação de exsudados, a localização do disco ótico, a deteção da linha central venosa dos vasos sanguíneos e segmentação da rede vascular, e a deteção de microaneurismas. De acrescentar que usando o método de deteção de microaneurismas, avaliouse também a capacidade de deteção da retinopatia diabética em imagens do fundo do olho. Para comparar o desempenho das metodologias desenvolvidas neste trabalho, foi realizado um levantamento do estado da arte, onde foram considerados os métodos mais relevantes descritos na literatura para cada um dos objetivos descritos anteriormente. Para facilitar a comparação entre métodos, o estado da arte foi dividido em metodologias de processamento de imagem e baseadas em aprendizagem máquina. Optouse no trabalho de investigação desenvolvido pela utilização de metodologias de análise espacial de imagem em detrimento de metodologias baseadas em aprendizagem máquina. Em particular, as metodologias baseadas no espaço de escalas mostraram ser efetivas na obtenção dos objetivos estabelecidos. Para a segmentação de exsudados foram usadas duas abordagens distintas. A primeira abordagem baseiase na curvatura em espaço de escalas em conjunto com a resposta máxima local de um detetor de manchas em espaço de escalas e limiares dinâmicos. A segunda abordagem baseiase na análise do mapa de distribuição de ruído em conjunto com operadores morfológicos e limiares adaptativos. Ambas as abordagens fazem uma segmentação dos exsudados de elevada precisão, além de lidarem eficazmente com a iluminação nãouniforme e a variação de contraste presente nas imagens do fundo do olho. A localização do disco ótico foi conseguida com uma nova técnica designada por campos de soma acumulativos, combinada com métodos de melhoramento da rede vascular. O algoritmo revela ser fiável e eficiente, particularmente em imagens patológicas. A robustez do método foi verificada pela sua avaliação em oito bases de dados. A deteção da linha central dos vasos sanguíneos foi obtida através de um detetor de cantos modificado em conjunto com filtros binários e limiares dinâmicos. A segmentação da rede vascular foi conseguida com um novo método de melhoramento de vasos sanguíneos em espaço de escalas. Os métodos desenvolvidos mostraram ser eficazes na deteção da linha central dos vasos sanguíneos e na segmentação da rede vascular. Finalmente, o método para a deteção de microaneurismas assenta num formalismo de espaço de escalas na deteção e na rotulagem dos microaneurismas. Para a rotulagem foi utilizada uma nova abordagem da vizinhança dos candidatos a microaneurismas. A deteção de microaneurismas permitiu avaliar também a deteção da retinopatia diabética. O método para a deteção de microaneurismas mostrou ser competitivo quando comparado com outros métodos, em particular em imagens de alta resolução. A deteção da retinopatia diabética exibiu um desempenho semelhante a outros métodos e a especialistas humanos. Os trabalhos descritos nesta tese mostram ser possível desenvolver uma abordagem fiável e robusta em espaço de escalas capaz de detetar diferentes estruturas anatómicas e sinais patológicos da retina. Além disso, os resultados obtidos mostram que apesar de a pesquisa mais recente concentrarse em metodologias de aprendizagem máquina, as metodologias de análise espacial apresentam resultados muito competitivos e tipicamente independentes do equipamento de aquisição das imagens. As metodologias desenvolvidas nesta tese podem ser importantes na definição de novos descritores e características, que podem melhorar significativamente o resultado de métodos automatizados

Automated Retinal Lesion Detection via Image Saliency Analysis

Background and objective:The detection of abnormalities such as lesions or leakage from retinal images is an important health informatics task for automated early diagnosis of diabetic and malarial retinopathy or other eye diseases, in order to prevent blindness and common systematic conditions. In this work, we propose a novel retinal lesion detection method by adapting the concepts of saliency. Methods :Retinal images are firstly segmented as superpixels, two new saliency feature representations: uniqueness and compactness, are then derived to represent the superpixels. The pixel level saliency is then estimated from these superpixel saliency values via a bilateral filter. These extracted saliency features form a matrix for low-rank analysis to achieve saliency detection. The precise contour of a lesion is finally extracted from the generated saliency map after removing confounding structures such as blood vessels, the optic disc, and the fovea. The main novelty of this method is that it is an effective tool for detecting different abnormalities at pixel-level from different modalities of retinal images, without the need to tune parameters. Results:To evaluate its effectiveness, we have applied our method to seven public datasets of diabetic and malarial retinopathy with four different types of lesions: exudate, hemorrhage, microaneurysms, and leakage. The evaluation was undertaken at pixel-level, lesion-level, or image-level according to ground truth availability in these datasets. Conclusions:The experimental results show that the proposed method outperforms existing state-of-the-art ones in applicability, effectiveness, and accuracy

Automatic Detection of Hard Exudates in Color Retinal Images Using Dynamic Threshold and SVM Classification: Algorithm Development and Evaluation

Diabetic retinopathy (DR) is one of the most common causes of visual impairment. Automatic detection of hard exudates (HE) from retinal photographs is an important step for detection of DR. However, most of existing algorithms for HE detection are complex and inefficient. We have developed and evaluated an automatic retinal image processing algorithm for HE detection using dynamic threshold and fuzzy C-means clustering (FCM) followed by support vector machine (SVM) for classification. The proposed algorithm consisted of four main stages: (i) imaging preprocessing; (ii) localization of optic disc (OD); (iii) determination of candidate HE using dynamic threshold in combination with global threshold based on FCM; and (iv) extraction of eight texture features from the candidate HE region, which were then fed into an SVM classifier for automatic HE classification. The proposed algorithm was trained and cross-validated (10 fold) on a publicly available e-ophtha EX database (47 images) on pixel-level, achieving the overall average sensitivity, PPV, and F-score of 76.5%, 82.7%, and 76.7%. It was tested on another independent DIARETDB1 database (89 images) with the overall average sensitivity, specificity, and accuracy of 97.5%, 97.8%, and 97.7%, respectively. In summary, the satisfactory evaluation results on both retinal imaging databases demonstrated the effectiveness of our proposed algorithm for automatic HE detection, by using dynamic threshold and FCM followed by an SVM for classification

A Multi-Anatomical Retinal Structure Segmentation System For Automatic Eye Screening Using Morphological Adaptive Fuzzy Thresholding

Eye exam can be as efficacious as physical one in determining health concerns. Retina screening can be the very first clue to detecting a variety of hidden health issues including pre-diabetes and diabetes. Through the process of clinical diagnosis and prognosis; ophthalmologists rely heavily on the binary segmented version of retina fundus image; where the accuracy of segmented vessels, optic disc and abnormal lesions extremely affects the diagnosis accuracy which in turn affect the subsequent clinical treatment steps. This thesis proposes an automated retinal fundus image segmentation system composed of three segmentation subsystems follow same core segmentation algorithm. Despite of broad difference in features and characteristics; retinal vessels, optic disc and exudate lesions are extracted by each subsystem without the need for texture analysis or synthesis. For sake of compact diagnosis and complete clinical insight, our proposed system can detect these anatomical structures in one session with high accuracy even in pathological retina images. The proposed system uses a robust hybrid segmentation algorithm combines adaptive fuzzy thresholding and mathematical morphology. The proposed system is validated using four benchmark datasets: DRIVE and STARE (vessels), DRISHTI-GS (optic disc), and DIARETDB1 (exudates lesions). Competitive segmentation performance is achieved, outperforming a variety of up-to-date systems and demonstrating the capacity to deal with other heterogenous anatomical structures

Application of deep learning techniques for biomedical data analysis

Deep learning and machine learning methods have been used for addressing the problems in the biomedical applications, such as diabetic retinopathy assessment and Parkinson's disease diagnosis. The severity of diabetic retinopathy is estimated by the expert's examination of fundus images based on the amount and location of three diabetic retinopathy signs (i.e., exudates, hemorrhages, and microaneurysms). An automatic and accurate system for detection of these signs can significantly help clinicians to make the best possible prognosis can result in reducing the risk of vision loss. For Parkinson's disease diagnosis, analysis of a speech voice is considered as the earliest symptom with the advantage of being non-intrusive and suitable for online applications. While some reported outcomes of the developed techniques have shown the good results and ongoing progress for these two applications, designing new algorithms is a thriving research field to overcome the poor sensitivity and specificity of the outcomes as well as the limitations such as dataset size and heuristic selection of the network parameters. This thesis has comprehensively studied and developed various deep learning frameworks for detection of diabetic retinopathy signs and diagnosis of Parkinson's disease. To improve the performance of the current systems, this work has had an investigation on different techniques: (i) color space investigation, (ii) examination of various deep learning methods, (iii) development of suitable pre/post-processing algorithms and (iv) appropriate selection of deep learning architectures and parameters. For diabetic retinopathy assessment, this thesis has proposed the new color space as the input for the deep learning models that obtained better replicability compared with the conventional color spaces. This has also shown the pre-trained model can extract more relevant features compared to the models which were trained from scratch. This has also presented a deep learning framework combined with the suitable pre and post-processing algorithms that increased the performance of the system. By investigation different architectures and parameters, the suitable deep learning model has been presented to distinguish between Parkinson's disease and healthy speech signal

Deep Learning Techniques for Automated Analysis and Processing of High Resolution Medical Imaging

Programa Oficial de Doutoramento en Computación . 5009V01[Abstract] Medical imaging plays a prominent role in modern clinical practice for numerous medical specialties. For instance, in ophthalmology, different imaging techniques are commonly used to visualize and study the eye fundus. In this context, automated image analysis methods are key towards facilitating the early diagnosis and adequate treatment of several diseases. Nowadays, deep learning algorithms have already demonstrated a remarkable performance for different image analysis tasks. However, these approaches typically require large amounts of annotated data for the training of deep neural networks. This complicates the adoption of deep learning approaches, especially in areas where large scale annotated datasets are harder to obtain, such as in medical imaging. This thesis aims to explore novel approaches for the automated analysis of medical images, particularly in ophthalmology. In this regard, the main focus is on the development of novel deep learning-based approaches that do not require large amounts of annotated training data and can be applied to high resolution images. For that purpose, we have presented a novel paradigm that allows to take advantage of unlabeled complementary image modalities for the training of deep neural networks. Additionally, we have also developed novel approaches for the detailed analysis of eye fundus images. In that regard, this thesis explores the analysis of relevant retinal structures as well as the diagnosis of different retinal diseases. In general, the developed algorithms provide satisfactory results for the analysis of the eye fundus, even when limited annotated training data is available.[Resumen] Las técnicas de imagen tienen un papel destacado en la práctica clínica moderna de numerosas especialidades médicas. Por ejemplo, en oftalmología es común el uso de diferentes técnicas de imagen para visualizar y estudiar el fondo de ojo. En este contexto, los métodos automáticos de análisis de imagen son clave para facilitar el diagnóstico precoz y el tratamiento adecuado de diversas enfermedades. En la actualidad, los algoritmos de aprendizaje profundo ya han demostrado un notable rendimiento en diferentes tareas de análisis de imagen. Sin embargo, estos métodos suelen necesitar grandes cantidades de datos etiquetados para el entrenamiento de las redes neuronales profundas. Esto complica la adopción de los métodos de aprendizaje profundo, especialmente en áreas donde los conjuntos masivos de datos etiquetados son más difíciles de obtener, como es el caso de la imagen médica. Esta tesis tiene como objetivo explorar nuevos métodos para el análisis automático de imagen médica, concretamente en oftalmología. En este sentido, el foco principal es el desarrollo de nuevos métodos basados en aprendizaje profundo que no requieran grandes cantidades de datos etiquetados para el entrenamiento y puedan aplicarse a imágenes de alta resolución. Para ello, hemos presentado un nuevo paradigma que permite aprovechar modalidades de imagen complementarias no etiquetadas para el entrenamiento de redes neuronales profundas. Además, también hemos desarrollado nuevos métodos para el análisis en detalle de las imágenes del fondo de ojo. En este sentido, esta tesis explora el análisis de estructuras retinianas relevantes, así como el diagnóstico de diferentes enfermedades de la retina. En general, los algoritmos desarrollados proporcionan resultados satisfactorios para el análisis de las imágenes de fondo de ojo, incluso cuando la disponibilidad de datos de entrenamiento etiquetados es limitada.[Resumo] As técnicas de imaxe teñen un papel destacado na práctica clínica moderna de numerosas especialidades médicas. Por exemplo, en oftalmoloxía é común o uso de diferentes técnicas de imaxe para visualizar e estudar o fondo de ollo. Neste contexto, os métodos automáticos de análises de imaxe son clave para facilitar o diagn ostico precoz e o tratamento adecuado de diversas enfermidades. Na actualidade, os algoritmos de aprendizaxe profunda xa demostraron un notable rendemento en diferentes tarefas de análises de imaxe. Con todo, estes métodos adoitan necesitar grandes cantidades de datos etiquetos para o adestramento das redes neuronais profundas. Isto complica a adopción dos métodos de aprendizaxe profunda, especialmente en áreas onde os conxuntos masivos de datos etiquetados son máis difíciles de obter, como é o caso da imaxe médica. Esta tese ten como obxectivo explorar novos métodos para a análise automática de imaxe médica, concretamente en oftalmoloxía. Neste sentido, o foco principal é o desenvolvemento de novos métodos baseados en aprendizaxe profunda que non requiran grandes cantidades de datos etiquetados para o adestramento e poidan aplicarse a imaxes de alta resolución. Para iso, presentamos un novo paradigma que permite aproveitar modalidades de imaxe complementarias non etiquetadas para o adestramento de redes neuronais profundas. Ademais, tamén desenvolvemos novos métodos para a análise en detalle das imaxes do fondo de ollo. Neste sentido, esta tese explora a análise de estruturas retinianas relevantes, así como o diagnóstico de diferentes enfermidades da retina. En xeral, os algoritmos desenvolvidos proporcionan resultados satisfactorios para a análise das imaxes de fondo de ollo, mesmo cando a dispoñibilidade de datos de adestramento etiquetados é limitada

Modélisation statistique des structures anatomiques de la rétine à partir d'images de fond d'oeil

L’examen non-invasif du fond d’oeil permet d’identifier sur la rétine les signes de nombreuses pathologies oculaires qui développent de graves symptômes pour le patient pouvant entraîner la cécité. Le réseau vasculaire rétinien peut de surcroît présenter des signes précurseurs de pathologies cardiovasculaires et cérébro-vasculaires. La rétine, où apparaissent ces pathologies, est constituée de plusieurs structures anatomiques dont la variabilité est importante au sein d’une population saine. Pour autant, les évaluations cliniques actuelles ne prennent pas en compte cette variabilité ce qui ne permet pas de détecter précocement ces pathologies. Ces évaluations se basent sur un ensemble restreint de mesures prélevées à partir de structures dont la segmentation manuelle est réalisable par les experts. De plus, elles sont basées sur un seuillage empirique déterminé par les cliniciens et appliqué sur chacune des mesures afin d’établir un diagnostic. Ainsi, les évaluations cliniques actuelles sont affectées par la grande variabilité des structures anatomiques de la rétine au sein de la population et elles n’évaluent pas les anomalies trop difficiles à mesurer manuellement. Dans ce contexte, il convient de proposer de nouvelles mesures cliniques qui tiennent compte de la variabilité normale à l’aide d’une modélisation statistique des structures anatomiques de la rétine. Cette modélisation statistique permet de mieux comprendre et identifier ce qui est normal et comment l’anatomie et ses attributs varient au sein d’une population saine. Cela permet ainsi d’identifier la présence de pathologies à l’aide de nouvelles mesures cliniques construites en tenant compte de la variabilité des attributs de l’anatomie. La modélisation statistique des structures anatomiques de la rétine est cependant difficile étant donné les variations morphologiques et topologiques de ces structures. Les changements morphologiques et topologiques du réseau vasculaire rétinien compliquent son analyse statistique ainsi que les outils de recalage, de segmentation et de représentation sémantique s’y appliquant. Les questions de recherches adressées dans cette thèse sont la production d’outils capables d’analyser la variabilité des structures anatomiques de la rétine et l’élaboration de nouvelles mesures cliniques tenant compte de la variabilité normale de ces structures. Pour répondre à ces questions de recherche, trois objectifs de recherche sont formulés. ----------ABSTRACT: Non-invasive retinal fundus examination allows clinicians to identify signs of many ocular conditions that develop critical symptoms affecting the patient and even leading to blindness. In addition, the retinal vascular network may present early signs of cardiovascular and cerebrovascular diseases. The retina, where these pathologies appear, is composed of several anatomical structures whose variability is considerable within a healthy population. Yet, current clinical evaluations do not take into account this variability, and this does not allow early detection of these pathologies. These evaluations are based on a limited set of measurements taken from structures whose manual segmentation is achievable by the experts. In addition, they are based on empirical thresholding determined by the clinicians and applied to each of the measurements to establish a diagnosis. Thus, current clinical assessments are affected by the large variability of anatomical structures of the retina within a healthy population and do not evaluate abnormalities that are too difficult to measure manually. In this context, it is advisable to propose new clinical measurements that take into account the normal variability using statistical modeling of the anatomical structures of the retina. Such a statistical modeling approach helps us to better understand and identify what is normal and how the anatomy and its attributes vary across a healthy population. This makes it possible to identify the presence of pathologies using new clinical measurements constructed by taking into account the variability of the anatomy’s attributes. Statistical modeling of the anatomical structures of the retina is difficult, however, given the morphological and topological variations of these structures. Morphological and topological changes in the retinal vascular network complicate its statistical analysis as well as the registration methods, segmentation and semantic representation applied to it. The research questions proposed in this thesis pertain to creating tools capable of analyzing the variability of the anatomical structures of the retina and proposing new clinical measures that take into account the normal variability of those structures. To answer these research questions, three research objectives are formulated