The automated detection of proliferative diabetic retinopathy using dual ensemble classification

Objective: Diabetic retinopathy (DR) is a retinal vascular disease that is caused by complications of diabetes. Proliferative diabetic retinopathy (PDR) is the advanced stage of the disease which carries a high risk of severe visual impairment. This stage is characterized by the growth of abnormal new vessels. We aim to develop a method for the automated detection of new vessels from retinal images. Methods: This method is based on a dual classification approach. Two vessel segmentation approaches are applied to create two separate binary vessel maps which each hold vital information. Local morphology, gradient and intensity features are measured using each binary vessel map to produce two separate 21-D feature vectors. Independent classification is performed for each feature vector using an ensemble system of bagged decision trees. These two independent outcomes are then combined to a produce a final decision. Results: Sensitivity and specificity results using a dataset of 60 images are 1.0000 and 0.9500 on a per image basis. Conclusions: The described automated system is capable of detecting the presence of new vessels

Deep learning of the retina enables phenome- and genome-wide analyses of the microvasculature.

Background: The microvasculature, the smallest blood vessels in the body, has key roles in maintenance of organ health as well as tumorigenesis. The retinal fundus is a window for human in vivo non-invasive assessment of the microvasculature. Large-scale complementary machine learning-based assessment of the retinal vasculature with phenome-wide and genome-wide analyses may yield new insights into human health and disease. Methods: We utilized 97,895 retinal fundus images from 54,813 UK Biobank participants. Using convolutional neural networks to segment the retinal microvasculature, we calculated fractal dimension (FD) as a measure of vascular branching complexity, and vascular density. We associated these indices with 1,866 incident ICD-based conditions (median 10y follow-up) and 88 quantitative traits, adjusting for age, sex, smoking status, and ethnicity. Results: Low retinal vascular FD and density were significantly associated with higher risks for incident mortality, hypertension, congestive heart failure, renal failure, type 2 diabetes, sleep apnea, anemia, and multiple ocular conditions, as well as corresponding quantitative traits. Genome-wide association of vascular FD and density identified 7 and 13 novel loci respectively, which were enriched for pathways linked to angiogenesis (e.g., VEGF, PDGFR, angiopoietin, and WNT signaling pathways) and inflammation (e.g., interleukin, cytokine signaling). Conclusions: Our results indicate that the retinal vasculature may serve as a biomarker for future cardiometabolic and ocular disease and provide insights on genes and biological pathways influencing microvascular indices. Moreover, such a framework highlights how deep learning of images can quantify an interpretable phenotype for integration with electronic health records, biomarker, and genetic data to inform risk prediction and risk modification

Detection and Classification of Diabetic Retinopathy Pathologies in Fundus Images

Diabetic Retinopathy (DR) is a disease that affects up to 80% of diabetics around the world. It is the second greatest cause of blindness in the Western world, and one of the leading causes of blindness in the U.S. Many studies have demonstrated that early treatment can reduce the number of sight-threatening DR cases, mitigating the medical and economic impact of the disease. Accurate, early detection of eye disease is important because of its potential to reduce rates of blindness worldwide. Retinal photography for DR has been promoted for decades for its utility in both disease screening and clinical research studies. In recent years, several research centers have presented systems to detect pathology in retinal images. However, these approaches apply specialized algorithms to detect specific types of lesion in the retina. In order to detect multiple lesions, these systems generally implement multiple algorithms. Furthermore, some of these studies evaluate their algorithms on a single dataset, thus avoiding potential problems associated with the differences in fundus imaging devices, such as camera resolution. These methodologies primarily employ bottom-up approaches, in which the accurate segmentation of all the lesions in the retina is the basis for correct determination. A disadvantage of bottom-up approaches is that they rely on the accurate segmentation of all lesions in order to measure performance. On the other hand, top-down approaches do not depend on the segmentation of specific lesions. Thus, top-down methods can potentially detect abnormalities not explicitly used in their training phase. A disadvantage of these methods is that they cannot identify specific pathologies and require large datasets to build their training models. In this dissertation, I merged the advantages of the top-down and bottom-up approaches to detect DR with high accuracy. First, I developed an algorithm based on a top-down approach to detect abnormalities in the retina due to DR. By doing so, I was able to evaluate DR pathologies other than microaneurysms and exudates, which are the main focus of most current approaches. In addition, I demonstrated good generalization capacity of this algorithm by applying it to other eye diseases, such as age-related macular degeneration. Due to the fact that high accuracy is required for sight-threatening conditions, I developed two bottom-up approaches, since it has been proven that bottom-up approaches produce more accurate results than top-down approaches for particular structures. Consequently, I developed an algorithm to detect exudates in the macula. The presence of this pathology is considered to be a surrogate for clinical significant macular edema (CSME), a sight-threatening condition of DR. The analysis of the optic disc is usually not taken into account in DR screening systems. However, there is a pathology called neovascularization that is present in advanced stages of DR, making its detection of crucial clinical importance. In order to address this problem, I developed an algorithm to detect neovascularization in the optic disc. These algorithms are based on amplitude-modulation and frequency-modulation (AM-FM) representations, morphological image processing methods, and classification algorithms. The methods were tested on a diverse set of large databases and are considered to be the state-of the art in this field

Genetic algorithm based feature selection combined with dual classification for the automated detection of proliferative diabetic retinopathy

Proliferative diabetic retinopathy (PDR) is a condition that carries a high risk of severe visual impairment. The hallmark of PDR is the growth of abnormal new vessels. In this paper, an automated method for the detection of new vessels from retinal images is presented. This method is based on a dual classification approach. Two vessel segmentation approaches are applied to create two separate binary vessel map which each hold vital information. Local morphology features are measured from each binary vessel map to produce two separate 4-D feature vectors. Independent classification is performed for each feature vector using a support vector machine (SVM) classifier. The system then combines these individual outcomes to produce a final decision. This is followed by the creation of additional features to generate 21-D feature vectors, which feed into a genetic algorithm based feature selection approach with the objective of finding feature subsets that improve the performance of the classification. Sensitivity and specificity results using a dataset of 60 images are 0.9138 and 0.9600, respectively, on a per patch basis and 1.000 and 0.975, respectively, on a per image basis

Retinal Vascular Measurement Tools for Diagnostic Feature Extraction

The contributions of this work are in the development of new and state of the art algorithms for retinal image analysis including optic disc detection, tortuosity estimation, and cross-over abnormality detection. The retina is one of the only areas of the human body that blood vessels can be visualized noninvasively. Retinal imaging has become a standard in the ophthalmologist’s office because it is an easy and inexpensive way to monitor not just eye health, but also systemic vascular diseases. Changes to the retinal vasculature can be the early signs of diseases such as diabetic and hypertensive retinopathy, of which early detection can save vision, money, and improve overall health for the patient. When looking at the retinal vasculature, ophthalmologists generally rely on a qualitative assessment which can make comparisons over time or between different ophthalmologists difficult. Computer aided systems are now able to quantify what the ophthalmologist is qualitatively measuring in what they consider to be the most important features of the vasculature. These include, but are not limited to, tortuosity, arteriolar narrowing, cross-over abnormalities, and artery-vein (AV) ratio. The University of Padova has created a semi-automatic system for detecting and quantifying retinal vessels starting from optic disc detection, vessel segmentation, width estimation, tortuosity calculation, AV classification, and AV ratio. We propose a new method for optic disc detection that converts the retinal image into a graph and exploits vessel enhancement methods to calculate edge weights in finding the shortest path between pairs of points on the periphery of the image. The line segment with the maximum number of shortest paths is considered the optic disc location. The method was tested on three publicly available datasets: DRIVE, DIARETDB1, and Messidor consisting of 40, 89, and 1200 images and achieved an accuracy of 100, 98.88, and 99.42% respectively. The second contribution is a new algorithm for calculating abnormalities at AV crossing points. In retinal images, Gunn’s sign appears as a tapering of the vein at a crossing point, while Salus’s sign presents as an S-shaped curving. This work presents a method for the automatic quantification of these two signs once a crossover has been detected; combining segmentation, artery vein classification, and morphological feature extraction techniques to calculate vein widths and angles entering and exiting the crossover. Results on two datasets show separation between the two classes and that we can reliably detect and quantify these signs under the right conditions. The last contribution in tortuosity consists of two parts. A comparative study was performed on several of the most popular methods for tortuosity estimation on a new vessel dataset. Results show that several methods have good Cohen’s kappa agreement with both graders, while the tortuosity density metric has the highest single metric average agreement across vessel type and grader. The second is a new way to enhance curvature in segmented vessels based on a difference of Gabor filters to create a curvature enhanced image. The proposed method was tested on the RET-TORT database using several methods to calculate tortuosity, and had best Pearson’s correlation of .94 for arteries and .882 for veins, outperforming single mathematical formulations on the data. This held true after testing the method on the propose dataset as well, having higher correlation values across grader and vessel type compared with other tortuosity metrics. Summary of Results: The optic disc detection method was tested on three publicly available datasets: DRIVE, DIARETDB1, and Messidor consisting of 40, 89, and 1200 images and achieved an accuracy of 100, 98.88, and 99.42% respectively. The AV nicking quantification method was tested on a small dataset of 10 crossing provided by doctors at Papageorgiou Hospital, Aristotle University of Thessaloniki, Thessaloniki, Greece. Results showed separation between the normal and abnormal classes for both the Gunn and Salus sign. The method was then tested on a larger, publicly available dataset which showed good separation for the Gunn sign. The proposed tortuosity method was tested on the RET-TORT database using several methods to calculate tortuosity, and had best Pearson’s correlation of .94 for arteries and .882 for veins, outperforming single mathematical formulations on the data. It was then tested on the dataset proposed in this thesis, further corroborating the effectiveness of the method

Automatic segmentation and classification methods using optical coherence tomography angiography (Octa): A review and handbook

Optical coherence tomography angiography (OCTA) is a promising technology for the non-invasive imaging of vasculature. Many studies in literature present automated algorithms to quantify OCTA images, but there is a lack of a review on the most common methods and their comparison considering multiple clinical applications (e.g., ophthalmology and dermatology). Here, we aim to provide readers with a useful review and handbook for automatic segmentation and classification methods using OCTA images, presenting a comparison of techniques found in the literature based on the adopted segmentation or classification method and on the clinical application. Another goal of this study is to provide insight into the direction of research in automated OCTA image analysis, especially in the current era of deep learning

Computational Analysis of Fundus Images: Rule-Based and Scale-Space Models

Fundus images are one of the most important imaging examinations in modern ophthalmology because they are simple, inexpensive and, above all, noninvasive. Nowadays, the acquisition and storage of highresolution fundus images is relatively easy and fast. Therefore, fundus imaging has become a fundamental investigation in retinal lesion detection, ocular health monitoring and screening programmes. Given the large volume and clinical complexity associated with these images, their analysis and interpretation by trained clinicians becomes a timeconsuming task and is prone to human error. Therefore, there is a growing interest in developing automated approaches that are affordable and have high sensitivity and specificity. These automated approaches need to be robust if they are to be used in the general population to diagnose and track retinal diseases. To be effective, the automated systems must be able to recognize normal structures and distinguish them from pathological clinical manifestations. The main objective of the research leading to this thesis was to develop automated systems capable of recognizing and segmenting retinal anatomical structures and retinal pathological clinical manifestations associated with the most common retinal diseases. In particular, these automated algorithms were developed on the premise of robustness and efficiency to deal with the difficulties and complexity inherent in these images. Four objectives were considered in the analysis of fundus images. Segmentation of exudates, localization of the optic disc, detection of the midline of blood vessels, segmentation of the vascular network and detection of microaneurysms. In addition, we also evaluated the detection of diabetic retinopathy on fundus images using the microaneurysm detection method. An overview of the state of the art is presented to compare the performance of the developed approaches with the main methods described in the literature for each of the previously described objectives. To facilitate the comparison of methods, the state of the art has been divided into rulebased methods and machine learningbased methods. In the research reported in this paper, rulebased methods based on image processing methods were preferred over machine learningbased methods. In particular, scalespace methods proved to be effective in achieving the set goals. Two different approaches to exudate segmentation were developed. The first approach is based on scalespace curvature in combination with the local maximum of a scalespace blob detector and dynamic thresholds. The second approach is based on the analysis of the distribution function of the maximum values of the noise map in combination with morphological operators and adaptive thresholds. Both approaches perform a correct segmentation of the exudates and cope well with the uneven illumination and contrast variations in the fundus images. Optic disc localization was achieved using a new technique called cumulative sum fields, which was combined with a vascular enhancement method. The algorithm proved to be reliable and efficient, especially for pathological images. The robustness of the method was tested on 8 datasets. The detection of the midline of the blood vessels was achieved using a modified corner detector in combination with binary philtres and dynamic thresholding. Segmentation of the vascular network was achieved using a new scalespace blood vessels enhancement method. The developed methods have proven effective in detecting the midline of blood vessels and segmenting vascular networks. The microaneurysm detection method relies on a scalespace microaneurysm detection and labelling system. A new approach based on the neighbourhood of the microaneurysms was used for labelling. Microaneurysm detection enabled the assessment of diabetic retinopathy detection. The microaneurysm detection method proved to be competitive with other methods, especially with highresolution images. Diabetic retinopathy detection with the developed microaneurysm detection method showed similar performance to other methods and human experts. The results of this work show that it is possible to develop reliable and robust scalespace methods that can detect various anatomical structures and pathological features of the retina. Furthermore, the results obtained in this work show that although recent research has focused on machine learning methods, scalespace methods can achieve very competitive results and typically have greater independence from image acquisition. The methods developed in this work may also be relevant for the future definition of new descriptors and features that can significantly improve the results of automated methods.As imagens do fundo do olho são hoje um dos principais exames imagiológicos da oftalmologia moderna, pela sua simplicidade, baixo custo e acima de tudo pelo seu carácter nãoinvasivo. A aquisição e armazenamento de imagens do fundo do olho com alta resolução é também relativamente simples e rápida. Desta forma, as imagens do fundo do olho são um exame fundamental na identificação de alterações retinianas, monitorização da saúde ocular, e em programas de rastreio. Considerando o elevado volume e complexidade clínica associada a estas imagens, a análise e interpretação das mesmas por clínicos treinados tornase uma tarefa morosa e propensa a erros humanos. Assim, há um interesse crescente no desenvolvimento de abordagens automatizadas, acessíveis em custo, e com uma alta sensibilidade e especificidade. Estas devem ser robustas para serem aplicadas à população em geral no diagnóstico e seguimento de doenças retinianas. Para serem eficazes, os sistemas de análise têm que conseguir detetar e distinguir estruturas normais de sinais patológicos. O objetivo principal da investigação que levou a esta tese de doutoramento é o desenvolvimento de sistemas automáticos capazes de detetar e segmentar as estruturas anatómicas da retina, e os sinais patológicos retinianos associados às doenças retinianas mais comuns. Em particular, estes algoritmos automatizados foram desenvolvidos segundo as premissas de robustez e eficácia para lidar com as dificuldades e complexidades inerentes a estas imagens. Foram considerados quatro objetivos de análise de imagens do fundo do olho. São estes, a segmentação de exsudados, a localização do disco ótico, a deteção da linha central venosa dos vasos sanguíneos e segmentação da rede vascular, e a deteção de microaneurismas. De acrescentar que usando o método de deteção de microaneurismas, avaliouse também a capacidade de deteção da retinopatia diabética em imagens do fundo do olho. Para comparar o desempenho das metodologias desenvolvidas neste trabalho, foi realizado um levantamento do estado da arte, onde foram considerados os métodos mais relevantes descritos na literatura para cada um dos objetivos descritos anteriormente. Para facilitar a comparação entre métodos, o estado da arte foi dividido em metodologias de processamento de imagem e baseadas em aprendizagem máquina. Optouse no trabalho de investigação desenvolvido pela utilização de metodologias de análise espacial de imagem em detrimento de metodologias baseadas em aprendizagem máquina. Em particular, as metodologias baseadas no espaço de escalas mostraram ser efetivas na obtenção dos objetivos estabelecidos. Para a segmentação de exsudados foram usadas duas abordagens distintas. A primeira abordagem baseiase na curvatura em espaço de escalas em conjunto com a resposta máxima local de um detetor de manchas em espaço de escalas e limiares dinâmicos. A segunda abordagem baseiase na análise do mapa de distribuição de ruído em conjunto com operadores morfológicos e limiares adaptativos. Ambas as abordagens fazem uma segmentação dos exsudados de elevada precisão, além de lidarem eficazmente com a iluminação nãouniforme e a variação de contraste presente nas imagens do fundo do olho. A localização do disco ótico foi conseguida com uma nova técnica designada por campos de soma acumulativos, combinada com métodos de melhoramento da rede vascular. O algoritmo revela ser fiável e eficiente, particularmente em imagens patológicas. A robustez do método foi verificada pela sua avaliação em oito bases de dados. A deteção da linha central dos vasos sanguíneos foi obtida através de um detetor de cantos modificado em conjunto com filtros binários e limiares dinâmicos. A segmentação da rede vascular foi conseguida com um novo método de melhoramento de vasos sanguíneos em espaço de escalas. Os métodos desenvolvidos mostraram ser eficazes na deteção da linha central dos vasos sanguíneos e na segmentação da rede vascular. Finalmente, o método para a deteção de microaneurismas assenta num formalismo de espaço de escalas na deteção e na rotulagem dos microaneurismas. Para a rotulagem foi utilizada uma nova abordagem da vizinhança dos candidatos a microaneurismas. A deteção de microaneurismas permitiu avaliar também a deteção da retinopatia diabética. O método para a deteção de microaneurismas mostrou ser competitivo quando comparado com outros métodos, em particular em imagens de alta resolução. A deteção da retinopatia diabética exibiu um desempenho semelhante a outros métodos e a especialistas humanos. Os trabalhos descritos nesta tese mostram ser possível desenvolver uma abordagem fiável e robusta em espaço de escalas capaz de detetar diferentes estruturas anatómicas e sinais patológicos da retina. Além disso, os resultados obtidos mostram que apesar de a pesquisa mais recente concentrarse em metodologias de aprendizagem máquina, as metodologias de análise espacial apresentam resultados muito competitivos e tipicamente independentes do equipamento de aquisição das imagens. As metodologias desenvolvidas nesta tese podem ser importantes na definição de novos descritores e características, que podem melhorar significativamente o resultado de métodos automatizados

Retinal vessel segmentation using textons

Segmenting vessels from retinal images, like segmentation in many other medical image domains, is a challenging task, as there is no unified way that can be adopted to extract the vessels accurately. However, it is the most critical stage in automatic assessment of various forms of diseases (e.g. Glaucoma, Age-related macular degeneration, diabetic retinopathy and cardiovascular diseases etc.). Our research aims to investigate retinal image segmentation approaches based on textons as they provide a compact description of texture that can be learnt from a training set. This thesis presents a brief review of those diseases and also includes their current situations, future trends and techniques used for their automatic diagnosis in routine clinical applications. The importance of retinal vessel segmentation is particularly emphasized in such applications. An extensive review of previous work on retinal vessel segmentation and salient texture analysis methods is presented. Five automatic retinal vessel segmentation methods are proposed in this thesis. The first method focuses on addressing the problem of removing pathological anomalies (Drusen, exudates) for retinal vessel segmentation, which have been identified by other researchers as a problem and a common source of error. The results show that the modified method shows some improvement compared to a previously published method. The second novel supervised segmentation method employs textons. We propose a new filter bank (MR11) that includes bar detectors for vascular feature extraction and other kernels to detect edges and photometric variations in the image. The k-means clustering algorithm is adopted for texton generation based on the vessel and non-vessel elements which are identified by ground truth. The third improved supervised method is developed based on the second one, in which textons are generated by k-means clustering and texton maps representing vessels are derived by back projecting pixel clusters onto hand labelled ground truth. A further step is implemented to ensure that the best combinations of textons are represented in the map and subsequently used to identify vessels in the test set. The experimental results on two benchmark datasets show that our proposed method performs well compared to other published work and the results of human experts. A further test of our system on an independent set of optical fundus images verified its consistent performance. The statistical analysis on experimental results also reveals that it is possible to train unified textons for retinal vessel segmentation. In the fourth method a novel scheme using Gabor filter bank for vessel feature extraction is proposed. The ii method is inspired by the human visual system. Machine learning is used to optimize the Gabor filter parameters. The experimental results demonstrate that our method significantly enhances the true positive rate while maintaining a level of specificity that is comparable with other approaches. Finally, we proposed a new unsupervised texton based retinal vessel segmentation method using derivative of SIFT and multi-scale Gabor filers. The lack of sufficient quantities of hand labelled ground truth and the high level of variability in ground truth labels amongst experts provides the motivation for this approach. The evaluation results reveal that our unsupervised segmentation method is comparable with the best other supervised methods and other best state of the art methods