Feature visualisation of classification of diabetic retinopathy using a convolutional neural network

Convolutional Neural Networks (CNNs) have been demonstrated to achieve state-of-the-art results on complex computer vision tasks, including medical image diagnosis of Diabetic Retinopathy (DR). CNNs are powerful because they determine relevant image features automatically. However, the current inability to demonstrate what these features are has led to CNNs being considered to be 'black box' methods whose results should not be trusted. This paper presents a method for identifying the learned features of a CNN and applies it in the context of the diagnosis of DR in fundus images using the well-known DenseNet. We train the CNN to diagnose and determine the severity of DR and then successfully extract feature maps from the CNN which identify the regions and features of the images which have led most strongly to the CNN prediction. This feature extraction process has great potential, particularly for encouraging confidence in CNN approaches from users and clinicians, and can aid in the further development of CNN methods. There is also potential for determining previously unidentified features which may contribute to a classification

A deep learning approach to the analysis of retinal images

This work is focused on the possible applications of Deep Learning in retinal fundus images analysis. Deep Learning is an advanced machine learning technique that is revolutionizing all data-based disciplines with unheard-of performances in signal analysis. In particular Artificial Intelligence, Computer Vision and Image Analysis are benefiting from huge improvement in image and sound classification, segmentation and pattern recognitionope

Towards Unsupervised Domain Adaptation for Diabetic Retinopathy Detection in the Tromsø Eye Study

Diabetic retinopathy (DR) is an eye disease which affects a third of the diabetic population. It is a preventable disease, but requires early detection for efficient treatment. While there has been increasing interest in applying deep learning techniques for DR detection in order to aid practitioners make more accurate diagnosis, these efforts are mainly focused on datasets that have been collected or created with ML in mind. In this thesis, however, we take a look at two particular datasets that have been collected at the University Hospital of North-Norway - UNN. These datasets have inherent problems that motivate the methodological choices in this work such as a variable number of input images and domain shift. We therefore contribute a multi-stream model for DR classification. The multi-stream model can model dependency across different images, can take in a variable of input of any size, is general in its detection such that the image processing is equal no matter which stream the image enters, and is compatible with the domain adaptation method ADDA, but we argue the model is compatible with many other methods. As a remedy for these problems, we propose a multi-stream deep learning architecture that is uniquely tailored to these datasets and illustrate how domain adaptation might be utilized within the framework to learn efficiently in the presence of domain shift. Our experiments demonstrates the models properties empirically, and shows it can deal with each of the presented problems. The model this paper contributes is a first step towards DR detection from these local datasets and, in the bigger picture, similar datasets worldwide

End-To-End Multi-Task Learning Approaches for the Joint Epiretinal Membrane Segmentation and Screening in OCT Images

Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG[Abstract] Background and objectives The Epiretinal Membrane (ERM) is an ocular disease that can cause visual distortions and irreversible vision loss. Patient sight preservation relies on an early diagnosis and on determining the location of the ERM in order to be treated and potentially removed. In this context, the visual inspection of the images in order to screen for ERM signs is a costly and subjective process. Methods In this work, we propose and study three end-to-end fully-automatic approaches for the simultaneous segmentation and screening of ERM signs in Optical Coherence Tomography images. These convolutional approaches exploit a multi-task learning context to leverage inter-task complementarity in order to guide the training process. The proposed architectures are combined with three different state of the art encoder architectures of reference in order to provide an exhaustive study of the suitability of each of the approaches for these tasks. Furthermore, these architectures work in an end-to-end manner, entailing a significant simplification of the development process since they are able to be trained directly from annotated images without the need for a series of purpose-specific steps. Results In terms of segmentation, the proposed models obtained a precision of 0.760 ± 0.050, a sensitivity of 0.768 ± 0.210 and a specificity of 0.945 ± 0.011. For the screening task, these models achieved a precision of 0.963 ± 0.068, a sensitivity of 0.816 ± 0.162 and a specificity of 0.983 ± 0.068. The obtained results show that these multi-task approaches are able to perform competitively with or even outperform single-task methods tailored for either the segmentation or the screening of the ERM. Conclusions These results highlight the advantages of using complementary knowledge related to the segmentation and screening tasks in the diagnosis of this relevant pathology, constituting the first proposal to address the diagnosis of the ERM from a multi-task perspective.This research was funded by Instituto de Salud Carlos III, Government of Spain, [grant number DTS18/00136]; Ministerio de Ciencia e Innovación y Universidades, Government of Spain, [grant number RTI2018-095894-B-I00]; Ministerio de Ciencia e Innovación, Government of Spain through the research project with [grant number PID2019-108435RB-I00]; Consellería de Cultura, Educación e Universidade, Xunta de Galicia, Grupos de Referencia Competitiva, [grant number ED431C 2020/24], Predoctoral grant [grant number ED481A 2021/161] and Postdoctoral grant [grant number ED481B 2021/059]; Axencia Galega de Innovación (GAIN), Xunta de Galicia, [grant number IN845D 2020/38]; CITIC, Centro de Investigación de Galicia [grant number ED431G 2019/01], receives financial support from Consellería de Educación, Universidade e Formación Profesional, Xunta de Galicia, through the ERDF (80%) and Secretaría Xeral de Universidades (20%). The funding sources had no role in the development of this work. Funding for open access charge: Universidade da Coruña/CISUGXunta de Galicia; ED431C 2020/24Xunta de Galicia; ED481A 2021/161Xunta de Galicia; ED481B 2021/059Xunta de Galicia; IN845D 2020/38Xunta de Galicia; ED431G 2019/0

Machine Learning Approaches for Automated Glaucoma Detection using Clinical Data and Optical Coherence Tomography Images

Glaucoma is a multi-factorial, progressive blinding optic-neuropathy. A variety of factors, including genetics, vasculature, anatomy, and immune factors, are involved. Worldwide more than 80 million people are affected by glaucoma, and around 300,000 in Australia, where 50% remain undiagnosed. Untreated glaucoma can lead to blindness. Early detection by Artificial intelligence (AI) is crucial to accelerate the diagnosis process and can prevent further vision loss. Many proposed AI systems have shown promising performance for automated glaucoma detection using two-dimensional (2D) data. However, only a few studies had optimistic outcomes for glaucoma detection and staging. Moreover, the automated AI system still faces challenges in diagnosing at the clinicians’ level due to the lack of interpretability of the ML algorithms and integration of multiple clinical data. AI technology would be welcomed by doctors and patients if the "black box" notion is overcome by developing an explainable, transparent AI system with similar pathological markers used by clinicians as the sign of early detection and progression of glaucomatous damage. Therefore, the thesis aimed to develop a comprehensive AI model to detect and stage glaucoma by incorporating a variety of clinical data and utilising advanced data analysis and machine learning (ML) techniques. The research first focuses on optimising glaucoma diagnostic features by combining structural, functional, demographic, risk factor, and optical coherence tomography (OCT) features. The significant features were evaluated using statistical analysis and trained in ML algorithms to observe the detection performance. Three crucial structural ONH OCT features: cross-sectional 2D radial B-scan, 3D vascular angiography and temporal-superior-nasal-inferior-temporal (TSNIT) B-scan, were analysed and trained in explainable deep learning (DL) models for automated glaucoma prediction. The explanation behind the decision making of DL models were successfully demonstrated using the feature visualisation. The structural features or distinguished affected regions of TSNIT OCT scans were precisely localised for glaucoma patients. This is consistent with the concept of explainable DL, which refers to the idea of making the decision-making processes of DL models transparent and interpretable to humans. However, artifacts and speckle noise often result in misinterpretation of the TSNIT OCT scans. This research also developed an automated DL model to remove the artifacts and noise from the OCT scans, facilitating error-free retinal layers segmentation, accurate tissue thickness estimation and image interpretation. Moreover, to monitor and grade glaucoma severity, the visual field (VF) test is commonly followed by clinicians for treatment and management. Therefore, this research uses the functional features extracted from VF images to train ML algorithms for staging glaucoma from early to advanced/severe stages. Finally, the selected significant features were used to design and develop a comprehensive AI model to detect and grade glaucoma stages based on the data quantity and availability. In the first stage, a DL model was trained with TSNIT OCT scans, and its output was combined with significant structural and functional features and trained in ML models. The best-performed ML model achieved an area under the curve (AUC): 0.98, an accuracy of 97.2%, a sensitivity of 97.9%, and a specificity of 96.4% for detecting glaucoma. The model achieved an overall accuracy of 90.7% and an F1 score of 84.0% for classifying normal, early, moderate, and advanced-stage glaucoma. In conclusion, this thesis developed and proposed a comprehensive, evidence-based AI model that will solve the screening problem for large populations and relieve experts from manually analysing a slew of patient data and associated misinterpretation problems. Moreover, this thesis demonstrated three structural OCT features that could be added as excellent diagnostic markers for precise glaucoma diagnosis

Diabetic Retinopathy Identification Using Parallel Convolutional Neural Network Based Feature Extractor and ELM Classifier

Diabetic retinopathy (DR) is an incurable retinal condition caused by excessive blood sugar that, if left untreated, can result in even blindness. A novel automated technique for DR detection has been proposed in this paper. To accentuate the lesions, the fundus images (FIs) were preprocessed using Contrast Limited Adaptive Histogram Equalization (CLAHE). A parallel convolutional neural network (PCNN) was employed for feature extraction and then the extreme learning machine (ELM) technique was utilized for the DR classification. In comparison to the similar CNN structure, the PCNN design uses fewer parameters and layers, which minimizes the time required to extract distinctive features. The effectiveness of the technique was evaluated on two datasets (Kaggle DR 2015 competition (Dataset 1; 34,984 FIs) and APTOS 2019 (3,662 FIs)), and the results are promising. For the two datasets mentioned, the proposed technique attained accuracies of 91.78 % and 97.27 % respectively. However, one of the study's subsidiary discoveries was that the proposed framework demonstrated stability for both larger and smaller datasets, as well as for balanced and imbalanced datasets. Furthermore, in terms of classifier performance metrics, model parameters and layers, and prediction time, the suggested approach outscored existing state-of-the-art models, which would add significant benefit for the medical practitioners in accurately identifying the DR

Towards PACE-CAD Systems

Despite phenomenal advancements in the availability of medical image datasets and the development of modern classification algorithms, Computer-Aided Diagnosis (CAD) has had limited practical exposure in the real-world clinical workflow. This is primarily because of the inherently demanding and sensitive nature of medical diagnosis that can have far-reaching and serious repercussions in case of misdiagnosis. In this work, a paradigm called PACE (Pragmatic, Accurate, Confident, & Explainable) is presented as a set of some of must-have features for any CAD. Diagnosis of glaucoma using Retinal Fundus Images (RFIs) is taken as the primary use case for development of various methods that may enrich an ordinary CAD system with PACE. However, depending on specific requirements for different methods, other application areas in ophthalmology and dermatology have also been explored. Pragmatic CAD systems refer to a solution that can perform reliably in day-to-day clinical setup. In this research two, of possibly many, aspects of a pragmatic CAD are addressed. Firstly, observing that the existing medical image datasets are small and not representative of images taken in the real-world, a large RFI dataset for glaucoma detection is curated and published. Secondly, realising that a salient attribute of a reliable and pragmatic CAD is its ability to perform in a range of clinically relevant scenarios, classification of 622 unique cutaneous diseases in one of the largest publicly available datasets of skin lesions is successfully performed. Accuracy is one of the most essential metrics of any CAD system's performance. Domain knowledge relevant to three types of diseases, namely glaucoma, Diabetic Retinopathy (DR), and skin lesions, is industriously utilised in an attempt to improve the accuracy. For glaucoma, a two-stage framework for automatic Optic Disc (OD) localisation and glaucoma detection is developed, which marked new state-of-the-art for glaucoma detection and OD localisation. To identify DR, a model is proposed that combines coarse-grained classifiers with fine-grained classifiers and grades the disease in four stages with respect to severity. Lastly, different methods of modelling and incorporating metadata are also examined and their effect on a model's classification performance is studied. Confidence in diagnosing a disease is equally important as the diagnosis itself. One of the biggest reasons hampering the successful deployment of CAD in the real-world is that medical diagnosis cannot be readily decided based on an algorithm's output. Therefore, a hybrid CNN architecture is proposed with the convolutional feature extractor trained using point estimates and a dense classifier trained using Bayesian estimates. Evaluation on 13 publicly available datasets shows the superiority of this method in terms of classification accuracy and also provides an estimate of uncertainty for every prediction. Explainability of AI-driven algorithms has become a legal requirement after Europe’s General Data Protection Regulations came into effect. This research presents a framework for easy-to-understand textual explanations of skin lesion diagnosis. The framework is called ExAID (Explainable AI for Dermatology) and relies upon two fundamental modules. The first module uses any deep skin lesion classifier and performs detailed analysis on its latent space to map human-understandable disease-related concepts to the latent representation learnt by the deep model. The second module proposes Concept Localisation Maps, which extend Concept Activation Vectors by locating significant regions corresponding to a learned concept in the latent space of a trained image classifier. This thesis probes many viable solutions to equip a CAD system with PACE. However, it is noted that some of these methods require specific attributes in datasets and, therefore, not all methods may be applied on a single dataset. Regardless, this work anticipates that consolidating PACE into a CAD system can not only increase the confidence of medical practitioners in such tools but also serve as a stepping stone for the further development of AI-driven technologies in healthcare

Automatic detection of microaneurysms in colour fundus images for diabetic retinopathy screening.

Regular eye screening is essential for the early detection and treatment of the diabetic retinopathy. This paper presents a novel automatic screening system for diabetic retinopathy that focuses on the detection of the earliest visible signs of retinopathy, which are microaneurysms. Microaneurysms are small dots on the retina, formed by ballooning out of a weak part of the capillary wall. The detection of the microaneurysms at an early stage is vital, and it is the first step in preventing the diabetic retinopathy. The paper first explores the existing systems and applications related to diabetic retinopathy screening, with a focus on the microaneurysm detection methods. The proposed decision support system consists of an automatic acquisition, screening and classification of diabetic retinopathy colour fundus images, which could assist in the detection and management of the diabetic retinopathy. Several feature extraction methods and the circular Hough transform have been employed in the proposed microaneurysm detection system, alongside the fuzzy histogram equalisation method. The latter method has been applied in the preprocessing stage of the diabetic retinopathy eye fundus images and provided improved results for detecting the microaneurysms

Detection of proliferative diabetic retinopathy in fundus images using convolution neural network

Convolution Neural Network (CNN) is one of the techniques under Artificial Neural Network (ANN) used to develop a Deep Learning Neural Network (DLNN) algorithm for detection of Proliferative Diabetic Retinopathy (PDR) on the fundus images. About 116 PDR and 150 Non-Proliferative Diabetic Retinopathy (NPDR) of fundus images retrieved from the publicly available MESSIDOR database applied in this research. This study consisted three objectives that included the execution of two pre-processing techniques on the data-set which were resizing and normalizing the fundus images, developed deep learning operational Artificial Intelligence (AI) network of feature extraction algorithm for detection of PDR on the fundus images and determined the output classification of the network encompassing the accuracy, sensitivity and specificity. There were five different parameters carried out along this research. Here, Parameter 5 showed the best performance among the five parameters based on the value of accuracy, sensitivity, and specificity that was 73.81%, 76%, and 69% respectively