ARA-net: an attention-aware retinal atrophy segmentation network coping with fundus images

BackgroundAccurately detecting and segmenting areas of retinal atrophy are paramount for early medical intervention in pathological myopia (PM). However, segmenting retinal atrophic areas based on a two-dimensional (2D) fundus image poses several challenges, such as blurred boundaries, irregular shapes, and size variation. To overcome these challenges, we have proposed an attention-aware retinal atrophy segmentation network (ARA-Net) to segment retinal atrophy areas from the 2D fundus image.MethodsIn particular, the ARA-Net adopts a similar strategy as UNet to perform the area segmentation. Skip self-attention connection (SSA) block, comprising a shortcut and a parallel polarized self-attention (PPSA) block, has been proposed to deal with the challenges of blurred boundaries and irregular shapes of the retinal atrophic region. Further, we have proposed a multi-scale feature flow (MSFF) to challenge the size variation. We have added the flow between the SSA connection blocks, allowing for capturing considerable semantic information to detect retinal atrophy in various area sizes.ResultsThe proposed method has been validated on the Pathological Myopia (PALM) dataset. Experimental results demonstrate that our method yields a high dice coefficient (DICE) of 84.26%, Jaccard index (JAC) of 72.80%, and F1-score of 84.57%, which outperforms other methods significantly.ConclusionOur results have demonstrated that ARA-Net is an effective and efficient approach for retinal atrophic area segmentation in PM

A method for quantifying sectoral optic disc pallor in fundus photographs and its association with peripapillary RNFL thickness

Purpose: To develop an automatic method of quantifying optic disc pallor in fundus photographs and determine associations with peripapillary retinal nerve fibre layer (pRNFL) thickness. Methods: We used deep learning to segment the optic disc, fovea, and vessels in fundus photographs, and measured pallor. We assessed the relationship between pallor and pRNFL thickness derived from optical coherence tomography scans in 118 participants. Separately, we used images diagnosed by clinical inspection as pale (N=45) and assessed how measurements compared to healthy controls (N=46). We also developed automatic rejection thresholds, and tested the software for robustness to camera type, image format, and resolution. Results: We developed software that automatically quantified disc pallor across several zones in fundus photographs. Pallor was associated with pRNFL thickness globally (\b{eta} = -9.81 (SE = 3.16), p < 0.05), in the temporal inferior zone (\b{eta} = -29.78 (SE = 8.32), p < 0.01), with the nasal/temporal ratio (\b{eta} = 0.88 (SE = 0.34), p < 0.05), and in the whole disc (\b{eta} = -8.22 (SE = 2.92), p < 0.05). Furthermore, pallor was significantly higher in the patient group. Lastly, we demonstrate the analysis to be robust to camera type, image format, and resolution. Conclusions: We developed software that automatically locates and quantifies disc pallor in fundus photographs and found associations between pallor measurements and pRNFL thickness. Translational relevance: We think our method will be useful for the identification, monitoring and progression of diseases characterized by disc pallor/optic atrophy, including glaucoma, compression, and potentially in neurodegenerative disorders.Comment: 44 pages, 20 figures, 7 tables, submitte

Machine Learning Approaches for Automated Glaucoma Detection using Clinical Data and Optical Coherence Tomography Images

Glaucoma is a multi-factorial, progressive blinding optic-neuropathy. A variety of factors, including genetics, vasculature, anatomy, and immune factors, are involved. Worldwide more than 80 million people are affected by glaucoma, and around 300,000 in Australia, where 50% remain undiagnosed. Untreated glaucoma can lead to blindness. Early detection by Artificial intelligence (AI) is crucial to accelerate the diagnosis process and can prevent further vision loss. Many proposed AI systems have shown promising performance for automated glaucoma detection using two-dimensional (2D) data. However, only a few studies had optimistic outcomes for glaucoma detection and staging. Moreover, the automated AI system still faces challenges in diagnosing at the clinicians’ level due to the lack of interpretability of the ML algorithms and integration of multiple clinical data. AI technology would be welcomed by doctors and patients if the "black box" notion is overcome by developing an explainable, transparent AI system with similar pathological markers used by clinicians as the sign of early detection and progression of glaucomatous damage. Therefore, the thesis aimed to develop a comprehensive AI model to detect and stage glaucoma by incorporating a variety of clinical data and utilising advanced data analysis and machine learning (ML) techniques. The research first focuses on optimising glaucoma diagnostic features by combining structural, functional, demographic, risk factor, and optical coherence tomography (OCT) features. The significant features were evaluated using statistical analysis and trained in ML algorithms to observe the detection performance. Three crucial structural ONH OCT features: cross-sectional 2D radial B-scan, 3D vascular angiography and temporal-superior-nasal-inferior-temporal (TSNIT) B-scan, were analysed and trained in explainable deep learning (DL) models for automated glaucoma prediction. The explanation behind the decision making of DL models were successfully demonstrated using the feature visualisation. The structural features or distinguished affected regions of TSNIT OCT scans were precisely localised for glaucoma patients. This is consistent with the concept of explainable DL, which refers to the idea of making the decision-making processes of DL models transparent and interpretable to humans. However, artifacts and speckle noise often result in misinterpretation of the TSNIT OCT scans. This research also developed an automated DL model to remove the artifacts and noise from the OCT scans, facilitating error-free retinal layers segmentation, accurate tissue thickness estimation and image interpretation. Moreover, to monitor and grade glaucoma severity, the visual field (VF) test is commonly followed by clinicians for treatment and management. Therefore, this research uses the functional features extracted from VF images to train ML algorithms for staging glaucoma from early to advanced/severe stages. Finally, the selected significant features were used to design and develop a comprehensive AI model to detect and grade glaucoma stages based on the data quantity and availability. In the first stage, a DL model was trained with TSNIT OCT scans, and its output was combined with significant structural and functional features and trained in ML models. The best-performed ML model achieved an area under the curve (AUC): 0.98, an accuracy of 97.2%, a sensitivity of 97.9%, and a specificity of 96.4% for detecting glaucoma. The model achieved an overall accuracy of 90.7% and an F1 score of 84.0% for classifying normal, early, moderate, and advanced-stage glaucoma. In conclusion, this thesis developed and proposed a comprehensive, evidence-based AI model that will solve the screening problem for large populations and relieve experts from manually analysing a slew of patient data and associated misinterpretation problems. Moreover, this thesis demonstrated three structural OCT features that could be added as excellent diagnostic markers for precise glaucoma diagnosis

Towards non-vascular fundus image analysis and disease detection

Assessment of retinal fundus image is very informative and preventive in early ocular disease detection. This non-invasive assessment of fundus images also helps in the early diagnosis of vascular diseases. This unique combination help in the early diagnosis of diseases. Applying image enhancement techniques with advanced Deep learning techniques helps to overcome such a challenging problem. Most Deep learning models give a diagnosis without attention to underlying pathological abnormalities. In this thesis, we tried to solve the problem in the same way as ophthalmologists and experts in the field approach the problem. We created models that can detect an Optic disc, Optic cup, and vascular regions in the image. This work can be integrated into any ocular disease detection, such as glaucoma, and vascular disease detection, such as diabetes. Extensive work is applied for better sampling when all models were suffering from a lack of data in the medical imaging field. The entire work on the retinal fundus image was in 2d images. In the extension of this work, we applied our knowledge to 3d MRI-Brain images. We attempt to predict attention scores in children, which is a big factor in the detection of kids with ADHD. But both work on fundus images and brain MRI images are under the umbrella of medical imaging. We believe this advancement in this line of research can be very valuable for future researchers in the area of automated medical imaging, especially in automated retinal disease diagnosis

딥러닝을 이용한 녹내장 진단 보조 시스템

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 협동과정 바이오엔지니어링전공, 2021. 2. 김희찬.본 논문에서는 딥 러닝 기반의 진단 보조 시스템을 제안하였다. 새로운 방법이 녹내장 데이터에 적용되었고 결과를 평가하였다. 첫번째 연구에서는 스펙트럼영역 빛간섭단층촬영기(SD-OCT)를 딥 러닝 분류 기를 이용해 분석하였다. 스펙트럼영역 빛간섭단층촬영기는 녹내장으로 인한 구조적 손상을 평가하기 위해 사용하는 장비이다. 분류 알고리즘은 합성 곱 신경망을 이용해 개발 되었으며, 스펙트럼영역 빛간섭단층촬영기의 망막신경섬유층(RNFL)과 황반부 신경절세포내망상층 (GCIPL) 사진을 이용해 학습했다. 제안한 방법은 두개의 이미지를 입력으로 받는 이중입력합성곱신경망(DICNN)이며, 딥 러닝 분류에서 효과적인 것으로 알려져 있다. 이중입력합성곱신경망은 망막신경섬유층 과 신경절세포층 의 두께 지도를 이용하여 학습 됐으며, 학습된 네트워크는 녹내장과 정상 군을 구분한다. 이중입력합성곱신경망은 정확도와 수신기동작특성곡선하면적 (AUC)으로 평가 되었다. 망막신경섬유층과 신경절세포층 두께 지도로 학습된 설계한 딥 러닝 모델을 조기 녹내장과 정상 군을 분류하는 성능을 평가하고 비교하였다. 성능평가 결과 이중입력합성곱신경망은 조기 녹내장을 분류하는데 0.869의 수신기동작특성곡선의넓이와 0.921의 민감도, 0.756의 특이도를 보였다. 두번째 연구에서는 딥 러닝을 이용해 시신경유두사진의 해상도와 대비, 색감, 밝기를 보정하는 방법을 제안하였다. 시신경유두사진은 녹내장을 진단하는데 있어 효과적인 것으로 알려져 있다. 하지만, 녹내장의 진단에서 환자의 나, 작은 동공, 매체 불투명성 등으로 인해 평가가 어려운 경우가 있다. 초 해상도와 보정 알고리즘은 초 해상도 적대적생성신경망을 통해 개발되었다. 원본 고해상도의 시신경 유두 사진은 저해상도 사진으로 축소되고, 보정된 고해상도 시신경유두사진으로 보정 되며, 보정된 사진은 시신경여백의 가시성과 근처 혈관을 잘 보이도록 후처리 알고리즘을 이용한다. 저해상도이미지를 보정된 고해상도이미지로 복원하는 과정을 초해상도적대적신경망을 통해 학습한다. 설계한 네트워크는 신호 대 잡음 비(PSNR)과 구조적유사성(SSIM), 평균평가점(MOS)를 이용해 평가 되었다. 현재의 연구는 딥 러닝이 안과 이미지를 4배 해상도와 구조적인 세부 항목이 잘 보이도록 개선할 수 있다는 것을 보여주었다. 향상된 시신경유두 사진은 시신경의 병리학적인 특성의 진단 정확도를 명확히 향상시킨다. 성능평가결과 평균 PSNR은 25.01 SSIM은 0.75 MOS는 4.33으로 나타났다. 세번째 연구에서는 환자 정보와 안과 영상(시신경유두 사진과 붉은색이 없는 망막신경섬유층 사진)을 이용해 녹내장 의심 환자를 분별하고 녹내장 의심 환자의 발병 연수를 예측하는 딥 러닝 모델을 개발하였다. 임상 데이터들은 녹내장을 진단하거나 예측하는데 유용한 정보들을 가지고 있다. 하지만, 어떻게 다양한 유형의 임상정보들을 조합하는 것이 각각의 환자들에 대해 잠재적인 녹내장을 예측하는데 어떤 영향을 주는지에 대한 연구가 진행 된 적이 없다. 녹내장 의 심자 분류와 발병 년 수 예측은 합성 곱 자동 인코더(CAE)를 비 지도적 특성 추출 기로 사용하고, 기계학습 분류 기와 회귀기를 통해 진행하였다. 설계한 모델은 정확도와 평균제곱오차(MSE)를 통해 평가 되었으며, 이미지 특징과 환자 특징은 조합했을 때 녹내장 의심 환자 분류와 발병 년 수 예측의 성능이 이미지 특징과 환자 특징을 각각 썼을 때보다 성능이 좋았다. 정답과의 MSE는 2.613으로 나타났다. 본 연구에서는 딥 러닝을 이용해 녹내장 관련 임상 데이터 중 망막신경섬유층, 신경절세포층 사진을 녹내장 진단에 이용되었고, 시신경유두 사진은 시신경의 병리학적인 진단 정확도를 높였고, 환자 정보는 보다 정확한 녹내장 의심 환자 분류와 발병 년 수 예측에 이용되었다. 향상된 녹내장 진단 성능은 기술적이고 임상적인 지표들을 통해 검증되었다.This paper presents deep learning-based methods for improving glaucoma diagnosis support systems. Novel methods were applied to glaucoma clinical cases and the results were evaluated. In the first study, a deep learning classifier for glaucoma diagnosis based on spectral-domain optical coherence tomography (SD-OCT) images was proposed and evaluated. Spectral-domain optical coherence tomography (SD-OCT) is commonly employed as an imaging modality for the evaluation of glaucomatous structural damage. The classification model was developed using convolutional neural network (CNN) as a base, and was trained with SD-OCT retinal nerve fiber layer (RNFL) and macular ganglion cell-inner plexiform layer (GCIPL) images. The proposed network architecture, termed Dual-Input Convolutional Neural Network (DICNN), showed great potential as an effective classification algorithm based on two input images. DICNN was trained with both RNFL and GCIPL thickness maps that enabled it to discriminate between normal and glaucomatous eyes. The performance of the proposed DICNN was evaluated with accuracy and area under the receiver operating characteristic curve (AUC), and was compared to other methods using these metrics. Compared to other methods, the proposed DICNN model demonstrated high diagnostic ability for the discrimination of early-stage glaucoma patients in normal subjects. AUC, sensitivity and specificity was 0.869, 0.921, 0.756 respectively. In the second study, a deep-learning method for increasing the resolution and improving the legibility of Optic-disc Photography(ODP) was proposed. ODP has been proven to be useful for optic nerve evaluation in glaucoma. But in clinical practice, limited patient cooperation, small pupil or media opacities can limit the performance of ODP. A model to enhance the resolution of ODP images, termed super-resolution, was developed using Super Resolution Generative Adversarial Network(SR-GAN). To train this model, high-resolution original ODP images were transformed into two counterparts: (1) down-scaled low-resolution ODPs, and (2) compensated high-resolution ODPs with enhanced visibility of the optic disc margin and surrounding retinal vessels which were produced using a customized image post-processing algorithm. The SR-GAN was trained to learn and recognize the differences between these two counterparts. The performance of the network was evaluated using Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), and Mean Opinion Score (MOS). The proposed study demonstrated that deep learning can be applied to create a generative model that is capable of producing enhanced ophthalmic images with 4x resolution and with improved structural details. The proposed method can be used to enhance ODPs and thereby significantly increase the detection accuracy of optic disc pathology. The average PSNR, SSIM and MOS was 25.01, 0.75, 4.33 respectively In the third study, a deep-learning model was used to classify suspected glaucoma and to predict subsequent glaucoma onset-year in glaucoma suspects using clinical data and retinal images (ODP & Red-free Fundus RNFL Photo). Clinical data contains useful information about glaucoma diagnosis and prediction. However, no study has been undertaken to investigate how combining different types of clinical information would be helpful for predicting the subsequent course of glaucoma in an individual patient. For this study, image features extracted using Convolutional Auto Encoder (CAE) along with clinical features were used for glaucoma suspect classification and onset-year prediction. The performance of the proposed model was evaluated using accuracy and Mean Squared Error (MSE). Combing the CAE extracted image features and clinical features improved glaucoma suspect classification and on-set year prediction performance as compared to using the image features and patient features separately. The average MSE between onset-year and predicted onset year was 2.613 In this study, deep learning methodology was applied to clinical images related to glaucoma. DICNN with RNFL and GCIPL images were used for classification of glaucoma, SR-GAN with ODP images were used to increase detection accuracy of optic disc pathology, and CAE & machine learning algorithm with clinical data and retinal images was used for glaucoma suspect classification and onset-year predication. The improved glaucoma diagnosis performance was validated using both technical and clinical parameters. The proposed methods as a whole can significantly improve outcomes of glaucoma patients by early detection, prediction and enhancing detection accuracy.Contents Abstract i Contents iv List of Tables vii List of Figures viii Chapter 1 General Introduction 1 1.1 Glaucoma 1 1.2 Deep Learning for Glaucoma Diagnosis 3 1.4 Thesis Objectives 3 Chapter 2 Dual-Input Convolutional Neural Network for Glaucoma Diagnosis using Spectral-Domain Optical Coherence Tomography 6 2.1 Introduction 6 2.1.1 Background 6 2.1.2 Related Work 7 2.2 Methods 8 2.2.1 Study Design 8 2.2.2 Dataset 9 2.2.3 Dual-Input Convolutional Neural Network (DICNN) 15 2.2.4 Training Environment 18 2.2.5 Statistical Analysis 19 2.3 Results 20 2.3.1 DICNN Performance 20 2.3.1 Grad-CAM for DICNN 34 2.4 Discussion 37 2.4.1 Research Significance 37 2.4.2 Limitations 40 2.5 Conclusion 42 Chapter 3 Deep-learning-based enhanced optic-disc photography 43 3.1 Introduction 43 3.1.1 Background 43 3.1.2 Needs 44 3.1.3 Related Work 45 3.2 Methods 46 3.2.1 Study Design 46 3.2.2 Dataset 46 3.2.2.1 Details on Customized Image Post-Processing Algorithm 47 3.2.3 SR-GAN Network 50 3.2.3.1 Design of Generative Adversarial Network 50 3.2.3.2 Loss Functions 55 3.2.4 Assessment of Clinical Implications of Enhanced ODPs 58 3.2.5 Statistical Analysis 60 3.2.6 Hardware Specifications & Software Specifications 60 3.3 Results 62 3.3.1 Training Loss of Modified SR-GAN 62 3.3.2 Performance of Final Network 66 3.3.3 Clinical Validation of Enhanced ODP by MOS comparison 77 3.3.4 Comparison of DH-Detection Accuracy 79 3.4 Discussion 80 3.4.1 Research Significance 80 3.4.2 Limitations 85 3.5 Conclusion 88 Chapter 4 Deep Learning Based Prediction of Glaucoma Onset Using Retinal Image and Patient Data 89 4.1 Introduction 89 4.1.1 Background 89 4.1.2 Related Work 90 4.2 Methods 90 4.2.1 Study Design 90 4.2.2 Dataset 91 4.2.3 Design of Overall System 94 4.2.4 Design of Convolutional Auto Encoder 95 4.2.5 Glaucoma Suspect Classification 97 4.2.6 Glaucoma Onset-Year Prediction 97 4.3 Result 99 4.3.1 Performance of Designed CAE 99 4.3.2 Performance of Designed Glaucoma Suspect Classification 101 4.3.3 Performance of Designed Glaucoma Onset-Year Prediction 105 4.4 Discussion 110 4.4.1 Research Significance 110 4.4.2 Limitations 110 4.5 Conclusion 111 Chapter 5 Summary and Future Works 112 5.1 Thesis Summary 112 5.2 Limitations and Future Works 113 Bibliography 115 Abstract in Korean 127 Acknowledgement 130Docto

Efficient Pyramid Channel Attention Network for Pathological Myopia Detection

Pathological myopia (PM) is the leading ocular disease for impaired vision and blindness worldwide. The key to detecting PM as early as possible is to detect informative features in global and local lesion regions, such as fundus tessellation, atrophy and maculopathy. However, applying classical convolutional neural networks (CNNs) to efficiently highlight global and local lesion context information in feature maps is quite challenging. To tackle this issue, we aim to fully leverage the potential of global and local lesion information with attention module design. Based on this, we propose an efficient pyramid channel attention (EPCA) module, which dynamically explores the relative importance of global and local lesion context information in feature maps. Then we combine the EPCA module with the backbone network to construct EPCA-Net for automatic PM detection based on fundus images. In addition, we construct a PM dataset termed PM-fundus by collecting fundus images of PM from publicly available datasets (e.g., the PALM dataset and ODIR dataset). The comprehensive experiments are conducted on three datasets, demonstrating that our EPCA-Net outperforms state-of-the-art methods in detecting PM. Furthermore, motivated by the recent pretraining-and-finetuning paradigm, we attempt to adapt pre-trained natural image models for PM detection by freezing them and treating the EPCA module and other attention modules as the adapters. The results show that our method with the pretraining-and-finetuning paradigm achieves competitive performance through comparisons to part of methods with traditional fine-tuning methods with fewer tunable parameters.Comment: 12 page