딥러닝을 이용한 녹내장 진단 보조 시스템

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 협동과정 바이오엔지니어링전공, 2021. 2. 김희찬.본 논문에서는 딥 러닝 기반의 진단 보조 시스템을 제안하였다. 새로운 방법이 녹내장 데이터에 적용되었고 결과를 평가하였다. 첫번째 연구에서는 스펙트럼영역 빛간섭단층촬영기(SD-OCT)를 딥 러닝 분류 기를 이용해 분석하였다. 스펙트럼영역 빛간섭단층촬영기는 녹내장으로 인한 구조적 손상을 평가하기 위해 사용하는 장비이다. 분류 알고리즘은 합성 곱 신경망을 이용해 개발 되었으며, 스펙트럼영역 빛간섭단층촬영기의 망막신경섬유층(RNFL)과 황반부 신경절세포내망상층 (GCIPL) 사진을 이용해 학습했다. 제안한 방법은 두개의 이미지를 입력으로 받는 이중입력합성곱신경망(DICNN)이며, 딥 러닝 분류에서 효과적인 것으로 알려져 있다. 이중입력합성곱신경망은 망막신경섬유층 과 신경절세포층 의 두께 지도를 이용하여 학습 됐으며, 학습된 네트워크는 녹내장과 정상 군을 구분한다. 이중입력합성곱신경망은 정확도와 수신기동작특성곡선하면적 (AUC)으로 평가 되었다. 망막신경섬유층과 신경절세포층 두께 지도로 학습된 설계한 딥 러닝 모델을 조기 녹내장과 정상 군을 분류하는 성능을 평가하고 비교하였다. 성능평가 결과 이중입력합성곱신경망은 조기 녹내장을 분류하는데 0.869의 수신기동작특성곡선의넓이와 0.921의 민감도, 0.756의 특이도를 보였다. 두번째 연구에서는 딥 러닝을 이용해 시신경유두사진의 해상도와 대비, 색감, 밝기를 보정하는 방법을 제안하였다. 시신경유두사진은 녹내장을 진단하는데 있어 효과적인 것으로 알려져 있다. 하지만, 녹내장의 진단에서 환자의 나, 작은 동공, 매체 불투명성 등으로 인해 평가가 어려운 경우가 있다. 초 해상도와 보정 알고리즘은 초 해상도 적대적생성신경망을 통해 개발되었다. 원본 고해상도의 시신경 유두 사진은 저해상도 사진으로 축소되고, 보정된 고해상도 시신경유두사진으로 보정 되며, 보정된 사진은 시신경여백의 가시성과 근처 혈관을 잘 보이도록 후처리 알고리즘을 이용한다. 저해상도이미지를 보정된 고해상도이미지로 복원하는 과정을 초해상도적대적신경망을 통해 학습한다. 설계한 네트워크는 신호 대 잡음 비(PSNR)과 구조적유사성(SSIM), 평균평가점(MOS)를 이용해 평가 되었다. 현재의 연구는 딥 러닝이 안과 이미지를 4배 해상도와 구조적인 세부 항목이 잘 보이도록 개선할 수 있다는 것을 보여주었다. 향상된 시신경유두 사진은 시신경의 병리학적인 특성의 진단 정확도를 명확히 향상시킨다. 성능평가결과 평균 PSNR은 25.01 SSIM은 0.75 MOS는 4.33으로 나타났다. 세번째 연구에서는 환자 정보와 안과 영상(시신경유두 사진과 붉은색이 없는 망막신경섬유층 사진)을 이용해 녹내장 의심 환자를 분별하고 녹내장 의심 환자의 발병 연수를 예측하는 딥 러닝 모델을 개발하였다. 임상 데이터들은 녹내장을 진단하거나 예측하는데 유용한 정보들을 가지고 있다. 하지만, 어떻게 다양한 유형의 임상정보들을 조합하는 것이 각각의 환자들에 대해 잠재적인 녹내장을 예측하는데 어떤 영향을 주는지에 대한 연구가 진행 된 적이 없다. 녹내장 의 심자 분류와 발병 년 수 예측은 합성 곱 자동 인코더(CAE)를 비 지도적 특성 추출 기로 사용하고, 기계학습 분류 기와 회귀기를 통해 진행하였다. 설계한 모델은 정확도와 평균제곱오차(MSE)를 통해 평가 되었으며, 이미지 특징과 환자 특징은 조합했을 때 녹내장 의심 환자 분류와 발병 년 수 예측의 성능이 이미지 특징과 환자 특징을 각각 썼을 때보다 성능이 좋았다. 정답과의 MSE는 2.613으로 나타났다. 본 연구에서는 딥 러닝을 이용해 녹내장 관련 임상 데이터 중 망막신경섬유층, 신경절세포층 사진을 녹내장 진단에 이용되었고, 시신경유두 사진은 시신경의 병리학적인 진단 정확도를 높였고, 환자 정보는 보다 정확한 녹내장 의심 환자 분류와 발병 년 수 예측에 이용되었다. 향상된 녹내장 진단 성능은 기술적이고 임상적인 지표들을 통해 검증되었다.This paper presents deep learning-based methods for improving glaucoma diagnosis support systems. Novel methods were applied to glaucoma clinical cases and the results were evaluated. In the first study, a deep learning classifier for glaucoma diagnosis based on spectral-domain optical coherence tomography (SD-OCT) images was proposed and evaluated. Spectral-domain optical coherence tomography (SD-OCT) is commonly employed as an imaging modality for the evaluation of glaucomatous structural damage. The classification model was developed using convolutional neural network (CNN) as a base, and was trained with SD-OCT retinal nerve fiber layer (RNFL) and macular ganglion cell-inner plexiform layer (GCIPL) images. The proposed network architecture, termed Dual-Input Convolutional Neural Network (DICNN), showed great potential as an effective classification algorithm based on two input images. DICNN was trained with both RNFL and GCIPL thickness maps that enabled it to discriminate between normal and glaucomatous eyes. The performance of the proposed DICNN was evaluated with accuracy and area under the receiver operating characteristic curve (AUC), and was compared to other methods using these metrics. Compared to other methods, the proposed DICNN model demonstrated high diagnostic ability for the discrimination of early-stage glaucoma patients in normal subjects. AUC, sensitivity and specificity was 0.869, 0.921, 0.756 respectively. In the second study, a deep-learning method for increasing the resolution and improving the legibility of Optic-disc Photography(ODP) was proposed. ODP has been proven to be useful for optic nerve evaluation in glaucoma. But in clinical practice, limited patient cooperation, small pupil or media opacities can limit the performance of ODP. A model to enhance the resolution of ODP images, termed super-resolution, was developed using Super Resolution Generative Adversarial Network(SR-GAN). To train this model, high-resolution original ODP images were transformed into two counterparts: (1) down-scaled low-resolution ODPs, and (2) compensated high-resolution ODPs with enhanced visibility of the optic disc margin and surrounding retinal vessels which were produced using a customized image post-processing algorithm. The SR-GAN was trained to learn and recognize the differences between these two counterparts. The performance of the network was evaluated using Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), and Mean Opinion Score (MOS). The proposed study demonstrated that deep learning can be applied to create a generative model that is capable of producing enhanced ophthalmic images with 4x resolution and with improved structural details. The proposed method can be used to enhance ODPs and thereby significantly increase the detection accuracy of optic disc pathology. The average PSNR, SSIM and MOS was 25.01, 0.75, 4.33 respectively In the third study, a deep-learning model was used to classify suspected glaucoma and to predict subsequent glaucoma onset-year in glaucoma suspects using clinical data and retinal images (ODP & Red-free Fundus RNFL Photo). Clinical data contains useful information about glaucoma diagnosis and prediction. However, no study has been undertaken to investigate how combining different types of clinical information would be helpful for predicting the subsequent course of glaucoma in an individual patient. For this study, image features extracted using Convolutional Auto Encoder (CAE) along with clinical features were used for glaucoma suspect classification and onset-year prediction. The performance of the proposed model was evaluated using accuracy and Mean Squared Error (MSE). Combing the CAE extracted image features and clinical features improved glaucoma suspect classification and on-set year prediction performance as compared to using the image features and patient features separately. The average MSE between onset-year and predicted onset year was 2.613 In this study, deep learning methodology was applied to clinical images related to glaucoma. DICNN with RNFL and GCIPL images were used for classification of glaucoma, SR-GAN with ODP images were used to increase detection accuracy of optic disc pathology, and CAE & machine learning algorithm with clinical data and retinal images was used for glaucoma suspect classification and onset-year predication. The improved glaucoma diagnosis performance was validated using both technical and clinical parameters. The proposed methods as a whole can significantly improve outcomes of glaucoma patients by early detection, prediction and enhancing detection accuracy.Contents Abstract i Contents iv List of Tables vii List of Figures viii Chapter 1 General Introduction 1 1.1 Glaucoma 1 1.2 Deep Learning for Glaucoma Diagnosis 3 1.4 Thesis Objectives 3 Chapter 2 Dual-Input Convolutional Neural Network for Glaucoma Diagnosis using Spectral-Domain Optical Coherence Tomography 6 2.1 Introduction 6 2.1.1 Background 6 2.1.2 Related Work 7 2.2 Methods 8 2.2.1 Study Design 8 2.2.2 Dataset 9 2.2.3 Dual-Input Convolutional Neural Network (DICNN) 15 2.2.4 Training Environment 18 2.2.5 Statistical Analysis 19 2.3 Results 20 2.3.1 DICNN Performance 20 2.3.1 Grad-CAM for DICNN 34 2.4 Discussion 37 2.4.1 Research Significance 37 2.4.2 Limitations 40 2.5 Conclusion 42 Chapter 3 Deep-learning-based enhanced optic-disc photography 43 3.1 Introduction 43 3.1.1 Background 43 3.1.2 Needs 44 3.1.3 Related Work 45 3.2 Methods 46 3.2.1 Study Design 46 3.2.2 Dataset 46 3.2.2.1 Details on Customized Image Post-Processing Algorithm 47 3.2.3 SR-GAN Network 50 3.2.3.1 Design of Generative Adversarial Network 50 3.2.3.2 Loss Functions 55 3.2.4 Assessment of Clinical Implications of Enhanced ODPs 58 3.2.5 Statistical Analysis 60 3.2.6 Hardware Specifications & Software Specifications 60 3.3 Results 62 3.3.1 Training Loss of Modified SR-GAN 62 3.3.2 Performance of Final Network 66 3.3.3 Clinical Validation of Enhanced ODP by MOS comparison 77 3.3.4 Comparison of DH-Detection Accuracy 79 3.4 Discussion 80 3.4.1 Research Significance 80 3.4.2 Limitations 85 3.5 Conclusion 88 Chapter 4 Deep Learning Based Prediction of Glaucoma Onset Using Retinal Image and Patient Data 89 4.1 Introduction 89 4.1.1 Background 89 4.1.2 Related Work 90 4.2 Methods 90 4.2.1 Study Design 90 4.2.2 Dataset 91 4.2.3 Design of Overall System 94 4.2.4 Design of Convolutional Auto Encoder 95 4.2.5 Glaucoma Suspect Classification 97 4.2.6 Glaucoma Onset-Year Prediction 97 4.3 Result 99 4.3.1 Performance of Designed CAE 99 4.3.2 Performance of Designed Glaucoma Suspect Classification 101 4.3.3 Performance of Designed Glaucoma Onset-Year Prediction 105 4.4 Discussion 110 4.4.1 Research Significance 110 4.4.2 Limitations 110 4.5 Conclusion 111 Chapter 5 Summary and Future Works 112 5.1 Thesis Summary 112 5.2 Limitations and Future Works 113 Bibliography 115 Abstract in Korean 127 Acknowledgement 130Docto

Technological Advances in the Diagnosis and Management of Pigmented Fundus Tumours

Choroidal naevi are the most common intraocular tumour. They can be pigmented or non-pigmented and have a predilection for the posterior uvea. The majority remain undetected and cause no harm but are increasingly found on routine community optometry examinations. Rarely does a naevus demonstrate growth or the onset of suspicious features to fulfil the criteria for a malignant melanoma. Because of this very small risk, optometrists commonly refer these patients to hospital eye units for a second opinion, triggering specialist examination and investigation, causing significant anxiety to patients and stretching medical resources. This PhD thesis introduces the MOLES acronym and scoring system that has been devised to categorise the risk of malignancy in choroidal melanocytic tumours according to Mushroom tumour shape, Orange pigment, Large tumour size, Enlarging tumour and Subretinal fluid. This is a simplified system that can be used without sophisticated imaging, and hence its main utility lies in the screening of patients with choroidal pigmented lesions in the community and general ophthalmology clinics. Under this system, lesions were categorised by a scoring system as ‘common naevus’, ‘low-risk naevus’, ‘high-risk naevus’ and ‘probable melanoma.’ According to the sum total of the scores, the MOLES system correlates well with ocular oncologists’ final diagnosis. The PhD thesis also describes a model of managing such lesions in a virtual pathway, showing that images of choroidal naevi evaluated remotely using a decision-making algorithm by masked non-medical graders or masked ophthalmologists is safe. This work prospectively validates a virtual naevus clinic model focusing on patient safety as the primary consideration. The idea of a virtual naevus clinic as a fast, one-stop, streamlined and comprehensive service is attractive for patients and healthcare systems, including an optimised patient experience with reduced delays and inconvenience from repeated visits. A safe, standardised model ensures homogeneous management of cases, appropriate and prompt return of care closer to home to community-based optometrists. This research work and strategies, such as the MOLES scoring system for triage, could empower community-based providers to deliver management of benign choroidal naevi without referral to specialist units. Based on the positive outcome of this prospective study and the MOLES studies, a ‘Virtual Naevus Clinic’ has been designed and adapted at Moorfields Eye Hospital (MEH) to prove its feasibility as a response to the COVID-19 pandemic, and with the purpose of reducing in-hospital patient journey times and increasing the capacity of the naevus clinics, while providing safe and efficient clinical care for patients. This PhD chapter describes the design, pathways, and operating procedures for the digitally enabled naevus clinics in Moorfields Eye Hospital, including what this service provides and how it will be delivered and supported. The author will share the current experience and future plan. Finally, the PhD thesis will cover a chapter that discusses the potential role of artificial intelligence (AI) in differentiating benign choroidal naevus from choroidal melanoma. The published clinical and imaging risk factors for malignant transformation of choroidal naevus will be reviewed in the context of how AI applied to existing ophthalmic imaging systems might be able to determine features on medical images in an automated way. The thesis will include current knowledge to date and describe potential benefits, limitations and key issues that could arise with this technology in the ophthalmic field. Regulatory concerns will be addressed with possible solutions on how AI could be implemented in clinical practice and embedded into existing imaging technology with the potential to improve patient care and the diagnostic process. The PhD will also explore the feasibility of developed automated deep learning models and investigate the performance of these models in diagnosing choroidal naevomelanocytic lesions based on medical imaging, including colour fundus and autofluorescence fundus photographs. This research aimed to determine the sensitivity and specificity of an automated deep learning algorithm used for binary classification to differentiate choroidal melanomas from choroidal naevi and prove that a differentiation concept utilising a machine learning algorithm is feasible