De Neve, Wesley

Janssens, Olivier

Kim, Mi Jung

Park, H.

Van Hoecke, Sofie

Zuallaert, Jasper

Ghent University Academic Bibliography

Web Applicable Computer-aided Diagnosis ofGlaucoma Using Deep LearningMijung Kim*† Olivier Janssens* Ho-min Park*†Jasper Zuallaert*† Sofie Van Hoecke* Wesley De Neve*††Center for Biotech Data Science, Ghent University Global Campus*IDLab, Department of Electronics and Information Systems, Ghent Universitymijung.kim@ugent.beAbstractGlaucoma is a major eye disease, leading to vision loss in the absence of propermedical treatment. Current diagnosis of glaucoma is performed by ophthalmol-ogists who are often analyzing several types of medical images generated bydifferent types of medical equipment. Capturing and analyzing these medicalimages is labor-intensive and expensive. In this paper, we present a novel computa-tional approach towards glaucoma diagnosis and localization, only making use ofeye fundus images that are analyzed by state-of-the-art deep learning techniques.Specifically, our approach leverages Convolutional Neural Networks (CNNs) andGradient-weighted Class Activation Mapping (Grad-CAM) for glaucoma diagnosisand localization, respectively. Quantitative and qualitative results, as obtainedfor a small-sized dataset with no segmentation ground truth, demonstrate that theproposed approach is promising, for instance achieving an accuracy of 0.91±0.02and an ROC-AUC score of 0.94 for the diagnosis task. Furthermore, we present apublicly available prototype web application that integrates our predictive model,with the goal of making effective glaucoma diagnosis available to a wide audience.1 IntroductionThe revival of convolutional neural networks (CNNs) and the public availability of large-scaledatasets like ImageNet [1] have led to significant performance improvements in computer vision.Unlike its predecessors, ImageNet contains more than one million natural images belonging to morethan thousand classes, contributing substantially to the successful deployment of CNNs. Indeed,ImageNet for instance allows constructing CNN-based predictive models that are more accurate andthat better generalize for the tasks of image segmentation and classification. Moreover, CNN-basedpredictive models have shown to be highly successful in medical image analysis [2–4]. For diabeticretinopathy [4], these models even managed to obtain a diagnosis accuracy that is as high as thediagnosis accuracy of human experts. However, most deep learning-based approaches towardsmedical image analysis face two major challenges. First, few medical image datasets are availablewith a volume sufficient to train a deep neural network from scratch, especially given that, byconvention, the deeper the neural networks constructed, the better the predictions (i.e., the higherthe classification accuracy). As a result, if not enough data are available, then the neural networksconstructed may suffer from overfitting. Second, apart from labels for the purpose of classification,most medical image datasets do not have proper annotations available for the purpose of segmentation,given the time-consuming and error-prone nature of annotation efforts.In our research, to tackle the two aforementioned challenges, we built a predictive model for computer-aided diagnosis (CAD) and localization of glaucoma, leveraging eye fundus images that are widelyMachine Learning for Health (ML4H) Workshop at NeurIPS 2018.arXiv:1812.02405v1  [cs.CV]  6 Dec 2018used in present-day hospitals, given that these images can be acquired at a relatively low cost.Additionally, based on our CAD model, we developed a novel tool for glaucoma diagnosis thattakes the form of a prototype web application1. The main contribution of this paper stems from thenovelty of our predictive model and its integration into a prototype web application. Indeed, althoughplenty of research efforts have been presented in the scientific literature on computer-aided diagnosisusing fundus images, to the best of our knowledge, no previous research efforts have simultaneouslytargeted both diagnosis and localization using a dataset with no segmentation ground truth.The rest of this paper is organized as follows. We explain our approach in Section 2. We subsequentlydiscuss our experimental setup and results in Section 3. Finally, we conclude our research in Section 4.Note that some of the research activities described in this paper have also been presented at TheSecond International Workshop on Deep Learning in Bioinformatics, Biomedicine, and HealthcareInformatics (DLB2H 2018).2 ArchitectureConvolutional Neural Network. Introduced by Simonyan and Zisserman at Oxford’s Visual Geom-etry Group (VGG), VGG networks [5] are currently some of the most widely used CNN architectures.Among these different architectures, VGG16 and VGG19 are the most popular ones, thanks totheir high classification accuracy. In our research, we decided to work with VGG16 over VGG19,given that the difference in effectiveness between the two is not significant, while VGG16 has lessparameters to learn than VGG19. However, as mentioned in the introductory section, VGG16 maysuffer from overfitting when directly applying this architecture to our eye fundus images. As such,to mitigate overfitting and to increase training effectiveness and efficiency, we applied a number ofchanges. To facilitate better model regularization, we introduced two techniques. First, we addeddropout layers (ratio: 0.5) at the end of each convolutional block. Second, next to the use of defaultdata augmentation techniques (that is, random cropping and random horizontal flipping), we alsoincorporated data augmentation techniques such as random brightness (using sigma ±0.8). Fur-thermore, we adopted the adaptive moment estimation (ADAM) optimizer [6] instead of StochasticGradient Descent (SGD), allowing us to handle multiple drawbacks of SGD more efficiently, like aslow convergence and a high variance in parameter updates. Lastly, we used a pre-trained VGG16model, so to be able to avoid a time-consuming weight initialization process and to considerablyreduce the training time [7]. From an architectural point-of-view, we also applied a few more changesto the vanilla VGG16 network. First, we used fully convolutional layers instead of fully connectedlayers. Second, in order to meet our classification needs, the number of channels in the last fullyconvolutional layer was reduced from 1000 to 2.Grad-CAM. Most sets of medical images do not have a ground truth (that is, golden standardannotations) for supervised segmentation or localization. As a result, localization needs to be treatedas an unsupervised learning problem. A couple of deep learning-based approaches are frequentlyused to segment images in an unsupervised fashion. One such approach is Class Activation Mapping(CAM), as proposed by [8]. CAM allows visualizing what CNNs look at when they classify aninput image, thus making it possible to visually understand why a CNN chooses one class overanother. More recently, another algorithm was published in [9], called Gradient-weighted ClassActivation Mapping (Grad-CAM), improving upon CAM. In particular, Grad-CAM uses the class-specific gradient information flowing into the final convolutional layer of a CNN to produce a coarselocalization map of the important regions in a given input image. In our research, this localizationprocess is activated only if the class is predicted as glaucomatous.3 ExperimentsDataset: fundus images. The total number of eye fundus images in our training dataset is 1,080,among which 220 images were used for validation purposes. Another 110 eye fundus images unseenduring training and validation were used for testing purposes. These images, with the horizontalresolution varying from 1,172 to 2,500 pixels and with the vertical resolution varying from 1,500to 3,200 pixels, have been collected at Samsung Medical Center in 2016. All images were labeledby an experienced ophthalmologist. The proportion of normal and glaucomatous images for both1www.medinoid.org2training and testing was 1:1 (that is, the dataset used was perfectly balanced). This study followed allguidelines for experimental investigation in human subjects, was approved by the Samsung MedicalCenter Institutional Review Board, and adhered to the tenets of the Declaration of Helsinki.Implementation. Our approach towards glaucoma diagnosis and localization was implemented inPython 2.7, using Tensorflow 1.7 and the , pre-trained VGG16 model over ImageNet from TensorflowSlim2. Furthermore, our approach was executed using two Intel(R) Xeon(R) E5-2620 2.4GHz CPUsand an NVIDIA GeForce GTX TITAN X GPU. Before feeding the input images to the network,they were center-cropped to remove excessive information not related to the classification task athand (e.g., removal of a black background) and resized to 224 × 224 × 3. In addition, for imagenormalization purposes, we changed the RGB means for our dataset as follows: 105.51, 54.52, and16.19 (in RGB order). After changing the last layer of the pre-trained model into fully convolutionallayer, the weights of all pre-trained layers were fine-tuned over our dataset. During fine-tuning, weused a batch size of 32, a learning rate of 0.0001, and data augmentation. We made use of 5-foldcross-validation for our experiments. The metrics used for evaluating the classification task are (1)accuracy, (2) recall, (3) precision, (4) F1 score, and (5) Receiver Operating Characteristic - AreaUnder the Curve (ROC-AUC).Experimental results. Considering its effectiveness, early stopping is one of the most commonlyused regularization methods in practice [10]. Thus, as shown in Figure 1, we early stopped thetraining at around epoch 80, coming with a validation loss that is the lowest and that is about to riseagain, even though the training loss is still decreasing over time. Apart from an accuracy of 0.91 andan ROC-AUC score of 0.94, our model achieved a recall of 0.91, a precision of 0.92, and an F1 scoreof 0.91. As mentioned in Section 1, no method is currently available that simultaneously tacklesclassification and localization. Moreover, a direct comparison with already existing approaches isdifficult, given a lack of publicly available datasets. In Table 1, some information can be foundabout the difference in effectiveness of our approach, as obtained for the dataset made available bySamsung Medical Center, and the effectiveness of the deep learning-based approach presented in [11],as obtained for the RIM-ONE dataset [12], for the classification task only.(a) Model error (b) Model accuracy (c) ROC-AUC curveFigure 1: Model evaluation.Table 1: Comparison with another deep learning model, using different datasets.Model Dataset # positives # negatives Input size Accuracy AUC[11] RIM-ONE [12] 200 255 227× 227× 3 0.88 -Proposed SMC 540 540 224× 224× 3 0.91 0.94Apart from the use of two different datasets, thus making a direct comparison impossible, wehypothesize that some of the differences in effectiveness can be attributed to the CNN architectureused and the way data augmentation has been used. In particular, [11] employs AlexNet, which wasreported to be less effective for the ILSVRC classification task than VGG16 [5]. Furthermore, theapproach of [11] did not apply any data augmentation technique. As a result, given the limited size ofthe dataset used, we believe chances are high that the model of [11] did not sufficiently generalize.Regarding localization, since this paper presents the first attempt to localize glaucoma without asegmentation ground truth, there are no proper baselines to compare against. Therefore, we asked2https://github.com/tensorflow/models/tree/master/research/slim3(a) Correctly localized suspicious areas. (b) Wrongly localized suspicious areas.Figure 2: Localization results.for the opinion of a clinician at Samsung Medical Center. That way, we found that most of theglaucomatous images were correctly localized, highlighting the corresponding suspicious area, asshown in Figure 2. However, we found that localization was also wrong for a few test examples (e.g.,due to blur), thus pointing to the need for a more rigorous computational and clinical analysis. Lastly,as illustrated by Figure 3, we built a publicly available prototype web application that integrates ourpredictive model, with the goal of making glaucoma diagnosis available to a wide audience.Figure 3: Prototype web application for glaucoma diagnosis. Users can provide this applicationdirectly with an input image using a drag and drop approach, as shown by the figure in the middle.Furthermore, the prototype web application shows the probability of having glaucoma, highlightingthe suspicious region found.4 Conclusions and Future WorkIn this paper, we introduced a predictive model for computer-aided diagnosis of glaucoma, leveragingCNNs and Grad-CAM for diagnosis and localization, respectively. Our model has been developedby making use of small-sized dataset of fundus eye images, only coming with annotations for thediagnosis task, not having annotations for the localization task. Furthermore, we integrated ourmodel into a publicly available prototype web application, with the ultimate goal of making effectiveglaucoma diagnosis available to a wide audience in a user-friendly way.AcknowledgmentsThe research and development efforts described in this paper were funded by Ghent University, GhentUniversity Global Campus, Flanders Innovation & Entrepreneurship (VLAIO), the Fund for ScientificResearch-Flanders (FWO-Flanders), and the European Union. These efforts were also supported bythe Basic Science Research Program of the National Research Foundation of Korea (NRF), fundedby the Ministry of Education (grant: 2017034834).4References[1] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scalehierarchical image database,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009.IEEE Conference on, pp. 248–255, IEEE, 2009.[2] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical imagesegmentation,” in International Conference on Medical image computing and computer-assistedintervention, pp. 234–241, Springer, 2015.[3] L. Heinrich, J. A. Bogovic, and S. Saalfeld, “Deep learning for isotropic super-resolutionfrom non-isotropic 3D electron microscopy,” in International Conference on Medical ImageComputing and Computer-Assisted Intervention, pp. 135–143, Springer, 2017.[4] V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan,K. Widner, T. Madams, J. Cuadros, et al., “Development and validation of a deep learningalgorithm for detection of diabetic retinopathy in retinal fundus photographs,” Jama, vol. 316,no. 22, pp. 2402–2410, 2016.[5] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale imagerecognition,” arXiv preprint arXiv:1409.1556, 2014.[6] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprintarXiv:1412.6980, 2014.[7] L. Torrey and J. Shavlik, “Transfer learning,” in Handbook of Research on Machine LearningApplications and Trends: Algorithms, Methods, and Techniques, pp. 242–264, IGI Global, 2010.[8] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features fordiscriminative localization,” in Computer Vision and Pattern Recognition (CVPR), 2016 IEEEConference on, pp. 2921–2929, IEEE, 2016.[9] R. R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra, “Grad-CAM:Why did you say that?,” arXiv preprint arXiv:1611.07450, 2016.[10] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.[11] B. Al-Bander, W. Al-Nuaimy, M. A. Al-Taee, and Y. Zheng, “Automated glaucoma diagnosisusing deep learning approach,” in Systems, Signals & Devices (SSD), 2017 14th InternationalMulti-Conference on, pp. 207–210, IEEE, 2017.[12] F. Fumero, S. Alayón, J. L. Sanchez, J. Sigut, and M. Gonzalez-Hernandez, “RIM-ONE: Anopen retinal image database for optic nerve evaluation,” in 2011 24th international symposiumon computer-based medical systems (CBMS), pp. 1–6, IEEE, 2011.5

Web applicable computer-aided diagnosis of glaucoma using deep learning

https://biblio.ugent.be/publication/8606439/file/8606440.pdf

Web applicable computer-aided diagnosis of glaucoma using deep learning

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography