Antal, Bálint

Csutak, Adrienne

Hajdu, András

Harangi, Balázs

Kovács, György

Kovács, László

Lázár, István

Nagy, Brigitta

Pető, Tünde

Szakács, József

Tavares, Mayo Kayann Guerra Silva

Tóth, János

EKE Repository of Publications

Data analysis applied to diabeticretinopathy screening: performanceevaluationBálint Antala∗†, Mayo Kayann Guerra Silva Tavaresb†,László Kovácsa†, Balázs Harangia, István Lázára,Brigitta Nagya, György Kovácsa, József Szakácsa,János Tótha, Tünde Petőc, Adrienne Csutakb‡,András Hajdua∗‡aFaculty of Informatics, University of Debrecen, HungarybDepartment of Ophthalmology, Faculty of Medicine, University of Debrecen, HungarycCentre for Public Health, Queen’s University Belfast, United KingdomSubmitted June 6, 2018 — Accepted October 17, 2018AbstractThe number of people with diabetes mellitus (DM) has risen from 108million in 1980 to 422 million in 2014. Diabetic retinopathy (DR) is oneof the most common causes of blindness in the developed world. A pooledanalysis of data from between 1980-2008 estimates that 93 million peoplearound the world have DR. In this paper, we present a computer-aided auto-mated image analysis system capable of handling images generated in real-lifescreening program. In this study, we analyzed 2932 color fundus images takenfrom 733 patients with DM, of which 454 (15%) images showed signs of DRvalidated by human graders. The system analyzed all images by detecting∗Corresponding author. E-mail: antal.balint@inf.unideb.hu, hajdu.andras@inf.unideb.huAddress: Faculty of Informatics, University of Debrecen, H-4002 Debrecen, POB 400, HungaryPhone: +36 52 512900/75124†B. Antal, M.K.G.S. Tavares, and L. Kovács contributed to this work equally.‡A. Csutak and A. Hajdu contributed to this work equally.Annales Mathematicae et Informaticae49 (2018) pp. 3–9doi: 10.33039/ami.2018.10.002http://ami.uni-eszterhazy.hu3anatomical components such as the optic disc, macula and vascular systemof the retina, then microaneurysms (MAs) and exudates as lesions. Oncethe presence/absence of the structures was determined, the combination ofthe results was subsequently used to provide a “DR/No DR” decision using amachine learning approach. The fundus images were graded by a trained andcertified expert grader as well and the final diagnosis was compared to theoutcome of the computer-based approach. The performances of the MA andexudate detectors used by the system were also evaluated. The area underthe ROC curve (AUC) was 0.90 with the best performing setting of the algo-rithm. The evaluation of the proposed approach shows that it performs wellagainst human graders and therefore might have the potential to be used in aclinical setting. There is a need for further evaluation on large scale, real-lifeclinical setting to explore its clinical utility.Keywords: diabetic retinopathy, image processing, automatic screening, de-cision support, distributed processingMSC: 68U10, 68M141. IntroductionDiabetes Mellitus (DM) is a chronic systemic disease characterized by an eleva-tion of the glucose level on the blood caused by lack of insulin in Type 1 DM andthe defective action of insulin in Type 2 DM. The prevalence of DM is rising inevery country; globally 422 million people have diabetes [1]. Of these, 175 mil-lion are undiagnosed and over 80% live in low- and middle-income countries [2].One important strategy for tackling the diabetes burden is to screen for undiag-nosed diabetes. By identifying and treating the disease early, we can minimizethe future risk of developing diabetic complications [2], and lessen the burden ofDR, nephropathy, peripheral neuropathy, cardiovascular problems, cerebrovasculardisease, hypertension and abnormalities of lipoprotein metabolism [3].DR is the most common complication of DM and it is the leading cause ofblindness in people of working-age worldwide. In the UK, where effective systematicscreening for DR has been in existence for over 10 years, DR is no longer the leadingcause of blindness in the working age group [4]. In most screening programs, DRis detected by trained graders analyzing the color fundus photographs taken forthis purpose. The imaging equipment is expensive to buy and maintain, and theresulting images require human grading. This process is both human resource andtraining heavy and is time-consuming favoring its use in more developed countries.It is difficult to implement comprehensive DR screening in the developing world,where personnel is not readily available and where priorities are competing for suchlimited resource. To overcome the aforementioned challenges, automated imageanalysis would be essential, and solutions are being developed to decrease theeconomic burden of human grading [5, 6, 7, 8].While the current automated systems are achieving ever increasing gradingaccuracy, their implementation is lagging behind due to consideration for scalabil-4 B. Antal, M. K. G. S. Tavares, L. Kovács and othersity issues around data processing and computational power needed in large-scalescreening programs.As the main contribution of this paper, we present an approach capable ofhandling big data required for processing images generated in DR screenings, andwe validate this approach by comparing the results to those generated by humanexpert graders. The approach is designed for processing large amounts of data atonce instead of a sequential analysis of images. We will also show that the completeanalysis of the fundus images outperforms the systems based on the detection oflesions like MA or exudates alone.2. MethodsThe method described in this paper allows the computer aided DR grading of fun-dus images to be run in a distributed manner on a computer grid. For the analysisitself, we have reused our previously published algorithm [9]. This algorithm relieson the features provided by several image processing methods that perform imagequality assessment and pre-screening, anatomical landmark detection (macula, op-tic disc, vascular network) and lesion detection (microaneurysms, exudates), anduses machine learning to classify images.The implementation was designed to allow scalable processing of the incomingimages in order to ensure the efficient analysis of large fundus image sets. We haveused the Son of Grid Engine framework to handle, distribute and share the tasksamong the available nodes. The algorithms were implemented in MATLAB, Java,C++, Perl, and shell scripts. The testing environment used consisted of 792 CPUcores, 808 GB RAM on 196 COTS computer nodes having various configurations(typically Intel Core 2 Quad processor and 4 GB RAM). Processing a single imagetook 15-20 minutes; however, 196 images can be analyzed parallel, so all of themcan be processed within this time frame. The system is linearly scalable, thus, byadding further nodes the throughput is increased by the number of nodes added.The system has a self-diagnostic module, which restarts the analysis of each failedimage. In practice, it is an efficient failure handling procedure, as comprehensivemonitoring of the system would be very resource demanding and failures appearindependently from the images (e.g., due to network communication errors, I/Otimeouts), therefore after some iterations each image is processed. A flow chart ofthe proposed system is shown in Figure 1.To train the system, a training set of 132 images was used, where only a “DR/NoDR” assignment was provided. The training elements were selected randomly andthe labels were assigned by a trained expert. We used a backward selection strategyand the soft average rule to select the ensemble of classifiers [9].To measure the performances of the exudate [10] and MA [11] detectors, the 0.5cut-off point was used for both, making them use the majority voting rule amongthe ensemble of detectors they use. That is, we considered those MA or exudatecandidates as actual lesions, where the majority of the respective lesion detectorsagreed on.Data analysis applied to diabetic retinopathy screening: performance evaluation 5Figure 1: Flow chart of the distributed diabetic retinopathy screen-ing system. Fundus images are sent to the grid system, where ascheduler optimally distributes the input to the computer nodes.As for its clinical setting, the system is able to receive an image set in a bulkupload and can directly return the results in email containing a “DR/No DR”labeling for each image. However, it is also possible to provide a probability valuebetween 0 and 1 for the stage of DR, which allows the ordering of the imagesregarding estimated severity. User registration is also supported, so the uploadeddatasets and the corresponding screening results can be safely checked via a webbrowser if the uploader prefers this option. Though our focus is on mass screening,it is also possible to visualize the features (e.g., lesions found) involved in thedecision within the system [12].3. ResultsTo evaluate the performance of our system we have analyzed 2932 fundus imageswith a resolution of 3072 × 2048 pixels and 45◦ FOV taken from 733 patients withDM. Within this dataset, human graders labeled 454 (15%) images as showingsigns of DR. In a recent study [13], the detection of referable retinopathy by humangraders has been found to have sensitivities between 77 and 100% with specificitiesbetween 94% and 100%. We considered the labeling by human graders as groundtruth in the evaluation.The system has three possible use cases to operate, namely exudate, MA andcomplex (combined) approach. Accordingly, we have fitted a Receiver OperatingCharacteristics (ROC) [14] curve of the findings to the proposed complex method,and also on those of the exudate and the MA detection only approaches. Therespective fitted areas under the ROC curves (AUC) are 0.90, 0.695, and 0.708 (seeFigure 2). We applied Friedman’s rank test [15] and Tukey’s post-hoc test [16]to compare the results of the three approaches. The analysis showed a significantdifference in the means of the three approaches (p<0.0001) and established that6 B. Antal, M. K. G. S. Tavares, L. Kovács and othersthe complex method performed significantly better than the other two.Figure 2: Receiver Operating Characteristics (ROC) curves ofthe complex method, the exudate detection only and the microa-neurysm (MA) detection only approaches.4. DiscussionThe approach presented here has been previously tested on a publicly availabledataset, where it was evaluated to have a 0.90 AUC, which is similar to that ofother state-of-the-art techniques: Hansen et al. [17] (0.878), Agurto et al. [18](0.89), and Abràmoff et al. [19] (0.937). However, these figures are not directlycomparable since they were established on datasets having different proportions ofpatients showing signs of DR [9], and depending on the prevalence of the disease,these results can vary widely. In our current analysis of the screening images, wefound very similar AUC to that of the published literature and so that places ouralgorithm well within the published limits.The distributed processing method makes it an ideal candidate to use as a back-end for large-scale screening and telemedicine approaches for two reasons: first, thelinear scalability of our system provides the constant processing time of a singleimage for the whole image set with including the same number of processing nodesas the number of images. This approach makes the screening predictable regardingtime and computational resources. Second, only a “DR/No DR” assignment for asmall training dataset is required to teach the system instead of hand-segmentedimages or lesion annotations. While the proposed approach performed well in thissetting, a better automation is required in the future for adjusting its parametersaccording to the user requirements regarding e.g., sensitivity/specificity.Data analysis applied to diabetic retinopathy screening: performance evaluation 7Acknowledgements. This work is supported by the EFOP-3.6.2-16-2017-00015project. The project has been supported by the European Union, co-financed by theEuropean Social Fund. A. Csutak was the recipient of Bolyai and Szodoray Grants.T. Pető was funded by NIHR BMRC at Moorfields Eye Hospital Foundation Trustand UCL Institute of Ophthalmology.References[1] World Health Organization, Diabetes Fact Sheet,URL: http://www.who.int/mediacentre/factsheets/fs312/en/[2] International Diabetes Federation, IDF Diabetes Atlas, 8th edition,URL: http://www.diabetesatlas.org[3] Bunce, C., Wormald, R., Leading causes of certification for blindness and partialsight in England & Wales, BMC Public Health, Vol. 6 (2006), 1–7.https://doi.org/10.1186/1471-2458-6-58[4] Graham-Rowe, E., Lorencatto, F., Lawrenson, J.G. et al., Barriers andenablers to diabetic retinopathy screening attendance: protocol for a systematicreview, Systematic Reviews, Vol. 5 (2016), 1–9.https://doi.org/10.1186/s13643-016-0309-2[5] Bhaskaranand, M., Cuadros, J., Ramachandra, C. et al., EyeArt + Eye-PACS: Automated retinal image analysis for diabetic retinopathy screening in atelemedicine system, Proceedings of the Ophthalmic Medical Image Analysis SecondInternational Workshop (OMIA 2015), Munich, Germany, October 9, 2015, 105–112.https://doi.org/10.17077/omia.1033[6] Walton, O.B., Garoon, R.B., Weng, C.Y. et al., Evaluation of automated tel-eretinal screening program for diabetic retinopathy, JAMA Ophthalmology, Vol. 134(2016), 204–209.https://doi.org/10.1001/jamaophthalmol.2015.5083[7] Scotland, G.S., McNamee, P., Fleming, A.D. et al., Costs and consequencesof automated algorithms versus manual grading for the detection of referable diabeticretinopathy, British Journal of Ophthalmology, Vol. 94 (2010), 712–719.https://doi.org/10.1136/bjo.2008.151126[8] Mansberger, S.L., Sheppler, C., Barker, G. et al., Long-term comparativeeffectiveness of telemedicine in providing diabetic retinopathy screening examina-tions: a randomized clinical trial, JAMA Ophthalmology, Vol. 133 (2015), 518–525.https://doi.org/10.1001/jamaophthalmol.2015.1[9] Antal, B., Hajdu, A., An ensemble-based system for automatic screening of dia-betic retinopathy, Knowledge-Based Systems, Vol. 60 (2014), 20–27.https://doi.org/10.1016/j.knosys.2013.12.023[10] Nagy, B., Antal, B., Harangi, B., Hajdu, A., Ensemble-based exudate de-tection in color fundus images, 7th International Symposium on Image and SignalProcessing and Analysis (ISPA 2011), Dubrovnik, Croatia, September 4–6, 2011,700–703.8 B. Antal, M. K. G. S. Tavares, L. Kovács and others[11] Antal, B., Hajdu, A., An ensemble-based system for microaneurysm detectionand diabetic retinopathy grading, IEEE Transactions on Biomedical Engineering,Vol. 59 (2012), 1720–1726.https://doi.org/10.1109/tbme.2012.2193126[12] Tóth, J., Kovács, L., Harangi, B. et al., An online benchmark system forimage processing algorithms, 5th IEEE International Conference on Cognitive In-focommunications (CogInfoCom 2014), Vietri sul Mare, Italy, November 5–7, 2014,377–382.https://doi.org/10.1109/coginfocom.2014.7020482[13] Tufail, A., Kapetanakis, V.V., Salas-Vega S. et al., An observational studyto assess if automated diabetic retinopathy image assessment software can replace oneor more steps of manual imaging grading and to determine their cost-effectiveness,Health Technology Assessment, Vol. 20 (2016), 1-72.https://doi.org/10.3310/hta20920[14] Hanley, J.A., McNeil, B.J., The meaning and use of the area under a receiveroperating characteristic (ROC) curve, Radiology, Vol. 143 (1982), 29–36.https://doi.org/10.1148/radiology.143.1.7063747[15] Friedman, M., The use of ranks to avoid the assumption of normality implicit in theanalysis of variance, Journal of the American Statistical Association, Vol. 32 (1937),675–701.https://doi.org/10.2307/2279372[16] Tukey, J.W., Comparing individual means in the analysis of variance, Biometrics,Vol. 5 (1949), 99–114.https://doi.org/10.2307/3001913[17] Hansen, M.B., Abràmoff, M.D., Folk J.C. et al., Results of automated retinalimage analysis for detection of diabetic retinopathy from the Nakuru Study, Kenya,PLoS ONE, Vol. 10 (2015), 1–9.https://doi.org/10.1371/journal.pone.0139148[18] Agurto, C., Barriga, E.S., Murray, V. et al., Automatic detection of diabeticretinopathy and age-related macular degeneration in digital fundus images, Inves-tigative Ophthalmology & Visual Science, Vol. 52 (2011), 5862–5871.https://doi.org/10.1167/iovs.10-7075[19] Abràmoff, M.D., Folk, J.C., Han, D.P. et al., Automated analysis of retinalimages for detection of referable diabetic retinopathy, JAMA Ophthalmology, Vol. 131(2013), 351–357.https://doi.org/10.1001/jamaophthalmol.2013.1743Data analysis applied to diabetic retinopathy screening: performance evaluation 9

Data analysis applied to diabetic retinopathy screening: Performance evaluation

http://publikacio.uni-eszterhazy.hu/2458/1/AMI_49_from3to9.pdf

Data analysis applied to diabetic retinopathy screening: Performance evaluation

Abstract

Similar works

Full text

Available Versions

EKE Repository of Publications