Plus Disease in Retinopathy of Prematurity: Diagnostic Trends in 2016 Versus 2007

To identify any temporal trends in the diagnosis of plus disease in retinopathy of prematurity (ROP) by experts. Reliability analysis. ROP experts were recruited in 2007 and 2016 to classify 34 wide-field fundus images of ROP as plus, pre-plus, or normal, coded as “3,” “2,” and “1,” respectively, in the database. The main outcome was the average calculated score for each image in each cohort. Secondary outcomes included correlation on the relative ordering of the images in 2016 vs 2007, interexpert agreement, and intraexpert agreement. The average score for each image was higher for 30 of 34 (88%) images in 2016 compared with 2007, influenced by fewer images classified as normal (P < .01), a similar number of pre-plus (P = .52), and more classified as plus (P < .01). The mean weighted kappa values in 2006 were 0.36 (range 0.21–0.60), compared with 0.22 (range 0–0.40) in 2016. There was good correlation between rankings of disease severity between the 2 cohorts (Spearman rank correlation ρ = 0.94), indicating near-perfect agreement on relative disease severity. Despite good agreement between cohorts on relative disease severity ranking, the higher average score and classifications for each image demonstrate that experts are diagnosing pre-plus and plus disease at earlier stages of disease severity in 2016, compared with 2007. This has implications for patient care, research, and teaching, and additional studies are needed to better understand this temporal trend in image-based plus disease diagnosis

Evaluation of a Deep Learning–Derived Quantitative Retinopathy of Prematurity Severity Scale

To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale. Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity. Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9. A quantitative vascular severity score (1–9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement. Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale. For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale. A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis

Variability in Plus Disease Diagnosis using Single and Serial Images

PurposeTo assess changes in retinopathy of prematurity (ROP) diagnosis in single and serial retinal images.DesignCohort study.ParticipantsCases of ROP recruited from the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) consortium evaluated by 7 graders.MethodsSeven ophthalmologists reviewed both single and 3 consecutive serial retinal images from 15 cases with ROP, and severity was assigned as plus, preplus, or none. Imaging data were acquired during routine ROP screening from 2011 to 2015, and a reference standard diagnosis was established for each image. A secondary analysis was performed using the i-ROP deep learning system to assign a vascular severity score (VSS) to each image, ranging from 1 to 9, with 9 being the most severe disease. This score has been previously demonstrated to correlate with the International Classification of ROP. Mean plus disease severity was calculated by averaging 14 labels per image in serial and single images to decrease noise.Main outcome measuresGrading severity of ROP as defined by plus, preplus, or no ROP.ResultsAssessment of serial retinal images changed the grading severity for &gt; 50% of the graders, although there was wide variability. Cohen's kappa ranged from 0.29 to 1.0, which showed a wide range of agreement from slight to perfect by each grader. Changes in the grading of serial retinal images were noted more commonly in cases of preplus disease. The mean severity in cases with a diagnosis of plus disease and no disease did not change between single and serial images. The ROP VSS demonstrated good correlation with the range of expert classifications of plus disease and overall agreement with the mode class (P&nbsp;= 0.001). The VSS correlated with mean plus disease severity by expert diagnosis (correlation coefficient, 0.89). The more aggressive graders tended to be influenced by serial images to increase the severity of their grading. The VSS also demonstrated agreement with disease progression across serial images, which progressed to preplus and plus disease.ConclusionsClinicians demonstrated variability in ROP diagnosis when presented with both single and serial images. The use of deep learning as a quantitative assessment of plus disease has the potential to standardize ROP diagnosis and treatment

Federated Learning for Multicenter Collaboration in Ophthalmology

To compare the performance of deep learning classifiers for the diagnosis of plus disease in retinopathy of prematurity (ROP) trained using 2 methods for developing models on multi-institutional data sets: centralizing data versus federated learning (FL) in which no data leave each institution. Evaluation of a diagnostic test or technology. Deep learning models were trained, validated, and tested on 5255 wide-angle retinal images in the neonatal intensive care units of 7 institutions as part of the Imaging and Informatics in ROP study. All images were labeled for the presence of plus, preplus, or no plus disease with a clinical label and a reference standard diagnosis (RSD) determined by 3 image-based ROP graders and the clinical diagnosis. We compared the area under the receiver operating characteristic curve (AUROC) for models developed on multi-institutional data, using a central approach initially, followed by FL, and compared locally trained models with both approaches. We compared the model performance (κ) with the label agreement (between clinical and RSD), data set size, and number of plus disease cases in each training cohort using the Spearman correlation coefficient (CC). Model performance using AUROC and linearly weighted κ. Four settings of experiment were used: FL trained on RSD against central trained on RSD, FL trained on clinical labels against central trained on clinical labels, FL trained on RSD against central trained on clinical labels, and FL trained on clinical labels against central trained on RSD (P = 0.046, P = 0.126, P = 0.224, and P = 0.0173, respectively). Four of the 7 (57%) models trained on local institutional data performed inferiorly to the FL models. The model performance for local models was positively correlated with the label agreement (between clinical and RSD labels, CC = 0.389, P = 0.387), total number of plus cases (CC = 0.759, P = 0.047), and overall training set size (CC = 0.924, P = 0.002). We found that a trained FL model performs comparably to a centralized model, confirming that FL may provide an effective, more feasible solution for interinstitutional learning. Smaller institutions benefit more from collaboration than larger institutions, showing the potential of FL for addressing disparities in resource access

Photodynamic therapy of subfoveal choroidal neovascularization in pathologic myopia with verteporfin - 1-year results of a randomized clinical trial - VIP report no. 1

Objective: To determine if photodynamic therapy with verteporfin (Visudyne; CIBA Vision Corp, Duluth, GA) can improve the chance of stabilizing or improving vision (m and best-corrected visual acuity (Snellen equivalent) of approximately 20/100 or better. Intervention: Patients were randomly assigned (2:1) to verteporfin (6 mg per square meter of body surface area; n = 81) or placebo (5% dextrose in water, n = 39) administered via intravenous infusion of 30 ml over 10 minutes. Fifteen minutes after the start of the infusion, a laser light at 689 nm was delivered at an intensity of 600 mW/cm(2) over 83 seconds to give a light dose of 50 J/cm(2) to a round spot size on the retina with a diameter of 1000 mum larger than the greatest linear dimension of the choroidal neovascular lesion. At follow-up examinations every 3 months, retreatment with either verteporfin or placebo (as assigned at baseline) was applied to areas of fluorescein leakage if present. Main Outcome Measures: The primary outcome was the proportion of eyes at the follow-up examination 12 months after study entry with fewer than eight letters (approximately 1.5 lines) of visual acuity lost, adhering to an intent-to-treat analysis. Results: At baseline, move than 90% of each group had evidence of classic CNV (regardless of whether occult CNV was present) and only 12 (15%) and 5 (13%) cases in the verteporfin and placebo groups, respectively, had occult CNV (regardless of whether classic CNV was present). Seventy-nine of the 81 verteporfin-treated patients (98%) compared with 36 of the 39 placebo-treated patients (92%) completed the month 12 examination. Visual acuity, contrast sensitivity, and fluorescein angiographic outcomes were better in the verteporfin-treated eyes than in the placebo-treated eyes at every follow-up examination through the month 12 examination. At the month 12 examination, 58 (72%) of the verteporfin-treated patients compared with 17 (44%) of the placebo-treated patients lost fewer than eight letters (P 1 line). Seventy (86%) of the verteporfin-treated patients compared with 26 (67%) of the placebo-treated patients lost fewer than 15 letters (P = 0.01), Few ocular or other systemic adverse events were associated with verteporfin therapy compared with placebo treatment. Conclusions: Because photodynamic therapy with verteporfin can safely increase the chance of stabilizing or improving vision in patients with subfoveal CNV from pathologic myopia compared with a placebo, we recommend ophthalmologists consider verteporfin therapy for treatment of such patients. Ophthalmology 2001; 108:841-852 (C) 2001 by the American Academy of Ophthalmology

Plus Disease in Retinopathy of Prematurity

To identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP). We developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD. Eight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units. Expert classification of images of plus disease in ROP. Interexpert agreement (weighted κ statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement). There was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted κ, 0–0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R2 = 0.82; and dataset B: P < 0.05 and adjusted R2 = 0.6615). There is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future

Verteporfin therapy of subfoveal choroidal neovascularization in age-related macular degeneration: Two-year results of a randomized clinical trial including lesions with occult with no classic choroidal neovascularization-verteporfin in photodynamic therapy report 2

PURPOSE: To determine if photodynamic therapy with verteporfin (Visudyne; Novartis AG, Bulach, Switzerland), termed verteporfin therapy, can safely reduce the risk of vision loss compared with a placebo (with sham treatment) in patients with subfoveal choroidal neovascularization caused by age-related macular degeneration who were identified with a lesion composed of occult with no classic choroidal neovascularization, or with presumed early onset classic choroidal neovascularization with good visual acuity letter score, METHODS: This was a double-masked, placebo controlled (sham treatment), randomized, multicenter clinical trial involving 28 ophthalmology practices in Europe and North America. The study population was patients with age related macular degeneration, with subfoveal choroidal neovascularization lesions measuring no greater than 5400 mum in greatest linear dimension with either 1) occult with no classic choroidal neovascularization, best-corrected visual acuity score of at least 50 (Snellen equivalent approximately 20/100), and evidence of hemorrhage or recent disease progression; or 2) evidence of classic choroidal neovascularization with a best-corrected visual acuity score of at least 70 (better than a Snellen equivalent of approximately 20/40); assigned randomly (2:1) to verteporfin therapy or placebo therapy. Verteporfin (6 mg per square meter of body surface area) or placebo (5% dextrose in water) was administered by means of intravenous infusion of 30 mi over 10 minutes. Fifteen minutes after the start of the infusion, a laser light at 689 nm delivered 50 J/cm(2) by application of an intensity of 600 mW/cm(2) over 83 seconds using a spot size with a diameter 1000 mum larger than the greatest linear dimension of the choroidal neovascularization lesion on the retina. At follow-up examinations every 3 months, retreatment with the same regimen was applied if angiography showed fluorescein leakage. The main outcome measure was at least moderate vision loss, that is, a loss of at least 15 letters (approximately 3 lines), adhering to an intent-to treat analysis with the last observation carried forward to impute for missing data. RESULTS: Two hundred ten (93%) and 193 (86%) of the 225 patients in the verteporfin group compared with 104 (91%) and 99 (87%) of the 114 patients in the placebo group completed the month 12 and 24 examinations, respectively. On average, verteporfin-treated patients received five treatments over the 24 months of follow-up. The primary outcome was similar for the verteporfin-treated and the placebo-treated eyes through the month 12 examination, although a number of secondary visual and angiographic outcomes significantly favored the verteporfin-treated group. Between the month 12 and 24 examinations, the treatment benefit grew so that by the month 24 examination, the vertepor-fin-treated eyes were less likely to have moderate or severe vision loss. Of the 225 verteporfin-treated patients, 121 (54%) compared with 76 (67%) of 114 placebo-treated patients lost at least 15 letters (P =.023). Likewise, 61 of the verteporfin-treated patients (30%) compared with 54 of the placebo-treated patients (47%) lost at least 30 letters (P = .001). Statistically significant results favoring verteporfin therapy at the month 24 examination were consistent between the total population and the subgroup of patients with a baseline lesion composition identified as occult choroidal neovascularization with no classic choroidal neovascularization, This subgroup included 166 of the 225 verteporfin-treated patients (74%) and 92 of the 114 placebo-treated patients (81%). In these patients, 91 of the verteporfin-treated group (55%) compared with 63 of the placebo-treated group (68%) lost at least 15 letters (P =.032), whereas 48 of the verteporfin-treated group (29%) and 43 of the placebo-treated group (47%) lost at least 30 letters (P =.004). Other secondary outcomes, including visual acuity letter score worse than 34 (approximate Snellen equivalent of 20/200 or worse), mean change in visual acuity letter score, development of classic choroidal neovascularization, progression of classic choroidal neovascularization and size of lesion, favored the verteporfin-treated group at both the month 12 and month 24 examination for both the entire study group and the subgroup of cases with occult with no classic choroidal neovascularization at baseline. Subgroup analyses of lesions composed of occult with no classic choroidal neovascularization at baseline suggested that the treatment benefit was greater for patients with either smaller lesions (4 disc areas or less) or lower levels of visual acuity (letter score less than 65, an approximate Snellen equivalent of 20/50(-1) or worse) at baseline. Prospectively planned multivariable analyses confirmed that these two baseline variables affected the magnitude of treatment benefit. Of the 123 verteporfin-treated patients and 64 placebo-heated patients with either visual acuity score Less than 65 or lesion size 4 disc areas or less at baseline, 60 (49%) and 48 (75%) lost at least 15 letters (P 2001 by Elsevier Science Inc. All rights reserved.)