Performance of Automated Machine Learning in Predicting Outcomes of Pneumatic Retinopexy

Purpose: Automated machine learning (AutoML) has emerged as a novel tool for medical professionals lacking coding experience, enabling them to develop predictive models for treatment outcomes. This study evaluated the performance of AutoML tools in developing models predicting the success of pneumatic retinopexy (PR) in treatment of rhegmatogenous retinal detachment (RRD). These models were then compared with custom models created by machine learning (ML) experts. Design: Retrospective multicenter study. Participants: Five hundred and thirty nine consecutive patients with primary RRD that underwent PR by a vitreoretinal fellow at 6 training hospitals between 2002 and 2022. Methods: We used 2 AutoML platforms: MATLAB Classification Learner and Google Cloud AutoML. Additional models were developed by computer scientists. We included patient demographics and baseline characteristics, including lens and macula status, RRD size, number and location of breaks, presence of vitreous hemorrhage and lattice degeneration, and physicians’ experience. The dataset was split into a training (n = 483) and test set (n = 56). The training set, with a 2:1 success-to-failure ratio, was used to train the MATLAB models. Because Google Cloud AutoML requires a minimum of 1000 samples, the training set was tripled to create a new set with 1449 datapoints. Additionally, balanced datasets with a 1:1 success-to-failure ratio were created using Python. Main Outcome Measures: Single-procedure anatomic success rate, as predicted by the ML models. F2 scores and area under the receiver operating curve (AUROC) were used as primary metrics to compare models. Results: The best performing AutoML model (F2 score: 0.85; AUROC: 0.90; MATLAB), showed comparable performance to the custom model (0.92, 0.86) when trained on the balanced datasets. However, training the AutoML model with imbalanced data yielded misleadingly high AUROC (0.81) despite low F2-score (0.2) and sensitivity (0.17). Conclusions: We demonstrated the feasibility of using AutoML as an accessible tool for medical professionals to develop models from clinical data. Such models can ultimately aid in the clinical decision-making, contributing to better patient outcomes. However, outcomes can be misleading or unreliable if used naively. Limitations exist, particularly if datasets contain missing variables or are highly imbalanced. Proper model selection and data preprocessing can improve the reliability of AutoML tools. Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article

METformin for the MINimization of Geographic Atrophy Progression (METforMIN): A Randomized Trial

Purpose: Metformin use has been associated with a decreased risk of age-related macular degeneration (AMD) progression in observational studies. We aimed to evaluate the efficacy of oral metformin for slowing geographic atrophy (GA) progression. Design: Parallel-group, multicenter, randomized phase II clinical trial. Participants: Participants aged ≥ 55 years without diabetes who had GA from atrophic AMD in ≥ 1 eye. Methods: We enrolled participants across 12 clinical centers and randomized participants in a 1:1 ratio to receive oral metformin (2000 mg daily) or observation for 18 months. Fundus autofluorescence imaging was obtained at baseline and every 6 months. Main Outcome Measures: The primary efficacy endpoint was the annualized enlargement rate of the square root-transformed GA area. Secondary endpoints included best-corrected visual acuity (BCVA) and low luminance visual acuity (LLVA) at each visit. Results: Of 66 enrolled participants, 34 (57 eyes) were randomized to the observation group and 32 (53 eyes) were randomized to the treatment group. The median follow-up duration was 13.9 and 12.6 months in the observation and metformin groups, respectively. The mean ± standard error annualized enlargement rate of square root transformed GA area was 0.35 ± 0.04 mm/year in the observation group and 0.42 ± 0.04 mm/year in the treatment group (risk difference = 0.07 mm/year, 95% confidence interval = −0.05 to 0.18 mm/year; P = 0.26). The mean ± standard error decline in BCVA was 4.8 ± 1.7 letters/year in the observation group and 3.4 ± 1.1 letters/year in the treatment group (P = 0.56). The mean ± standard error decline in LLVA was 7.3 ± 2.5 letters/year in the observation group and 0.8 ± 2.2 letters/year in the treatment group (P = 0.06). Fourteen participants in the metformin group experienced nonserious adverse events related to metformin, with gastrointestinal side effects as the most common. No serious adverse events were attributed to metformin. Conclusions: The results of this trial as conducted do not support oral metformin having effects on reducing the progression of GA. Additional placebo-controlled trials are needed to explore the role of metformin for AMD, especially for earlier stages of the disease. Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article