HE-MAN -- Homomorphically Encrypted MAchine learning with oNnx models

Machine learning (ML) algorithms are increasingly important for the success of products and services, especially considering the growing amount and availability of data. This also holds for areas handling sensitive data, e.g. applications processing medical data or facial images. However, people are reluctant to pass their personal sensitive data to a ML service provider. At the same time, service providers have a strong interest in protecting their intellectual property and therefore refrain from publicly sharing their ML model. Fully homomorphic encryption (FHE) is a promising technique to enable individuals using ML services without giving up privacy and protecting the ML model of service providers at the same time. Despite steady improvements, FHE is still hardly integrated in today's ML applications. We introduce HE-MAN, an open-source two-party machine learning toolset for privacy preserving inference with ONNX models and homomorphically encrypted data. Both the model and the input data do not have to be disclosed. HE-MAN abstracts cryptographic details away from the users, thus expertise in FHE is not required for either party. HE-MAN 's security relies on its underlying FHE schemes. For now, we integrate two different homomorphic encryption schemes, namely Concrete and TenSEAL. Compared to prior work, HE-MAN supports a broad range of ML models in ONNX format out of the box without sacrificing accuracy. We evaluate the performance of our implementation on different network architectures classifying handwritten digits and performing face recognition and report accuracy and latency of the homomorphically encrypted inference. Cryptographic parameters are automatically derived by the tools. We show that the accuracy of HE-MAN is on par with models using plaintext input while inference latency is several orders of magnitude higher compared to the plaintext case

Predicting Drusen Regression from OCT in Patients with Age-Related Macular Degeneration

Age-related macular degeneration (AMD) is a leading cause of blindness in developed countries. The presence of drusen is the hallmark of early/intermediate AMD, and their sudden regression is strongly associated with the onset of late AMD. In this work we propose a predictive model of drusen regression using optical coherence tomography (OCT) based features. First, a series of automated image analysis steps are applied to segment and characterize individual drusen and their development. Second, from a set of quantitative features, a random forest classifiser is employed to predict the occurrence of individual drusen regression within the following 12 months. The predictive model is trained and evaluated on a longitudinal OCT dataset of 44 eyes from 26 patients using leave-one-patient-out cross-validation. The model achieved an area under the ROC curve of 0.81, with a sensitivity of 0.74 and a specificity of 0.73. The presence of hyperreflective foci and mean drusen signal intensity were found to be the two most important features for the prediction. This preliminary study shows that predicting drusen regression is feasible and is a promising step toward identification of imaging biomarkers of incoming regression

Automated Fovea Detection in Spectral Domain Optical Coherence Tomography Scans of Exudative Macular Disease

In macular spectral domain optical coherence tomography (SD-OCT) volumes, detection of the foveal center is required for accurate and reproducible follow-up studies, structure function correlation, and measurement grid positioning. However, disease can cause severe obscuring or deformation of the fovea, thus presenting a major challenge in automated detection. We propose a fully automated fovea detection algorithm to extract the fovea position in SD-OCT volumes of eyes with exudative maculopathy. The fovea is classified into 3 main appearances to both specify the detection algorithm used and reduce computational complexity. Based on foveal type classification, the fovea position is computed based on retinal nerve fiber layer thickness. Mean absolute distance between system and clinical expert annotated fovea positions from a dataset comprised of 240 SD-OCT volumes was 162.3 µm in cystoid macular edema and 262 µm in nAMD. The presented method has cross-vendor functionality, while demonstrating accurate and reliable performance close to typical expert interobserver agreement. The automatically detected fovea positions may be used as landmarks for intra- and cross-patient registration and to create a joint reference frame for extraction of spatiotemporal features in “big data.” Furthermore, reliable analyses of retinal thickness, as well as retinal structure function correlation, may be facilitated

Ganglion cell layer thickening in well-controlled patients with type 1 diabetes: an early sign for diabetic retinopathy?

PURPOSE To evaluate early changes in retinal layers using optical coherence tomography (OCT) in patients with long-standing type 1 diabetes (DM1) receiving intensified insulin therapy. METHODS In a cross-sectional case-control study 150 patients with DM1 and 150 age- and sex-matched healthy control participants underwent OCT imaging. Scans of both eyes were analysed for different layers (NFL, GCL (+IPL), INL, outer layer complex (OLC, including OPL, ONL and ELM) and photoreceptors (PR)) in all subfields of an ETDRS grid. All analyses were performed semi-automatically using custom software by certified graders of the Vienna Reading Center. ANOVA models were used to compare the mean thickness of the layers between patients and controls. RESULTS Six hundred eyes with 512 datapoints in 49 b-scans in each OCT were analysed. Mean thickness in patients/controls was 31.35 μm/30.65 μm (NFL, p = 0.0347), 76.7 μm/73.15 μm (GCL, p ≤ 0.0001), 36.29 μm/37.13 μm (INL, p = 0.0116), 114.34 μm/112.02 μm (OLC, p < 0.0001) and 44.71 μm/44.69 μm (PR, p = 0.9401). When evaluating the ETDRS subfields separately for clinically meaningful hypotheses, a significant swelling of the GCL in patients could be found uniformly and a central swelling for the OLC, whereas the distribution of NFL and INL thickening suggests that their statistical significance was not clinically relevant. CONCLUSION These preliminary results demonstrate that preclinical retinal changes in patients with long-standing DM1 can be found by retinal layer evaluation. However, the changes are layer-specific, with significant thickening of the GCL and less so of the OLC suggesting a role as an early sign for diffuse swelling and the evolution of DME even in well-controlled diabetes

Detection and Differentiation of Intraretinal Hemorrhage in Spectral Domain Optical Coherence Tomography.

PURPOSE The purpose of this study was to classify and detect intraretinal hemorrhage (IRH) in spectral domain optical coherence tomography (SD-OCT). METHODS Initially the presentation of IRH in BRVO-patients in SD-OCT was described by one reader comparing color-fundus (CF) and SD-OCT using dedicated software. Based on these established characteristics, the presence and the severity of IRH in SD-OCT and CF were assessed by two other masked readers and the inter-device and the inter-observer agreement were evaluated. Further the area of IRH was compared. RESULTS About 895 single B-scans of 24 eyes were analyzed. About 61% of SD-OCT scans and 46% of the CF-images were graded for the presence of IRH (concordance: 73%, inter-device agreement: k = 0.5). However, subdivided into previously established severity levels of dense (CF: 21.3% versus SD-OCT: 34.7%, k = 0.2), flame-like (CF: 15.5% versus SD-OCT: 45.5%, k = 0.3), and dot-like (CF: 32% versus SD-OCT: 24.4%, k = 0.2) IRH, the inter-device agreement was weak. The inter-observer agreement was strong with k = 0.9 for SD-OCT and k = 0.8 for CF. The mean area of IRH detected on SD-OCT was significantly greater than on CF (SD-OCT: 11.5 ± 4.3 mm(2) versus CF: 8.1 ± 5.5 mm(2), p = 0.008). CONCLUSIONS IRH seems to be detectable on SD-OCT; however, the previously established severity grading agreed weakly with that assessed by CF

Machine Learning of the Progression of Intermediate Age-Related Macular Degeneration Based on OCT Imaging

Purpose: To develop a data-driven interpretable predictive model of incoming drusen regression as a sign of disease activity and identify optical coherence tomography (OCT) biomarkers associated with its risk in intermediate age-related macular degeneration (AMD). Methods: Patients with AMD were observed every 3 months, using Spectralis OCT imaging, for a minimum duration of 12 months and up to a period of 60 months. Segmentation of drusen and the overlying layers was obtained using a graph-theoretic method, and the hyperreflective foci were segmented using a voxel classification method. Automated image analysis steps were then applied to identify and characterize individual drusen at baseline, and their development was monitored at every follow-up visit. Finally, a machine learning method based on a sparse Cox proportional hazard regression was developed to estimate a risk score and predict the incoming regression of individual drusen. Results: The predictive model was trained and evaluated on a longitudinal dataset of 61 eyes from 38 patients using cross-validation. The mean follow-up time was 37.8 13.8 months. A total of 944 drusen were identified at baseline, out of which 249 (26%) regressed during follow-up. The prediction performance was evaluated as area under the curve (AUC) for different time periods. Prediction within the first 2 years achieved an AUC of 0.75. Conclusions: The predictive model proposed in this study represents a promising step toward image-guided prediction of AMD progression. Machine learning is expected to accelerate and contribute to the development of new therapeutics that delay the progression of AMD.(VLID)484328

A systematic comparison of spectral-domain optical coherence tomography and Fundus Autofluorescence in patients with geographic atrophy

Purpose To evaluate spectral-domain optical coherence tomography (SD-OCT) in providing reliable and reproducible parameters for grading geographic atrophy (GA) compared with fundus autofluorescence (FAF) images acquired by confocal scanning laser ophthalmoscopy (cSLO). Design Prospective observational study. Participants A total of 81 eyes of 42 patients with GA. Methods Patients with atrophic age-related macular degeneration (AMD) were enrolled on the basis of total GA lesion size ranging from 0.5 to 7 disc areas and best-corrected visual acuity of at least 20/200. A novel combined cSLO-SD-OCT system (Spectralis HRA-OCT, Heidelberg Engineering, Heidelberg, Germany) was used to grade foveal involvement and to manually measure disease extent at the level of the outer neurosensory layers and retinal pigment epithelium (RPE) at the site of GA lesions. Two readers of the Vienna Reading Center graded all obtained volume stacks (20×20 degrees), and the results were correlated to FAF. Main Outcome Measures Choroidal signal enhancements and alterations of the RPE, external limiting membrane (ELM), and outer plexiform layer by SD-OCT. These parameters were compared with the lesion measured with severely decreased FAF. Results Foveal involvement or sparing was definitely identified in 75 of 81 eyes based on SD-OCT by both graders (inter-grader agreement: κ=0.6, P < 0.01). In FAF, inter-grader agreement regarding foveal involvement was lower (48/81 eyes, inter-grader agreement: κ=0.3, P < 0.01). Severely decreased FAF was measured over a mean area of 8.97 mm2 for grader 1 (G1) and 9.54 mm2 for grader 2 (G2), consistent with the mean SD-OCT quantification of the sub-RPE choroidal signal enhancement (8.9 mm2 [G1] −9.4 mm2 [G2]) and ELM loss with 8.7 mm2 (G1) −10.2 mm2 (G2). In contrast, complete morphologic absence of the RPE layer by SD-OCT was significantly smaller than the GA size in FAF (R2=0.400). Inter-reader agreement was highest regarding complete choroidal signal enhancement (0.98) and ELM loss (0.98). Conclusions Absence of FAF in GA lesions is consistent with morphologic RPE loss or advanced RPE disruption and is associated with alterations of the outer retinal layers as identified by SD-OCT. Lesion size is precisely determinable by SD-OCT, and foveal involvement is more accurate by SD-OCT than by FAF. Financial Disclosure(s) Proprietary or commercial disclosure may be found after the references

Evaluating the impact of vitreomacular adhesion on anti-VEGF therapy for retinal vein occlusion using machine learning

Vitreomacular adhesion (VMA) represents a prognostic biomarker in the management of exudative macular disease using anti-vascular endothelial growth factor (VEGF) agents. However, manual evaluation of VMA in 3D optical coherence tomography (OCT) is laborious and data on its impact on therapy of retinal vein occlusion (RVO) are limited. The aim of this study was to (1) develop a fully automated segmentation algorithm for the posterior vitreous boundary and (2) to study the effect of VMA on anti-VEGF therapy for RVO. A combined machine learning/graph cut segmentation algorithm for the posterior vitreous boundary was designed and evaluated. 391 patients with central/branch RVO under standardized ranibizumab treatment for 6/12 months were included in a systematic post-hoc analysis. VMA (70%) was automatically differentiated from non-VMA (30%) using the developed method combined with unsupervised clustering. In this proof-of-principle study, eyes with VMA showed larger BCVA gains than non-VMA eyes (BRVO: 1512 vs. 1111 letters, p=0.02; CRVO: 1814 vs. 913 letters, p<0.01) and received a similar number of retreatments. However, this association diminished after adjustment for baseline BCVA, also when using more fine-grained VMA classes. Our study illustrates that machine learning represents a promising path to assess imaging biomarkers in OCT.(VLID)460644

The Distribution of Leakage on Fluorescein Angiography in Diabetic Macular Edema: A New Approach to Its Etiology

Purpose: To determine the distribution of leakage on fluorescein angiography (FA) and explore the clinically protective role of astrocytes against damage to the inner blood retinal barrier (iBRB) in diabetic macular edema (DME). Methods: A consecutive case series of 87 eyes of 87 patients with DME was included. We measured the leakage area in each field of the Early Treatment Diabetic Retinopathy Study (ETDRS) grid on late-phase FA images. The normative thickness of the nerve fiber layer (NFL), in which the astrocytes are confined, was derived from a previous work using spectral-domain optical coherence tomography. We explored the difference in leakage areas in every two fields. Moreover, we investigated the correlation between the mean of the leakage area and the mean of thickness of the NFL in each ETDRS field. Results: The leakage areas in the nasal, inferior, superior, and temporal fields were 2.34 mm, 2.84 mm, 3.03 mm, and 3.96 mm. The difference in leakage area between each two fields was significant in all cases (P < 0.05) except between the inferior and superior fields (P = 0.65). The temporal field was the only field that showed leakage in all 87 cases. The correlation between the leakage area and the thickness of the NFL in the ETDRS fields was negative and highly significant: r = 0.96 (95% confidence interval 0.99 to 0.02). Conclusion: The distribution of leakage correlates inversely and statistically significantly with the thickness of the NFL, suggesting astrocytes in the NFL play a pivotal role in preventing damage to the iBRB and subsequent evolution of microaneurysms in DME. Moreover, fluid extravasation due to damage to the iBRB is expressed earlier in the temporal than in the other three fields.(VLID)484321