Building trust in real-world data: lessons from INSIGHT, the UK's health data research hub for eye health and oculomics

PURPOSE OF REVIEW: In this review, we consider the challenges of creating a trusted resource for real-world data in ophthalmology, based on our experience of establishing INSIGHT, the UK's Health Data Research Hub for Eye Health and Oculomics. RECENT FINDINGS: The INSIGHT Health Data Research Hub maximizes the benefits and impact of historical, patient-level UK National Health Service (NHS) electronic health record data, including images, through making it research-ready including curation and anonymisation. It is built around a shared 'north star' of enabling research for patient benefit. INSIGHT has worked to establish patient and public trust in the concept and delivery of INSIGHT, with efficient and robust governance processes that support safe and secure access to data for researchers. By linking to systemic data, there is an opportunity for discovery of novel ophthalmic biomarkers of systemic diseases ('oculomics'). Datasets that provide a representation of the whole population are an important tool to address the increasingly recognized threat of health data poverty. SUMMARY: Enabling efficient, safe access to routinely collected clinical data is a substantial undertaking, especially when this includes imaging modalities, but provides an exceptional resource for research. Research and innovation built on inclusive real-world data is an important tool in ensuring that discoveries and technologies of the future may not only favour selected groups, but also work for all patients

AI as a Medical Device for Ophthalmic Imaging in Europe, Australia, and the United States:Protocol for a Systematic Scoping Review of Regulated Devices

BACKGROUND: Artificial intelligence as a medical device (AIaMD) has the potential to transform many aspects of ophthalmic care, such as improving accuracy and speed of diagnosis, addressing capacity issues in high-volume areas such as screening, and detecting novel biomarkers of systemic disease in the eye (oculomics). In order to ensure that such tools are safe for the target population and achieve their intended purpose, it is important that these AIaMD have adequate clinical evaluation to support any regulatory decision. Currently, the evidential requirements for regulatory approval are less clear for AIaMD compared to more established interventions such as drugs or medical devices. There is therefore value in understanding the level of evidence that underpins AIaMD currently on the market, as a step toward identifying what the best practices might be in this area. In this systematic scoping review, we will focus on AIaMD that contributes to clinical decision-making (relating to screening, diagnosis, prognosis, and treatment) in the context of ophthalmic imaging.OBJECTIVE: This study aims to identify regulator-approved AIaMD for ophthalmic imaging in Europe, Australia, and the United States; report the characteristics of these devices and their regulatory approvals; and report the available evidence underpinning these AIaMD.METHODS: The Food and Drug Administration (United States), the Australian Register of Therapeutic Goods (Australia), the Medicines and Healthcare products Regulatory Agency (United Kingdom), and the European Database on Medical Devices (European Union) regulatory databases will be searched for ophthalmic imaging AIaMD through a snowballing approach. PubMed and clinical trial registries will be systematically searched, and manufacturers will be directly contacted for studies investigating the effectiveness of eligible AIaMD. Preliminary regulatory database searches, evidence searches, screening, data extraction, and methodological quality assessment will be undertaken by 2 independent review authors and arbitrated by a third at each stage of the process.RESULTS: Preliminary searches were conducted in February 2023. Data extraction, data synthesis, and assessment of methodological quality commenced in October 2023. The review is on track to be completed and submitted for peer review by April 2024.CONCLUSIONS: This systematic review will provide greater clarity on ophthalmic imaging AIaMD that have achieved regulatory approval as well as the evidence that underpins them. This should help adopters understand the range of tools available and whether they can be safely incorporated into their clinical workflow, and it should also support developers in navigating regulatory approval more efficiently.INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/52602.</p

Concordance of randomised controlled trials for artificial intelligence interventions with the CONSORT-AI reporting guidelines

The Consolidated Standards of Reporting Trials extension for Artificial Intelligence interventions (CONSORT-AI) was published in September 2020. Since its publication, several randomised controlled trials (RCTs) of AI interventions have been published but their completeness and transparency of reporting is unknown. This systematic review assesses the completeness of reporting of AI RCTs following publication of CONSORT-AI and provides a comprehensive summary of RCTs published in recent years. 65 RCTs were identified, mostly conducted in China (37%) and USA (18%). Median concordance with CONSORT-AI reporting was 90% (IQR 77–94%), although only 10 RCTs explicitly reported its use. Several items were consistently under-reported, including algorithm version, accessibility of the AI intervention or code, and references to a study protocol. Only 3 of 52 included journals explicitly endorsed or mandated CONSORT-AI. Despite a generally high concordance amongst recent AI RCTs, some AI-specific considerations remain systematically poorly reported. Further encouragement of CONSORT-AI adoption by journals and funders may enable more complete adoption of the full CONSORT-AI guidelines

Curr Opin Ophthalmol

The application of artificial intelligence (AI) technologies in screening and diagnosing retinal diseases may play an important role in telemedicine and has potential to shape modern healthcare ecosystems, including within ophthalmology. In this article, we examine the latest publications relevant to AI in retinal disease and discuss the currently available algorithms. We summarize four key requirements underlining the successful application of AI algorithms in real-world practice: processing massive data; practicability of an AI model in ophthalmology; policy compliance and the regulatory environment; and balancing profit and cost when developing and maintaining AI models. The Vision Academy recognizes the advantages and disadvantages of AI-based technologies and gives insightful recommendations for future directions

Measuring Inflammation in the Vitreous and Retina: A Narrative Review.

Uveitis consists of a group of syndromes characterised by intraocular inflammation, accounting for up to 15% of visual loss in the western world and 10% worldwide. Assessment of intraocular inflammation has been limited to clinician-dependent, subjective grading. Developments in imaging technology, such as optical coherence tomography (OCT), have enabled the development of objective, quantitative measures of inflammatory activity. Important quantitative metrics including central macular thickness and vitreous signal intensity allow longitudinal monitoring of disease activity and can be used in conjunction with other imaging modalities enabling holistic assessment of ocular inflammation. Ongoing work into the validation of instrument-based measures alongside development of core outcome sets is crucial for standardisation of clinical trial endpoints and developing guidance for quantitative multi-modal imaging approaches. This review outlines methods of grading inflammation in the vitreous and retina, with a focus on the use of OCT as an objective measure of disease activity

Measuring Inflammation in the Vitreous and Retina: A Narrative Review

Uveitis consists of a group of syndromes characterised by intraocular inflammation, accounting for up to 15% of visual loss in the western world and 10% worldwide. Assessment of intraocular inflammation has been limited to clinician-dependent, subjective grading. Developments in imaging technology, such as optical coherence tomography (OCT), have enabled the development of objective, quantitative measures of inflammatory activity. Important quantitative metrics including central macular thickness and vitreous signal intensity allow longitudinal monitoring of disease activity and can be used in conjunction with other imaging modalities enabling holistic assessment of ocular inflammation. Ongoing work into the validation of instrument-based measures alongside development of core outcome sets is crucial for standardisation of clinical trial endpoints and developing guidance for quantitative multi-modal imaging approaches. This review outlines methods of grading inflammation in the vitreous and retina, with a focus on the use of OCT as an objective measure of disease activity

Stability of OCT and OCTA in the Intensive Therapy Unit Setting.

To assess the stability of retinal structure and blood flow measures over time and in different clinical settings using portable optical coherence tomography angiography (OCTA) as a potential biomarker of central perfusion in critical illness, 18 oesophagectomy patients completed retinal structure and blood flow measurements by portable OCT and OCTA in the eye clinic and intensive therapy unit (ITU) across three timepoints: (1) pre-operation in a clinic setting; (2) 24-48 h post-operation during ITU admission; and (3) seven days post-operation, if the patient was still admitted. Blood flow and macular structural measures were stable between the examination settings, with no consistent variation between pre- and post-operation scans, while retinal nerve fibre layer thickness increased in the post-operative scans (+2.31 µm, = 0.001). Foveal avascular zone (FAZ) measurements were the most stable, with an intraclass correlation coefficient of up to 0.92 for right eye FAZ area. Blood flow and structural measures were lower in left eyes than right eyes. Retinal blood flow assessed in patients before and during an ITU stay using portable OCTA showed no systematic differences between the clinical settings. The stability of retinal blood flow measures suggests the potential for portable OCTA to provide clinically useful measures in ITU patients

Instrument-based Tests for Measuring Anterior Chamber Cells in Uveitis: A Systematic Review.

: New instrument-based techniques for anterior chamber (AC) cell counting can offer automation and objectivity above clinician assessment. This review aims to identify such instruments and its correlation with clinician estimates. : Using standard systematic review methodology, we identified and tabulated the outcomes of studies reporting reliability and correlation between instrument-based measurements and clinician AC cell grading. : From 3470 studies, 6 reported correlation between an instrument-based AC cell count to clinician grading. The two instruments were optical coherence tomography (OCT) and laser flare-cell photometry (LFCP). Correlation between clinician grading and LFCP was 0.66-0.87 and 0.06-0.97 between clinician grading and OCT. OCT volume scans demonstrated correlation between 0.75 and 0.78. Line scans in the middle AC demonstrated higher correlation (0.73-0.97) than in the inferior AC (0.06-0.56). : AC cell count by OCT and LFP can achieve high levels of correlation with clinician grading, whilst offering additional advantages of speed, automation, and objectivity

A Datasheet for the INSIGHT Birmingham, Solihull, and Black Country Diabetic Retinopathy Screening Dataset

Purpose: Diabetic retinopathy (DR) is the most common microvascular complication associated with diabetes mellitus (DM), affecting approximately 40% of this patient population. Early detection of DR is vital to ensure monitoring of disease progression and prompt sight saving treatments as required. This article describes the data contained within the INSIGHT Birmingham, Solihull, and Black Country Diabetic Retinopathy Dataset. Design: Dataset descriptor for routinely collected eye screening data. Participants: All diabetic patients aged 12 years and older, attending annual digital retinal photography-based screening within the Birmingham, Solihull, and Black Country Eye Screening Programme. Methods: The INSIGHT Health Data Research Hub for Eye Health is a National Health Service (NHS)–led ophthalmic bioresource that provides researchers with safe access to anonymized, routinely collected data from contributing NHS hospitals to advance research for patient benefit. This report describes the INSIGHT Birmingham, Solihull, and Black Country DR Screening Dataset, a dataset of anonymized images and linked screening data derived from the United Kingdom’s largest regional DR screening program. Main Outcome Measures: This dataset consists of routinely collected data from the eye screening program. The data primarily include retinal photographs with the associated DR grading data. Additional data such as corresponding demographic details, information regarding patients’ diabetic status, and visual acuity data are also available. Further details regarding available data points are available in the supplementary information, in addition to the INSIGHT webpage included below. Results: At the time point of this analysis (December 31, 2019), the dataset comprised 6 202 161 images from 246 180 patients, with a dataset inception date of January 1, 2007. The dataset includes 1 360 547 grading episodes between R0M0 and R3M1. Conclusions: This dataset descriptor article summarizes the content of the dataset, how it has been curated, and what its potential uses are. Data are available through a structured application process for research studies that support discovery, clinical evidence analyses, and innovation in artificial intelligence technologies for patient benefit. Further information regarding the data repository and contact details can be found at https://www.insight.hdrhub.org/. Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article