Five critical quality criteria for artificial intelligence-based prediction models

To raise the quality of clinical artificial intelligence (AI) prediction modelling studies in the cardiovascular health domain and thereby improve their impact and relevancy, the editors for digital health, innovation, and quality standards of the European Heart Journal propose five minimal quality criteria for AI-based prediction model development and validation studies: complete reporting, carefully defined intended use of the model, rigorous validation, large enough sample size, and openness of code and software

Prediction models for diagnosis and prognosis of covid-19: : systematic review and critical appraisal

Readers’ note This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity. Funding: LW, BVC, LH, and MDV acknowledge specific funding for this work from Internal Funds KU Leuven, KOOR, and the COVID-19 Fund. LW is a postdoctoral fellow of Research Foundation-Flanders (FWO) and receives support from ZonMw (grant 10430012010001). BVC received support from FWO (grant G0B4716N) and Internal Funds KU Leuven (grant C24/15/037). TPAD acknowledges financial support from the Netherlands Organisation for Health Research and Development (grant 91617050). VMTdJ was supported by the European Union Horizon 2020 Research and Innovation Programme under ReCoDID grant agreement 825746. KGMM and JAAD acknowledge financial support from Cochrane Collaboration (SMF 2018). KIES is funded by the National Institute for Health Research (NIHR) School for Primary Care Research. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care. GSC was supported by the NIHR Biomedical Research Centre, Oxford, and Cancer Research UK (programme grant C49297/A27294). JM was supported by the Cancer Research UK (programme grant C49297/A27294). PD was supported by the NIHR Biomedical Research Centre, Oxford. MOH is supported by the National Heart, Lung, and Blood Institute of the United States National Institutes of Health (grant R00 HL141678). ICCvDH and BCTvB received funding from Euregio Meuse-Rhine (grant Covid Data Platform (coDaP) interref EMR187). The funders played no role in study design, data collection, data analysis, data interpretation, or reporting.Peer reviewedPublisher PD

Cardiovascular vulnerability predicts hospitalisation in primary care clinically suspected and confirmed COVID-19 patients: A model development and validation study

Objectives Cardiovascular conditions were shown to be predictive of clinical deterioration in hospitalised patients with coronavirus disease 2019 (COVID-19). Whether this also holds for outpatients managed in primary care is yet unknown. The aim of this study was to determine the incremental value of cardiovascular vulnerability in predicting the risk of hospital referral in primary care COVID-19 outpatients. Design Analysis of anonymised routine care data extracted from electronic medical records from three large Dutch primary care registries. Setting Primary care. Participants Consecutive adult patients seen in primary care for COVID-19 symptoms in the ‘first wave’ of COVID-19 infections (March 1 2020 to June 1 2020) and in the ‘second wave’ (June 1 2020 to April 15 2021) in the Netherlands. Outcome measures A multivariable logistic regression model was fitted to predict hospital referral within 90 days after first COVID-19 consultation in primary care. Data from the ‘first wave’ was used for derivation (n = 5,475 patients). Age, sex, the interaction between age and sex, and the number of cardiovascular conditions and/or diabetes (0, 1, or ≥2) were pre-specified as candidate predictors. This full model was (i) compared to a simple model including only age and sex and its interaction, and (ii) externally validated in COVID-19 patients during the ‘second wave’ (n = 16,693). Results The full model performed better than the simple model (likelihood ratio test p<0.001). Older male patients with multiple cardiovascular conditions and/or diabetes had the highest predicted risk of hospital referral, reaching risks above 15–20%, whereas on average this risk was 5.1%. The temporally validated c-statistic was 0.747 (95%CI 0.729–0.764) and the model showed good calibration upon validation. Conclusions For patients with COVID-19 symptoms managed in primary care, the risk of hospital referral was on average 5.1%. Older, male and cardiovascular vulnerable COVID-19 patients are more at risk for hospital referral

Automated CT quantification methods for the assessment of interstitial lung disease in collagen vascular diseases : A systematic review

Interstitial lung disease (ILD) is highly prevalent in collagen vascular diseases and reduction of ILD is an important therapeutic target. To that end, reliable quantification of pulmonary disease severity is of great significance. This study systematically reviewed the literature on automated computed tomography (CT) quantification methods for assessing ILD in collagen vascular diseases. PRISMA-DTA guidelines for systematic reviews were used and 19 original research articles up to January 2018 were included based on a MEDLINE/Pubmed and Embase search. Quantitative CT methods were categorized as histogram assessment (12 studies) or pattern/texture recognition (7 studies). R 2 for correlation with visual ILD scoring ranged from 0.143 (p < 0.01) to 0.687 (p < 0.0001), for FVC from 0.048 (p < 0.0001) to 0.504 (p < 0.0001) and for DLCO from 0.015 (p = 0.61) to 0.449 (p < 0.0001). Automated CT methods are independent of reader's expertise and are a promising tool in the quantification of ILD in collagen vascular disease patients

Clinical prediction models for mortality in patients with covid-19:external validation and individual participant data meta-analysis

OBJECTIVE: To externally validate various prognostic models and scoring rules for predicting short term mortality in patients admitted to hospital for covid-19. DESIGN: Two stage individual participant data meta-analysis. SETTING: Secondary and tertiary care. PARTICIPANTS: 46 914 patients across 18 countries, admitted to a hospital with polymerase chain reaction confirmed covid-19 from November 2019 to April 2021. DATA SOURCES: Multiple (clustered) cohorts in Brazil, Belgium, China, Czech Republic, Egypt, France, Iran, Israel, Italy, Mexico, Netherlands, Portugal, Russia, Saudi Arabia, Spain, Sweden, United Kingdom, and United States previously identified by a living systematic review of covid-19 prediction models published in The BMJ, and through PROSPERO, reference checking, and expert knowledge. MODEL SELECTION AND ELIGIBILITY CRITERIA: Prognostic models identified by the living systematic review and through contacting experts. A priori models were excluded that had a high risk of bias in the participant domain of PROBAST (prediction model study risk of bias assessment tool) or for which the applicability was deemed poor. METHODS: Eight prognostic models with diverse predictors were identified and validated. A two stage individual participant data meta-analysis was performed of the estimated model concordance (C) statistic, calibration slope, calibration-in-the-large, and observed to expected ratio (O:E) across the included clusters. MAIN OUTCOME MEASURES: 30 day mortality or in-hospital mortality. RESULTS: Datasets included 27 clusters from 18 different countries and contained data on 46 914patients. The pooled estimates ranged from 0.67 to 0.80 (C statistic), 0.22 to 1.22 (calibration slope), and 0.18 to 2.59 (O:E ratio) and were prone to substantial between study heterogeneity. The 4C Mortality Score by Knight et al (pooled C statistic 0.80, 95% confidence interval 0.75 to 0.84, 95% prediction interval 0.72 to 0.86) and clinical model by Wang et al (0.77, 0.73 to 0.80, 0.63 to 0.87) had the highest discriminative ability. On average, 29% fewer deaths were observed than predicted by the 4C Mortality Score (pooled O:E 0.71, 95% confidence interval 0.45 to 1.11, 95% prediction interval 0.21 to 2.39), 35% fewer than predicted by the Wang clinical model (0.65, 0.52 to 0.82, 0.23 to 1.89), and 4% fewer than predicted by Xie et al’s model (0.96, 0.59 to 1.55, 0.21 to 4.28). CONCLUSION: The prognostic value of the included models varied greatly between the data sources. Although the Knight 4C Mortality Score and Wang clinical model appeared most promising, recalibration (intercept and slope updates) is needed before implementation in routine care