Training and clinical testing of artificial intelligence derived right atrial cardiovascular magnetic resonance measurements

BACKGROUND: Right atrial (RA) area predicts mortality in patients with pulmonary hypertension, and is recommended by the European Society of Cardiology/European Respiratory Society pulmonary hypertension guidelines. The advent of deep learning may allow more reliable measurement of RA areas to improve clinical assessments. The aim of this study was to automate cardiovascular magnetic resonance (CMR) RA area measurements and evaluate the clinical utility by assessing repeatability, correlation with invasive haemodynamics and prognostic value. METHODS: A deep learning RA area CMR contouring model was trained in a multicentre cohort of 365 patients with pulmonary hypertension, left ventricular pathology and healthy subjects. Inter-study repeatability (intraclass correlation coefficient (ICC)) and agreement of contours (DICE similarity coefficient (DSC)) were assessed in a prospective cohort (n = 36). Clinical testing and mortality prediction was performed in n = 400 patients that were not used in the training nor prospective cohort, and the correlation of automatic and manual RA measurements with invasive haemodynamics assessed in n = 212/400. Radiologist quality control (QC) was performed in the ASPIRE registry, n = 3795 patients. The primary QC observer evaluated all the segmentations and recorded them as satisfactory, suboptimal or failure. A second QC observer analysed a random subcohort to assess QC agreement (n = 1018). RESULTS: All deep learning RA measurements showed higher interstudy repeatability (ICC 0.91 to 0.95) compared to manual RA measurements (1st observer ICC 0.82 to 0.88, 2nd observer ICC 0.88 to 0.91). DSC showed high agreement comparing automatic artificial intelligence and manual CMR readers. Maximal RA area mean and standard deviation (SD) DSC metric for observer 1 vs observer 2, automatic measurements vs observer 1 and automatic measurements vs observer 2 is 92.4 ± 3.5 cm2, 91.2 ± 4.5 cm2 and 93.2 ± 3.2 cm2, respectively. Minimal RA area mean and SD DSC metric for observer 1 vs observer 2, automatic measurements vs observer 1 and automatic measurements vs observer 2 was 89.8 ± 3.9 cm2, 87.0 ± 5.8 cm2 and 91.8 ± 4.8 cm2. Automatic RA area measurements all showed moderate correlation with invasive parameters (r = 0.45 to 0.66), manual (r = 0.36 to 0.57). Maximal RA area could accurately predict elevated mean RA pressure low and high-risk thresholds (area under the receiver operating characteristic curve artificial intelligence = 0.82/0.87 vs manual = 0.78/0.83), and predicted mortality similar to manual measurements, both p < 0.01. In the QC evaluation, artificial intelligence segmentations were suboptimal at 108/3795 and a low failure rate of 16/3795. In a subcohort (n = 1018), agreement by two QC observers was excellent, kappa 0.84. CONCLUSION: Automatic artificial intelligence CMR derived RA size and function are accurate, have excellent repeatability, moderate associations with invasive haemodynamics and predict mortality

Accuracy and cost-effectiveness of dynamic contrast-enhanced CT in the characterisation of solitary pulmonary nodules — the SPUtNIk study

$\textbf{Introduction:}$ Solitary pulmonary nodules (SPNs) are common on CT. The most cost-effective investigation algorithm is still to be determined. Dynamic contrastenhanced CT (DCE-CT) is an established diagnostic test not widely available in the UK currently. $\textbf{Methods and analysis:}$ The SPUtNIk study will assess the diagnostic accuracy, clinical utility and cost-effectiveness of DCE-CT, alongside the current CT and 18-flurodeoxyglucose-positron emission tomography) ($^{18}$FDG-PET)-CT nodule characterisation strategies in the National Health Service (NHS). Image acquisition and data analysis for $^{18}$FDG-PET-CT and DCE-CT will follow a standardised protocol with central review of 10% to ensure quality assurance. Decision analytic modelling will assess the likely costs and health outcomes resulting from incorporation of DCE-CT into management strategies for patients with SPNs. $\textbf{Ethics and dissemination:}$ Approval has been granted by the South West Research Ethics Committee. Ethics reference number 12/SW/0206. The results of the trial will be presented at national and international meetings and published in an Health Technology Assessment (HTA) Monograph and in peer-reviewed journals.The trial is funded by the National Institute for Health Research HTA Programme (grant no: 09/22/117) and is being run by Southampton Clinical Trials Unit, directed by Professor Gareth Griffiths and part funded by Cancer Research UK. NRQ and RCR are part funded by the Cambridge Biomedical Research Centre and the Cancer Research Network: Eastern

Fully automatic cardiac four chamber and great vessel segmentation on CT pulmonary angiography using deep learning

Introduction: Computed tomography pulmonary angiography (CTPA) is an essential test in the work-up of suspected pulmonary vascular disease including pulmonary hypertension and pulmonary embolism. Cardiac and great vessel assessments on CTPA are based on visual assessment and manual measurements which are known to have poor reproducibility. The primary aim of this study was to develop an automated whole heart segmentation (four chamber and great vessels) model for CTPA. Methods: A nine structure semantic segmentation model of the heart and great vessels was developed using 200 patients (80/20/100 training/validation/internal testing) with testing in 20 external patients. Ground truth segmentations were performed by consultant cardiothoracic radiologists. Failure analysis was conducted in 1,333 patients with mixed pulmonary vascular disease. Segmentation was achieved using deep learning via a convolutional neural network. Volumetric imaging biomarkers were correlated with invasive haemodynamics in the test cohort. Results: Dice similarity coefficients (DSC) for segmented structures were in the range 0.58–0.93 for both the internal and external test cohorts. The left and right ventricle myocardium segmentations had lower DSC of 0.83 and 0.58 respectively while all other structures had DSC >0.89 in the internal test cohort and >0.87 in the external test cohort. Interobserver comparison found that the left and right ventricle myocardium segmentations showed the most variation between observers: mean DSC (range) of 0.795 (0.785–0.801) and 0.520 (0.482–0.542) respectively. Right ventricle myocardial volume had strong correlation with mean pulmonary artery pressure (Spearman's correlation coefficient = 0.7). The volume of segmented cardiac structures by deep learning had higher or equivalent correlation with invasive haemodynamics than by manual segmentations. The model demonstrated good generalisability to different vendors and hospitals with similar performance in the external test cohort. The failure rates in mixed pulmonary vascular disease were low (<3.9%) indicating good generalisability of the model to different diseases. Conclusion: Fully automated segmentation of the four cardiac chambers and great vessels has been achieved in CTPA with high accuracy and low rates of failure. DL volumetric biomarkers can potentially improve CTPA cardiac assessment and invasive haemodynamic prediction

A systematic review of right ventricular diastolic assessment by 4D flow CMR

Background. Four-dimensional flow cardiovascular magnetic resonance (4D flow CMR) is a noninvasive novel imaging technology that can be used to visualise and assess right ventricular function. The aim of this systematic review is to summarise the literature available on 4D flow CMR methods used to determine right ventricular diastolic function. Methods. A systematic review of current literature was carried out to ascertain what is known about right ventricular assessment by quantification of 4D flow CMR. Structured searches were carried out on Medline and EMBASE in December 2018. PG and NB screened the titles and abstracts for relevance. Results. Of the 20 articles screened, 5 studies met eligibility for systematic review. After a further search on pubmed 1 more relevant article was found and added to the review. Conclusions. These proposed methods using 4D flow CMR can quantify right ventricular diastolic assessment. The evidence gathered is mainly observational, featuring single-centred studies. Larger, multicentre studies are required to validate the proposed techniques, evaluate reproducibility, and investigate the clinical applicability that 4D flow CMR offers compared to standard practices. PROSPERO registration number is CRD42019121492

Impact of solitary pulmonary nodule size on qualitative and quantitative assessment using 18F-fluorodeoxyglucose PET/CT: the SPUTNIK trial

Purpose: To compare qualitative and semi-quantitative PET/CT criteria, and the impact of nodule size on the diagnosis of solitary pulmonary nodules in a prospective multicentre trial. / Methods: Patients with an SPN on CT ≥ 8 and ≤ 30 mm were recruited to the SPUTNIK trial at 16 sites accredited by the UK PET Core Lab. Qualitative assessment used a five-point ordinal PET-grade compared to the mediastinal blood pool, and a combined PET/CT grade using the CT features. Semi-quantitative measures included SUVmax of the nodule, and as an uptake ratio to the mediastinal blood pool (SURBLOOD) or liver (SURLIVER). The endpoints were diagnosis of lung cancer via biopsy/histology or completion of 2-year follow-up. Impact of nodule size was analysed by comparison between nodule size tertiles. / Results: Three hundred fifty-five participants completed PET/CT and 2-year follow-up, with 59% (209/355) malignant nodules. The AUCs of the three techniques were SUVmax 0.87 (95% CI 0.83;0.91); SURBLOOD 0.87 (95% CI 0.83; 0.91, p = 0.30 versus SUVmax); and SURLIVER 0.87 (95% CI 0.83; 0.91, p = 0.09 vs. SUVmax). The AUCs for all techniques remained stable across size tertiles (p > 0.1 for difference), although the optimal diagnostic threshold varied by size. For nodules  16 mm, an SUVmax ≥ 3.6 or visual PET uptake greater than the mediastinum was the most accurate. / Conclusion: In this multicentre trial, SUVmax was the most accurate technique for the diagnosis of solitary pulmonary nodules. Diagnostic thresholds should be altered according to nodule size. / Trial registration: ISRCTN - ISRCTN30784948. ClinicalTrials.gov - NCT0201306

Establishing minimally important differences for cardiac MRI end-points in pulmonary arterial hypertension

Background Cardiac magnetic resonance (CMR) is the gold standard technique to assess biventricular volumes and function, and is increasingly being considered as an end-point in clinical studies. Currently, with the exception of right ventricular (RV) stroke volume and RV end-diastolic volume, there is only limited data on minimally important differences (MIDs) reported for CMR metrics. Our study aimed to identify MIDs for CMR metrics based on US Food and Drug Administration recommendations for a clinical outcome measure that should reflect how a patient “feels, functions or survives”. Methods Consecutive treatment-naïve patients with pulmonary arterial hypertension (PAH) between 2010 and 2022 who had two CMR scans (at baseline prior to treatment and 12 months following treatment) were identified from the ASPIRE registry. All patients were followed up for 1 additional year after the second scan. For both scans, cardiac measurements were obtained from a validated fully automated segmentation tool. The MID in CMR metrics was determined using two distribution-based (0.5sd and minimal detectable change) and two anchor-based (change difference and generalised linear model regression) methods benchmarked to how a patient “feels” (emPHasis-10 quality of life questionnaire), “functions” (incremental shuttle walk test) or “survives” for 1-year mortality to changes in CMR measurements. Results 254 patients with PAH were included (mean±sd age 53±16 years, 79% female and 66% categorised as intermediate risk based on the 2022 European Society of Cardiology/European Respiratory Society risk score). We identified a 5% absolute increase in RV ejection fraction and a 17 mL decrease in RV end-diastolic or end-systolic volumes as the MIDs for improvement. Conversely, a 5% decrease in RV ejection fraction and a 10 mL increase in RV volumes were associated with worsening. Conclusions This study establishes clinically relevant CMR MIDs for how a patient “feels, functions or survives” in response to PAH treatment. These findings provide further support for the use of CMR as a clinically relevant clinical outcome measure and will aid trial size calculations for studies using CMR

Dynamic contrast-enhanced CT compared with positron emission tomography CT to characterise solitary pulmonary nodules : the SPUtNIk diagnostic accuracy study and economic modelling

Background Current pathways recommend positron emission tomography–computerised tomography for the characterisation of solitary pulmonary nodules. Dynamic contrast-enhanced computerised tomography may be a more cost-effective approach. Objectives To determine the diagnostic performances of dynamic contrast-enhanced computerised tomography and positron emission tomography–computerised tomography in the NHS for solitary pulmonary nodules. Systematic reviews and a health economic evaluation contributed to the decision-analytic modelling to assess the likely costs and health outcomes resulting from incorporation of dynamic contrast-enhanced computerised tomography into management strategies. Design Multicentre comparative accuracy trial. Setting Secondary or tertiary outpatient settings at 16 hospitals in the UK. Participants Participants with solitary pulmonary nodules of ≥ 8 mm and of ≤ 30 mm in size with no malignancy in the previous 2 years were included. Interventions Baseline positron emission tomography–computerised tomography and dynamic contrast-enhanced computer tomography with 2 years’ follow-up. Main outcome measures Primary outcome measures were sensitivity, specificity and diagnostic accuracy for positron emission tomography–computerised tomography and dynamic contrast-enhanced computerised tomography. Incremental cost-effectiveness ratios compared management strategies that used dynamic contrast-enhanced computerised tomography with management strategies that did not use dynamic contrast-enhanced computerised tomography. Results A total of 380 patients were recruited (median age 69 years). Of 312 patients with matched dynamic contrast-enhanced computer tomography and positron emission tomography–computerised tomography examinations, 191 (61%) were cancer patients. The sensitivity, specificity and diagnostic accuracy for positron emission tomography–computerised tomography and dynamic contrast-enhanced computer tomography were 72.8% (95% confidence interval 66.1% to 78.6%), 81.8% (95% confidence interval 74.0% to 87.7%), 76.3% (95% confidence interval 71.3% to 80.7%) and 95.3% (95% confidence interval 91.3% to 97.5%), 29.8% (95% confidence interval 22.3% to 38.4%) and 69.9% (95% confidence interval 64.6% to 74.7%), respectively. Exploratory modelling showed that maximum standardised uptake values had the best diagnostic accuracy, with an area under the curve of 0.87, which increased to 0.90 if combined with dynamic contrast-enhanced computerised tomography peak enhancement. The economic analysis showed that, over 24 months, dynamic contrast-enhanced computerised tomography was less costly (£3305, 95% confidence interval £2952 to £3746) than positron emission tomography–computerised tomography (£4013, 95% confidence interval £3673 to £4498) or a strategy combining the two tests (£4058, 95% confidence interval £3702 to £4547). Positron emission tomography–computerised tomography led to more patients with malignant nodules being correctly managed, 0.44 on average (95% confidence interval 0.39 to 0.49), compared with 0.40 (95% confidence interval 0.35 to 0.45); using both tests further increased this (0.47, 95% confidence interval 0.42 to 0.51). Limitations The high prevalence of malignancy in nodules observed in this trial, compared with that observed in nodules identified within screening programmes, limits the generalisation of the current results to nodules identified by screening. Conclusions Findings from this research indicate that positron emission tomography–computerised tomography is more accurate than dynamic contrast-enhanced computerised tomography for the characterisation of solitary pulmonary nodules. A combination of maximum standardised uptake value and peak enhancement had the highest accuracy with a small increase in costs. Findings from this research also indicate that a combined positron emission tomography–dynamic contrast-enhanced computerised tomography approach with a slightly higher willingness to pay to avoid missing small cancers or to avoid a ‘watch and wait’ policy may be an approach to consider. Future work Integration of the dynamic contrast-enhanced component into the positron emission tomography–computerised tomography examination and the feasibility of dynamic contrast-enhanced computerised tomography at lung screening for the characterisation of solitary pulmonary nodules should be explored, together with a lower radiation dose protocol. Study registration This study is registered as PROSPERO CRD42018112215 and CRD42019124299, and the trial is registered as ISRCTN30784948 and ClinicalTrials.gov NCT02013063

Comparative Accuracy and Cost-Effectiveness of Dynamic Contrast Enhanced Computed Tomography and Positron Emission Tomography in the Characterisation of Solitary Pulmonary Nodules

Abstract Introduction: Dynamic contrast-enhanced computed tomography (DCE-CT) and Positron Emission Tomography/Computed Tomography (PET/CT) have a high reported accuracy for the diagnosis of malignancy in solitary pulmonary nodules. The aim of this study was to compare the accuracy and cost-effectiveness of these. Methods: In this prospective multicentre trial, 380 participants with a solitary pulmonary nodule (8-30mm) and no recent history of malignancy underwent DCE-CT and PET/CT. All patients underwent either biopsy with histological diagnosis or completed CT follow-up. Primary outcome measures were sensitivity, specificity, and overall diagnostic accuracy for PET/CT and DCE-CT. Costs and cost-effectiveness were estimated from a healthcare provider perspective using a decision-model. Results: 312 participants (47% female, 68.1±9.0 years) completed the study, with 61% rate of malignancy at 2 years. The sensitivity, specificity, positive predictive value and negative predictive values for DCE-CT were 95.3% [95% CI 91.3;97.5], 29.8% [95% CI 22.3;38.4], 68.2% [95% CI 62.4%;73.5%] and 80.0% [95% CI 66.2;89.1] respectively, and for PET/CT were 79.1% [95% CI 72.7;84.2], 81.8% [95% CI 74.0;87.7], 87.3%[95% CI 81.5;91.5) and 71·2% [95% CI 63.2;78.1]. The area under the receiver operator characteristic curve (AUROC) for DCE-CT and PET/CT was 0.62 [95%CI 0.58;0.67] and 0.80 [95%CI 0.76;0.85] respectively (p<0.001). Combined results significantly increased diagnostic accuracy over PET/CT alone (AUROC=0.90 [95%CI 0.86;0.93], p<0.001). DCE-CT was preferred when the willingness to pay per incremental cost per correctly treated malignancy was below £9000. Above £15500 a combined approach was preferred. Conclusions: PET/CT has a superior diagnostic accuracy to DCE-CT for the diagnosis of solitary pulmonary nodules. Combining both techniques improves the diagnostic accuracy over either test alone and could be cost-effective. (Clinical trials.gov - NCT02013063)

Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR

Introduction: Cardiac magnetic resonance (CMR) is of diagnostic and prognostic value in a range of cardiopulmonary conditions. Current methods for evaluating CMR studies are laborious and time-consuming, contributing to delays for patients. As the demand for CMR increases, there is a growing need to automate this process. The application of artificial intelligence (AI) to CMR is promising, but the evaluation of these tools in clinical practice has been limited. This study assessed the clinical viability of an automatic tool for measuring cardiac volumes on CMR. Methods: Consecutive patients who underwent CMR for any indication between January 2022 and October 2022 at a single tertiary centre were included prospectively. For each case, short-axis CMR images were segmented by the AI tool and manually to yield volume, mass and ejection fraction measurements for both ventricles. Automated and manual measurements were compared for agreement and the quality of the automated contours was assessed visually by cardiac radiologists. Results: 462 CMR studies were included. No statistically significant difference was demonstrated between any automated and manual measurements (p > 0.05; independent T-test). Intraclass correlation coefficient and Bland-Altman analysis showed excellent agreement across all metrics (ICC > 0.85). The automated contours were evaluated visually in 251 cases, with agreement or minor disagreement in 229 cases (91.2%) and failed segmentation in only a single case (0.4%). The AI tool was able to provide automated contours in under 90 s. Conclusions: Automated segmentation of both ventricles on CMR by an automatic tool shows excellent agreement with manual segmentation performed by CMR experts in a retrospective real-world clinical cohort. Implementation of the tool could improve the efficiency of CMR reporting and reduce delays between imaging and diagnosis