How should studies using AI be reported? Lessons from a systematic review in cardiac MRI

Recent years have seen a dramatic increase in studies presenting artificial intelligence (AI) tools for cardiac imaging. Amongst these are AI tools that undertake segmentation of structures on cardiac MRI (CMR), an essential step in obtaining clinically relevant functional information. The quality of reporting of these studies carries significant implications for advancement of the field and the translation of AI tools to clinical practice. We recently undertook a systematic review to evaluate the quality of reporting of studies presenting automated approaches to segmentation in cardiac MRI (Alabed et al. 2022 Quality of reporting in AI cardiac MRI segmentation studies—a systematic review and recommendations for future studies. Frontiers in Cardiovascular Medicine 9:956811). 209 studies were assessed for compliance with the Checklist for AI in Medical Imaging (CLAIM), a framework for reporting. We found variable—and sometimes poor—quality of reporting and identified significant and frequently missing information in publications. Compliance with CLAIM was high for descriptions of models (100%, IQR 80%–100%), but lower than expected for descriptions of study design (71%, IQR 63–86%), datasets used in training and testing (63%, IQR 50%–67%) and model performance (60%, IQR 50%–70%). Here, we present a summary of our key findings, aimed at general readers who may not be experts in AI, and use them as a framework to discuss the factors determining quality of reporting, making recommendations for improving the reporting of research in this field. We aim to assist researchers in presenting their work and readers in their appraisal of evidence. Finally, we emphasise the need for close scrutiny of studies presenting AI tools, even in the face of the excitement surrounding AI in cardiac imaging

The reproducibility of manual RV/LV ratio measurement on CT pulmonary angiography

Objectives: Right ventricular (RV) dysfunction carries elevated risk in acute pulmonary embolism (PE). An increased ratio between the size of the right and left ventricles (RV/LV ratio) is a biomarker of RV dysfunction. This study evaluated the reproducibility of RV/LV ratio measurement on CT pulmonary angiography (CTPA). Methods: 20 inpatient CTPA scans performed to assess for acute PE were retrospectively identified from a tertiary UK centre. Each scan was evaluated by 14 radiologists who provided a qualitative overall opinion on the presence of RV dysfunction and measured the RV/LV ratio. Using a threshold of 1.0, the RV/LV ratio measurements were classified as positive (≥1.0) or negative (<1.0) for RV dysfunction. Interobserver agreement was quantified using the Fleiss κ and intraclass correlation coefficient (ICC). Results: Qualitative opinion of RV dysfunction showed weak agreement (κ = 0.42, 95% CI 0.37–0.46). The mean RV/LV ratio measurement for all cases was 1.28 ± 0.68 with significant variation between reporters (p < 0.001). Although agreement for RV/LV measurement was good (ICC = 0.83, 95% CI 0.73–0.91), categorisation of RV dysfunction according to RV/LV ratio measurements showed weak agreement (κ = 0.46, 95% CI 0.41–0.50). Conclusion: Both qualitative opinion and quantitative manual RV/LV ratio measurement show poor agreement for identifying RV dysfunction on CTPA. Advances in knowledge: Caution should be exerted if using manual RV/LV ratio measurements to inform clinical risk stratification and management decisions

Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging

Background Segmentation of cardiac structures is an important step in evaluation of the heart on imaging. There has been growing interest in how artificial intelligence (AI) methods—particularly deep learning (DL)—can be used to automate this process. Existing AI approaches to cardiac segmentation have mostly focused on cardiac MRI. This systematic review aimed to appraise the performance and quality of supervised DL tools for the segmentation of cardiac structures on CT. Methods Embase and Medline databases were searched to identify related studies from January 1, 2013 to December 4, 2023. Original research studies published in peer-reviewed journals after January 1, 2013 were eligible for inclusion if they presented supervised DL-based tools for the segmentation of cardiac structures and non-coronary great vessels on CT. The data extracted from eligible studies included information about cardiac structure(s) being segmented, study location, DL architectures and reported performance metrics such as the Dice similarity coefficient (DSC). The quality of the included studies was assessed using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Results 18 studies published after 2020 were included. The DSC scores median achieved for the most commonly segmented structures were left atrium (0.88, IQR 0.83–0.91), left ventricle (0.91, IQR 0.89–0.94), left ventricle myocardium (0.83, IQR 0.82–0.92), right atrium (0.88, IQR 0.83–0.90), right ventricle (0.91, IQR 0.85–0.92), and pulmonary artery (0.92, IQR 0.87–0.93). Compliance of studies with CLAIM was variable. In particular, only 58% of studies showed compliance with dataset description criteria and most of the studies did not test or validate their models on external data (81%). Conclusion Supervised DL has been applied to the segmentation of various cardiac structures on CT. Most showed similar performance as measured by DSC values. Existing studies have been limited by the size and nature of the training datasets, inconsistent descriptions of ground truth annotations and lack of testing in external data or clinical settings. Systematic Review Registration: [www.crd.york.ac.uk/prospero/], PROSPERO [CRD42023431113]

Challenges in conducting community-driven research created by differing ways of talking and thinking about science: a researcher&#x2019;s perspective

Increasingly, health scientists are becoming aware that research collaborations that include community partnerships can be an effective way to broaden the scope and enhance the impact of research aimed at improving public health. Such collaborations extend the reach of academic scientists by integrating a variety of perspectives and thus strengthening the applicability of the research. Communication challenges can arise, however, when attempting to address specific research questions in these collaborations. In particular, inconsistencies can exist between scientists and community members in the use and interpretation of words and other language features, particularly when conducting research with a biomedical component. Additional challenges arise from differing perceptions of the investigative process. There may be divergent perceptions about how research questions should and can be answered, and in expectations about requirements of research institutions and research timelines. From these differences, misunderstandings can occur about how the results will ultimately impact the community. These communication issues are particularly challenging when scientists and community members are from different ethnic and linguistic backgrounds that may widen the gap between ways of talking and thinking about science, further complicating the interactions and exchanges that are essential for effective joint research efforts. Community-driven research that aims to describe the burden of disease associated with Helicobacter pylori infection is currently underway in northern Aboriginal communities located in the Yukon and Northwest Territories, Canada, with the goal of identifying effective public health strategies for reducing health risks from this infection. This research links community representatives, faculty from various disciplines at the University of Alberta, as well as territorial health care practitioners and officials. This highly collaborative work will be used to illustrate, from a researcher&#x2019;s perspective, some of the challenges of conducting public health research in teams comprising members with varying backgrounds. The consequences of these challenges will be outlined, and potential solutions will be offered

The Vital Role of Social Workers in Community Partnerships: The Alliance for Gay, Lesbian, Bisexual, Transgender and Questioning Youth

The account of The Alliance for Gay, Lesbian, Bisexual, Transgender, and Questioning (GLBTQ) Youth formation offers a model for developing com- munity-based partnerships. Based in a major urban area, this university-community collaboration was spearheaded by social workers who were responsible for its original conceptualization, for generating community support, and for eventual staffing, administration, direct service provision, and program evaluation design. This article presents the strategic development and evolution of this community- based service partnership, highlighting the roles of schools of social work, academics, and social work students in concert with community funders, practitioners and youth, in responding to the needs of a vulnerable population

A systematic review of artificial intelligence tools for chronic pulmonary embolism on CT pulmonary angiography

Background: Chronic pulmonary embolism (PE) may result in pulmonary hypertension (CTEPH). Automated CT pulmonary angiography (CTPA) interpretation using artificial intelligence (AI) tools has the potential for improving diagnostic accuracy, reducing delays to diagnosis and yielding novel information of clinical value in CTEPH. This systematic review aimed to identify and appraise existing studies presenting AI tools for CTPA in the context of chronic PE and CTEPH. Methods: MEDLINE and EMBASE databases were searched on 11 September 2023. Journal publications presenting AI tools for CTPA in patients with chronic PE or CTEPH were eligible for inclusion. Information about model design, training and testing was extracted. Study quality was assessed using compliance with the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Results: Five studies were eligible for inclusion, all of which presented deep learning AI models to evaluate PE. First study evaluated the lung parenchymal changes in chronic PE and two studies used an AI model to classify PE, with none directly assessing the pulmonary arteries. In addition, a separate study developed a CNN tool to distinguish chronic PE using 2D maximum intensity projection reconstructions. While another study assessed a novel automated approach to quantify hypoperfusion to help in the severity assessment of CTEPH. While descriptions of model design and training were reliable, descriptions of the datasets used in training and testing were more inconsistent. Conclusion: In contrast to AI tools for evaluation of acute PE, there has been limited investigation of AI-based approaches to characterising chronic PE and CTEPH on CTPA. Existing studies are limited by inconsistent reporting of the data used to train and test their models. This systematic review highlights an area of potential expansion for the field of AI in medical image interpretation. There is limited knowledge of A systematic review of artificial intelligence tools for chronic pulmonary embolism in CT. This systematic review provides an assessment on research that examined deep learning algorithms in detecting CTEPH on CTPA images, the number of studies assessing the utility of deep learning on CTPA in CTEPH was unclear and should be highlighted

Evaluating the performance of artificial intelligence software for lung nodule detection on chest radiographs in a retrospective real-world UK population

Objectives Early identification of lung cancer on chest radiographs improves patient outcomes. Artificial intelligence (AI) tools may increase diagnostic accuracy and streamline this pathway. This study evaluated the performance of commercially available AI-based software trained to identify cancerous lung nodules on chest radiographs. Design This retrospective study included primary care chest radiographs acquired in a UK centre. The software evaluated each radiograph independently and outputs were compared with two reference standards: (1) the radiologist report and (2) the diagnosis of cancer by multidisciplinary team decision. Failure analysis was performed by interrogating the software marker locations on radiographs. Participants 5722 consecutive chest radiographs were included from 5592 patients (median age 59 years, 53.8% women, 1.6% prevalence of cancer). Results Compared with radiologist reports for nodule detection, the software demonstrated sensitivity 54.5% (95% CI 44.2% to 64.4%), specificity 83.2% (82.2% to 84.1%), positive predictive value (PPV) 5.5% (4.6% to 6.6%) and negative predictive value (NPV) 99.0% (98.8% to 99.2%). Compared with cancer diagnosis, the software demonstrated sensitivity 60.9% (50.1% to 70.9%), specificity 83.3% (82.3% to 84.2%), PPV 5.6% (4.8% to 6.6%) and NPV 99.2% (99.0% to 99.4%). Normal or variant anatomy was misidentified as an abnormality in 69.9% of the 943 false positive cases. Conclusions The software demonstrated considerable underperformance in this real-world patient cohort. Failure analysis suggested a lack of generalisability in the training and testing datasets as a potential factor. The low PPV carries the risk of over-investigation and limits the translation of the software to clinical practice. Our findings highlight the importance of training and testing software in representative datasets, with broader implications for the implementation of AI tools in imaging

Semi-automatic thresholding of RV trabeculation improves repeatability and diagnostic value in suspected pulmonary hypertension

Objectives: Right ventricle (RV) mass is an imaging biomarker of mean pulmonary artery pressure (MPAP) and pulmonary vascular resistance (PVR). Some methods of RV mass measurement on cardiac MRI (CMR) exclude RV trabeculation. This study assessed the reproducibility of measurement methods and evaluated whether the inclusion of trabeculation in RV mass affects diagnostic accuracy in suspected pulmonary hypertension (PH). Materials and methods: Two populations were enrolled prospectively. (i) A total of 144 patients with suspected PH who underwent CMR followed by right heart catheterization (RHC). Total RV mass (including trabeculation) and compacted RV mass (excluding trabeculation) were measured on the end-diastolic CMR images using both semi-automated pixel-intensity-based thresholding and manual contouring techniques. (ii) A total of 15 healthy volunteers and 15 patients with known PH. Interobserver agreement and scan-scan reproducibility were evaluated for RV mass measurements using the semi-automated thresholding and manual contouring techniques. Results: Total RV mass correlated more strongly with MPAP and PVR (r = 0.59 and 0.63) than compacted RV mass (r = 0.25 and 0.38). Using a diagnostic threshold of MPAP ≥ 25 mmHg, ROC analysis showed better performance for total RV mass (AUC 0.77 and 0.81) compared to compacted RV mass (AUC 0.61 and 0.66) when both parameters were indexed for LV mass. Semi-automated thresholding was twice as fast as manual contouring (p < 0.001). Conclusion: Using a semi-automated thresholding technique, inclusion of trabecular mass and indexing RV mass for LV mass (ventricular mass index), improves the diagnostic accuracy of CMR measurements in suspected PH

ESE Clinical Practice Guideline on functioning and nonfunctioning pituitary adenomas in pregnancy

Pregnancies are rare in women with pituitary adenomas, which may relate to hormone excess from secretory subtypes such as prolactinomas or corticotroph adenomas. Decreased fertility may also result from pituitary hormone deficiencies due to compression of the gland by large tumours and/or surgical or radiation treatment of the lesion. Counselling premenopausal women with pituitary adenomas about their chance of conceiving spontaneously or with assisted reproductive technology, and the optimal pre-conception treatment, should start at the time of initial diagnosis. The normal physiological changes during pregnancy need to be considered when interpreting endocrine tests in women with pituitary adenomas. Dose adjustments in hormone substitution therapies may be needed across the trimesters. When medical therapy is used for pituitary hormone excess, consideration should be given to the known efficacy and safety data specific to pregnant women for each therapeutic option. In healthy women, pituitary gland size increases during pregnancy. Since some pituitary adenomas also enlarge during pregnancy, there is a risk of visual impairment, especially in women with macroadenomas or tumours near the optic chiasm. Pituitary apoplexy represents a rare acute complication of adenomas requiring surveillance, with surgical intervention needed in some cases. This guideline describes the choice and timing of diagnostic tests and treatments from the pre-conception stage until after delivery, taking into account adenoma size, location and endocrine activity. In most cases, pregnant women with pituitary adenomas should be managed by a multidisciplinary team in a centre specialised in the treatment of such tumours

Venous thromboembolism in Cushing syndrome:results from an EuRRECa and Endo-ERN survey

Background: Patients with Cushing syndrome (CS) are at increased risk of venous thromboembolism (VTE). Objective: The aim was to evaluate the current management of new cases of CS with a focus on VTE and thromboprophylaxis. Design and methods: A survey was conducted within those that report in the electronic reporting tool (e-REC) of the European Registries for Rare Endocrine Conditions (EuRRECa) and the involved main thematic groups (MTG’s) of the European Reference Networks for Rare Endocrine Disorders (Endo-ERN) on new patients with CS from January 2021 to July 2022. Results: Of 222 patients (mean age 44 years, 165 females), 141 patients had Cushing disease (64%), 69 adrenal CS (31%), and 12 patients with ectopic CS (5.4%). The mean follow-up period post-CS diagnosis was 15 months (range 3–30). Cortisol-lowering medications were initiated in 38% of patients. One hundred fifty-four patients (69%) received thromboprophylaxis (including patients on chronic anticoagulant treatment), of which low-molecular-weight heparins were used in 96% of cases. VTE was reported in six patients (2.7%), of which one was fatal: two long before CS diagnosis, two between diagnosis and surgery, and two postoperatively. Three patients were using thromboprophylaxis at time of the VTE diagnosis. The incidence rate of VTE in patients after Cushing syndrome diagnosis in our study cohort was 14.6 (95% CI 5.5; 38.6) per 1000 person-years. Conclusion: Thirty percent of patients with CS did not receive preoperative thromboprophylaxis during their active disease stage, and half of the VTE cases even occurred during this stage despite thromboprophylaxis. Prospective trials to establish the optimal thromboprophylaxis strategy in CS patients are highly needed. Significance statement The incidence rate of venous thromboembolism in our study cohort was 14.6 (95% CI 5.5; 38.6) per 1000 person-years. Notably, this survey showed that there is great heterogeneity regarding time of initiation and duration of thromboprophylaxis in expert centers throughout Europe.</p