METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII

Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. Critical relevance statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. Key points: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score (https://metricsscore.github.io/metrics/METRICS.html) and a repository created to collect feedback from the radiomics community (https://github.com/metricsscore/metrics). Graphical Abstract: [Figure not available: see fulltext.

AI-Driven Multiclass Diagnosis in Chest X-rays: A Radiology Resident Training Experience

Chest X-rays (CXRs) are fundamental in the diagnosis of various thoracic pathologies, offering crucial insights into lung conditions. Yet, accurately and efficiently interpreting CXRs poses a challenge due to the intricate nature of thoracic anatomy, the overlapping structures, and the diverse array of potential abnormalities. Integration of AI-powered deep learning software in CXR interpretation holds promise as a solution, augmenting diagnostic precision and expediting the detection of intricate multiclass findings. Additionally, these AI tools serve as invaluable educational resources for radiology residents, fostering their growth and competence in interpreting thoracic CXRs. This study aimed to evaluate the impact of AI-based tools on diagnostic performance and confidence of radiology residents in detecting multiclass findings on CXR. In our institution, a retrospective review was conducted on all CXRs performed in June 2023 at the 2nd Radiology Unit in Pisa University Hospital, using qXR software (Qure.ai®) for labeling. The software output comprised the original postero-anterior radiographic image overlaid with segmentation and labels indicating the identified findings. A radiologist with over a decade of expertise in chest radiography was considered as the gold standard for this study. Each CXR was evaluated by two radiology residents, one in the first year and the other in the fourth year of their residency training, for the presence of ten different findings, including blunted costophrenic angle, cardiomegaly, cavity, consolidations, fibrosis, hilar enlargement, nodules, opacities, pneumothorax, and pleural effusion. The residents rated their confidence levels on a scale from 0 (completely uncertain) to 10 (absolutely certain). Following this, the processed image from qXR was provided, enabling the radiology trainees to re-evaluate based on this input and restate their confidence levels. Parameters such as accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were calculated for qXR and for each radiology resident, both with and without qXR. Statistical analyses were conducted using SPSS v.28.0, employing the t-student paired test to compare the diagnostic confidence levels before and after the adoption of qXR and the t-student unpaired test to compare diagnostic confidence levels between the first-year and fourth-year radiology residents. A p-value of less than 0.05 (two-tailed) was considered statistically significant. qXR exhibited outstanding performance in terms of accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. Both radiology residents showed improvement in diagnostic accuracy and diagnostic confidence levels using qXR. Specifically, a statistically significant difference (p<0.005) was observed in the level of diagnostic confidence before and after the adoption of qXR (Qure.ai®) for both radiology residents. Moreover, the level of diagnostic confidence was significantly higher (p<0.05) for the fourth-year radiology resident compared to the first-year radiology resident. In conclusion, qXR has showcased its potential role within the residents' training program, affirming its valuable contribution to enhancing diagnostic accuracy and confidence in the interpretation of multiclass findings on CXR

Ruolo della [18F]FDG-PET/TC in pazienti con carcinoma della tiroide Iodio refrattario in terapia con farmaci a bersaglio molecolare (Lenvatinib)

Tra il dicembre 2014 e settembre 2016, sono stati arruolati 33 pazienti affetti da Carcinoma della tiroide progressivo e metastatico Iodio-refrattario nel Dipartimento di Endocrinologia dell’Università di Pisa per essere sottoposti a trattamento con una dose di 24 mg/die di Lenvatinib. Lo scopo della tesi è quello di identificare il ruolo della PET/TC con FDG nella gestione della terapia di questi pazienti e nella definizione della prognosi

Artificial Intelligence-Based Software with CE Mark for Chest X-ray Interpretation: Opportunities and Challenges

Chest X-ray (CXR) is the most important technique for performing chest imaging, despite its well-known limitations in terms of scope and sensitivity. These intrinsic limitations of CXR have prompted the development of several artificial intelligence (AI)-based software packages dedicated to CXR interpretation. The online database “AI for radiology” was queried to identify CE-marked AI-based software available for CXR interpretation. The returned studies were divided according to the targeted disease. AI-powered computer-aided detection software is already widely adopted in screening and triage for pulmonary tuberculosis, especially in countries with few resources and suffering from high a burden of this disease. AI-based software has also been demonstrated to be valuable for the detection of lung nodules detection, automated flagging of positive cases, and post-processing through the development of digital bone suppression software able to produce digital bone suppressed images. Finally, the majority of available CE-marked software packages for CXR are designed to recognize several findings, with potential differences in sensitivity and specificity for each of the recognized findings

Natural language processing to convert unstructured COVID-19 chest-CT reports into structured reports

Background: Structured reporting has been demonstrated to increase report completeness and to reduce error rate, also enabling data mining of radiological reports. Still, structured reporting is perceived by radiologists as a fragmented reporting style, limiting their freedom of expression. Purpose: A deep learning-based natural language processing method was developed to automatically convert unstructured COVID-19 chest CT reports into structured reports. Methods: Two hundred-two COVID-19 chest CT were retrospectively reviewed by two experienced radiologists, who wrote for each exam a free-form text radiological report and coherently filled the template provided by the Italian Society of Medical and Interventional Radiology, used as ground-truth. A semi-supervised convolutional neural network was implemented to extract 62 categorical variables from the report. Two iterations were carried-out, the first without fine-tuning, the second one performing a fine-tuning. The performance was measured using the mean accuracy and the F1 mean score. An error analysis was performed to identify errors entirely attributable to incorrect processing of the model. Results: The algorithm achieved a mean accuracy of 93.7% and an F1 score 93.8% in the first iteration. Most of the errors were exclusively attributable to wrong inference (46%). In the second iteration the model achieved for both parameters 95,8% and percentage of errors attributable to wrong inference decreased to 26%. Conclusions: The convolutional neural network achieved an optimal performance in the automated conversion of free-form text into structured radiological reports, overcoming all the limitation attributed to structured reporting and finally paving the way for data mining of radiological report

The Role of Radiomics in Salivary Gland Imaging: A Systematic Review and Radiomics Quality Assessment

Background: Radiomics of salivary gland imaging can support clinical decisions in different clinical scenarios, such as tumors, radiation-induced xerostomia and sialadenitis. This review aims to evaluate the methodological quality of radiomics studies on salivary gland imaging. Material and Methods: A systematic search was performed, and the methodological quality was evaluated using the radiomics quality score (RQS). Subgroup analyses according to the first author&rsquo;s professional role (medical or not medical), journal type (radiological journal or other) and the year of publication (2021 or before) were performed. The correlation of RQS with the number of patients was calculated. Results: Twenty-three articles were included (mean RQS 11.34 &plusmn; 3.68). Most studies well-documented the imaging protocol (87%), while neither prospective validations nor cost-effectiveness analyses were performed. None of the included studies provided open-source data. A statistically significant difference in RQS according to the year of publication was found (p = 0.009), with papers published in 2021 having slightly higher RQSs than older ones. No differences according to journal type or the first author&rsquo;s professional role were demonstrated. A moderate relationship between the overall RQS and the number of patients was found. Conclusions: Radiomics application in salivary gland imaging is increasing. Although its current clinical applicability can be affected by the somewhat inadequate quality of the papers, a significant improvement in radiomics methodologies has been demonstrated in the last year

Imaging Diagnosis of Hepatocellular Carcinoma: A State-of-the-Art Review

Hepatocellular carcinoma (HCC) remains not only a cause of a considerable part of oncologic mortality, but also a diagnostic and therapeutic challenge for healthcare systems worldwide. Early detection of the disease and consequential adequate therapy are imperative to increase patients’ quality of life and survival. Imaging plays, therefore, a crucial role in the surveillance of patients at risk, the detection and diagnosis of HCC nodules, as well as in the follow-up post-treatment. The unique imaging characteristics of HCC lesions, deriving mainly from the assessment of their vascularity on contrast-enhanced computed tomography (CT), magnetic resonance (MR) or contrast-enhanced ultrasound (CEUS), allow for a more accurate, noninvasive diagnosis and staging. The role of imaging in the management of HCC has further expanded beyond the plain confirmation of a suspected diagnosis due to the introduction of ultrasound and hepatobiliary MRI contrast agents, which allow for the detection of hepatocarcinogenesis even at an early stage. Moreover, the recent technological advancements in artificial intelligence (AI) in radiology contribute an important tool for the diagnostic prediction, prognosis and evaluation of treatment response in the clinical course of the disease. This review presents current imaging modalities and their central role in the management of patients at risk and with HCC

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

The main aim of the present systematic review was a comprehensive overview of the Radiomics Quality Score (RQS)-based systematic reviews to highlight common issues and challenges of radiomics research application and evaluate the relationship between RQS and review features