A multicenter evaluation of a deep learning software (LungQuant) for lung parenchyma characterization in COVID-19 pneumonia

BackgroundThe role of computed tomography (CT) in the diagnosis and characterization of coronavirus disease 2019 (COVID-19) pneumonia has been widely recognized. We evaluated the performance of a software for quantitative analysis of chest CT, the LungQuant system, by comparing its results with independent visual evaluations by a group of 14 clinical experts. The aim of this work is to evaluate the ability of the automated tool to extract quantitative information from lung CT, relevant for the design of a diagnosis support model.MethodsLungQuant segments both the lungs and lesions associated with COVID-19 pneumonia (ground-glass opacities and consolidations) and computes derived quantities corresponding to qualitative characteristics used to clinically assess COVID-19 lesions. The comparison was carried out on 120 publicly available CT scans of patients affected by COVID-19 pneumonia. Scans were scored for four qualitative metrics: percentage of lung involvement, type of lesion, and two disease distribution scores. We evaluated the agreement between the LungQuant output and the visual assessments through receiver operating characteristics area under the curve (AUC) analysis and by fitting a nonlinear regression model.ResultsDespite the rather large heterogeneity in the qualitative labels assigned by the clinical experts for each metric, we found good agreement on the metrics compared to the LungQuant output. The AUC values obtained for the four qualitative metrics were 0.98, 0.85, 0.90, and 0.81.ConclusionsVisual clinical evaluation could be complemented and supported by computer-aided quantification, whose values match the average evaluation of several independent clinical experts

Making data big for a deep-learning analysis: Aggregation of public COVID-19 datasets of lung computed tomography scans

Lung Computed Tomography (CT) is an imaging technique useful to assess the severity of COVID-19 infection in symptomatic patients and to monitor its evolution over time. Lung CT can be analysed with the support of deep learning methods for both aforementioned tasks. We have developed a U-net based algorithm to segment the COVID-19 lesions. Unfortunately, public datasets populated with a huge amount of labelled CT scans of patients affected by COVID-19 are not available. In this work, we first review all the currently available public datasets of COVID-19 CT scans, presenting an extensive description of their characteristics. Then, we describe the design of the U-net we developed for the automated identification of COVID-19 lung lesions. Finally, we discuss the results obtained by using the different publicly available datasets. In particular, we trained the U-net on the dataset made available within the COVID-19 Lung CT Lesion Segmentation Challenge 2020, and we tested it on data from the MosMed and the COVID-19-CT-Seg datasets to explore the transferability of the model and to assess whether the image annotation process affects the detection performances. We evaluated the performance of the system in lesion segmentation in terms of the Dice index, which measures the overlap between the ground truth and the predicted masks. The proposed U-net segmentation model reaches a Dice index equal to 0.67, 0.42 and 0.58 on the independent validation sets of the COVID-19 Lung CT Lesion Segmentation Challenge 2020, on the MosMed and on the COVID-19-CT-Seg datasets, respectively. This work focusing on lesion segmentation constitutes a preliminary work for a more accurate analysis of COVID-19 lesions, based for example on the extraction and analysis of radiomic features

Quantification of pulmonary involvement in COVID-19 pneumonia: an upgrade of the LungQuant software for lung CT segmentation

Computed tomography (CT) scans are used to evaluate the severity of lung involvement in patients affected by COVID-19 pneumonia. Here, we present an improved version of the LungQuant automatic segmentation software (LungQuantv2), which implements a cascade of three deep neural networks (DNNs) to segment the lungs and the lung lesions associated with COVID-19 pneumonia. The first network (BB-net) defines a bounding box enclosing the lungs, the second one (U-net1) outputs the mask of the lungs, and the final one (U-net2) generates the mask of the COVID-19 lesions. With respect to the previous version (LungQuantv1), three main improvements are introduced: the BB-net, a new term in the loss function in the U-net for lesion segmentation and a post-processing procedure to separate the right and left lungs. The three DNNs were optimized, trained and tested on publicly available CT scans. We evaluated the system segmentation capability on an independent test set consisting of ten fully annotated CT scans, the COVID-19-CT-Seg benchmark dataset. The test performances are reported by means of the volumetric dice similarity coefficient (vDSC) and the surface dice similarity coefficient (sDSC) between the reference and the segmented objects. LungQuantv2 achieves a vDSC (sDSC) equal to 0.96 ± 0.01 (0.97 ± 0.01) and 0.69 ± 0.08 (0.83 ± 0.07) for the lung and lesion segmentations, respectively. The output of the segmentation software was then used to assess the percentage of infected lungs, obtaining a Mean Absolute Error (MAE) equal to 2%

Quantification of pulmonary involvement in COVID-19 pneumonia by means of a cascade of two U-nets: training and assessment on multiple datasets using different annotation criteria

none14noPurpose: This study aims at exploiting artificial intelligence (AI) for the identification, segmentation and quantification of COVID-19 pulmonary lesions. The limited data availability and the annotation quality are relevant factors in training AI-methods. We investigated the effects of using multiple datasets, heterogeneously populated and annotated according to different criteria. Methods: We developed an automated analysis pipeline, the LungQuant system, based on a cascade of two U-nets. The first one (U-net1) is devoted to the identification of the lung parenchyma; the second one (U-net2) acts on a bounding box enclosing the segmented lungs to identify the areas affected by COVID-19 lesions. Different public datasets were used to train the U-nets and to evaluate their segmentation performances, which have been quantified in terms of the Dice Similarity Coefficients. The accuracy in predicting the CT-Severity Score (CT-SS) of the LungQuant system has been also evaluated. Results: Both the volumetric DSC (vDSC) and the accuracy showed a dependency on the annotation quality of the released data samples. On an independent dataset (COVID-19-CT-Seg), both the vDSC and the surface DSC (sDSC) were measured between the masks predicted by LungQuant system and the reference ones. The vDSC (sDSC) values of 0.95±0.01 and 0.66±0.13 (0.95±0.02 and 0.76±0.18, with 5 mm tolerance) were obtained for the segmentation of lungs and COVID-19 lesions, respectively. The system achieved an accuracy of 90% in CT-SS identification on this benchmark dataset. Conclusion: We analysed the impact of using data samples with different annotation criteria in training an AI-based quantification system for pulmonary involvement in COVID-19 pneumonia. In terms of vDSC measures, the U-net segmentation strongly depends on the quality of the lesion annotations. Nevertheless, the CT-SS can be accurately predicted on independent test sets, demonstrating the satisfactory generalization ability of the LungQuant.noneLizzi F.; Agosti A.; Brero F.; Cabini R.F.; Fantacci M.E.; Figini S.; Lascialfari A.; Laruina F.; Oliva P.; Piffer S.; Postuma I.; Rinaldi L.; Talamonti C.; Retico A.Lizzi, F.; Agosti, A.; Brero, F.; Cabini, R. F.; Fantacci, M. E.; Figini, S.; Lascialfari, A.; Laruina, F.; Oliva, P.; Piffer, S.; Postuma, I.; Rinaldi, L.; Talamonti, C.; Retico, A

Diagnostic yield and cost-effectiveness of “dynamic” exome analysis in epilepsy with neurodevelopmental disorders: A tertiary-center experience in Northern Italy

Background: The advent of next-generation sequencing (NGS) techniques in clinical practice led to a significant advance in gene discovery. We aimed to describe diagnostic yields of a “dynamic” exome-based approach in a cohort of patients with epilepsy associated with neurodevel-opmental disorders. Methods: We conducted a retrospective, observational study on 72 probands. All patients underwent a first diagnostic level of a 135 gene panel, a second of 297 genes for inconclusive cases, and finally, a whole-exome sequencing for negative cases. Diagnostic yields at each step and cost-effectiveness were the objects of statistical analysis. Results: Overall diagnostic yield in our cohort was 37.5%: 29% of diagnoses derived from the first step analysis, 5.5% from the second step, and 3% from the third. A significant difference emerged between the three diagnostic steps (p < 0.01), between the first and second (p = 0.001), and the first and third (p << 0.001). The cost-effectiveness plane indicated that our exome-based “dynamic” approach was better in terms of cost savings and higher diagnostic rate. Conclusions: Our findings suggested that “dynamic” NGS techniques applied to well-phenotyped individuals can save both time and resources. In patients with unexplained epilepsy comorbid with NDDs, our approach might maximize the number of diagnoses achieved

Preliminary report on harmonization of features extraction process using the ComBat tool in the multi-center “Blue Sky Radiomics” study on stage III unresectable NSCLC

Background and purpose: In the retrospective-prospective multi-center "Blue Sky Radiomics” study (NCT04364776), we plan to test a pre-defined radiomic signature in a series of stage III unresectable NSCLC patients undergoing chemoradiotherapy and maintenance immunotherapy. As a necessary preliminary step, we explore the influence of different image-acquisition parameters on radiomic features’ reproducibility and apply methods for harmonization. Material and methods: We identified the primary lung tumor on two computed tomography (CT) series for each patient, acquired before and after chemoradiation with i.v. contrast medium and with different scanners. Tumor segmentation was performed by two oncological imaging specialists (thoracic radiologist and radio-oncologist) using the Oncentra Masterplan® software. We extracted 42 radiomic features from the specific ROIs (LIFEx). To assess the impact of different acquisition parameters on features extraction, we used the Combat tool with nonparametric adjustment and the longitudinal version (LongComBat). Results: We defined 14 CT acquisition protocols for the harmonization process. Before harmonization, 76% of the features were significantly influenced by these protocols. After, all extracted features resulted in being independent of the acquisition parameters. In contrast, 5% of the LongComBat harmonized features still depended on acquisition protocols. Conclusions: We reduced the impact of different CT acquisition protocols on radiomic features extraction in a group of patients enrolled in a radiomic study on stage III NSCLC. The harmonization process appears essential for the quality of radiomic data and for their reproducibility. ClinicalTrials.gov Identifier: NCT04364776, First Posted:April 28, 2020, Actual Study Start Date: April 15, 2020, https://clinicaltrials.gov/ct2/show/NCT04364776

A multicenter evaluation of a deep learning software (LungQuant) for lung parenchyma characterization in COVID-19 pneumonia

Background: The role of computed tomography (CT) in the diagnosis and characterization of coronavirus disease 2019 (COVID-19) pneumonia has been widely recognized. We evaluated the performance of a software for quantitative analysis of chest CT, the LungQuant system, by comparing its results with independent visual evaluations by a group of 14 clinical experts. The aim of this work is to evaluate the ability of the automated tool to extract quantitative information from lung CT, relevant for the design of a diagnosis support model. Methods: LungQuant segments both the lungs and lesions associated with COVID-19 pneumonia (ground-glass opacities and consolidations) and computes derived quantities corresponding to qualitative characteristics used to clinically assess COVID-19 lesions. The comparison was carried out on 120 publicly available CT scans of patients affected by COVID-19 pneumonia. Scans were scored for four qualitative metrics: percentage of lung involvement, type of lesion, and two disease distribution scores. We evaluated the agreement between the LungQuant output and the visual assessments through receiver operating characteristics area under the curve (AUC) analysis and by fitting a nonlinear regression model. Results: Despite the rather large heterogeneity in the qualitative labels assigned by the clinical experts for each metric, we found good agreement on the metrics compared to the LungQuant output. The AUC values obtained for the four qualitative metrics were 0.98, 0.85, 0.90, and 0.81. Conclusions: Visual clinical evaluation could be complemented and supported by computer-aided quantification, whose values match the average evaluation of several independent clinical experts. Key points: We conducted a multicenter evaluation of the deep learning-based LungQuant automated software.We translated qualitative assessments into quantifiable metrics to characterize coronavirus disease 2019 (COVID-19) pneumonia lesions.Comparing the software output to the clinical evaluations, results were satisfactory despite heterogeneity of the clinical evaluations.An automatic quantification tool may contribute to improve the clinical workflow of COVID-19 pneumonia