Semi-supervised learning towards automated segmentation of PET images with limited annotations: Application to lymphoma patients

The time-consuming task of manual segmentation challenges routine systematic quantification of disease burden. Convolutional neural networks (CNNs) hold significant promise to reliably identify locations and boundaries of tumors from PET scans. We aimed to leverage the need for annotated data via semi-supervised approaches, with application to PET images of diffuse large B-cell lymphoma (DLBCL) and primary mediastinal large B-cell lymphoma (PMBCL). We analyzed 18F-FDG PET images of 292 patients with PMBCL (n=104) and DLBCL (n=188) (n=232 for training and validation, and n=60 for external testing). We employed FCM and MS losses for training a 3D U-Net with different levels of supervision: i) fully supervised methods with labeled FCM (LFCM) as well as Unified focal and Dice loss functions, ii) unsupervised methods with Robust FCM (RFCM) and Mumford-Shah (MS) loss functions, and iii) Semi-supervised methods based on FCM (RFCM+LFCM), as well as MS loss in combination with supervised Dice loss (MS+Dice). Unified loss function yielded higher Dice score (mean +/- standard deviation (SD)) (0.73 +/- 0.03; 95% CI, 0.67-0.8) compared to Dice loss (p-value<0.01). Semi-supervised (RFCM+alpha*LFCM) with alpha=0.3 showed the best performance, with a Dice score of 0.69 +/- 0.03 (95% CI, 0.45-0.77) outperforming (MS+alpha*Dice) for any supervision level (any alpha) (p<0.01). The best performer among (MS+alpha*Dice) semi-supervised approaches with alpha=0.2 showed a Dice score of 0.60 +/- 0.08 (95% CI, 0.44-0.76) compared to another supervision level in this semi-supervised approach (p<0.01). Semi-supervised learning via FCM loss (RFCM+alpha*LFCM) showed improved performance compared to supervised approaches. Considering the time-consuming nature of expert manual delineations and intra-observer variabilities, semi-supervised approaches have significant potential for automated segmentation workflows

Observer study-based evaluation of TGAN architecture used to generate oncological PET images

The application of computer-vision algorithms in medical imaging has increased rapidly in recent years. However, algorithm training is challenging due to limited sample sizes, lack of labeled samples, as well as privacy concerns regarding data sharing. To address these issues, we previously developed (Bergen et al. 2022) a synthetic PET dataset for Head and Neck (H and N) cancer using the temporal generative adversarial network (TGAN) architecture and evaluated its performance segmenting lesions and identifying radiomics features in synthesized images. In this work, a two-alternative forced-choice (2AFC) observer study was performed to quantitatively evaluate the ability of human observers to distinguish between real and synthesized oncological PET images. In the study eight trained readers, including two board-certified nuclear medicine physicians, read 170 real/synthetic image pairs presented as 2D-transaxial using a dedicated web app. For each image pair, the observer was asked to identify the real image and input their confidence level with a 5-point Likert scale. P-values were computed using the binomial test and Wilcoxon signed-rank test. A heat map was used to compare the response accuracy distribution for the signed-rank test. Response accuracy for all observers ranged from 36.2% [27.9-44.4] to 63.1% [54.8-71.3]. Six out of eight observers did not identify the real image with statistical significance, indicating that the synthetic dataset was reasonably representative of oncological PET images. Overall, this study adds validity to the realism of our simulated H&N cancer dataset, which may be implemented in the future to train AI algorithms while favoring patient confidentiality and privacy protection

The image biomarker standardization initiative: Standardized convolutional filters for reproducible radiomics and enhanced clinical insights

Standardizing convolutional filters that enhance specific structures and patterns in medical imaging enables reproducible radiomics analyses, improving consistency and reliability for enhanced clinical insights. Filters are commonly used to enhance specific structures and patterns in images, such as vessels or peritumoral regions, to enable clinical insights beyond the visible image using radiomics. However, their lack of standardization restricts reproducibility and clinical translation of radiomics decision support tools. In this special report, teams of researchers who developed radiomics software participated in a three-phase study (September 2020 to December 2022) to establish a standardized set of filters. The first two phases focused on finding reference filtered images and reference feature values for commonly used convolutional filters: mean, Laplacian of Gaussian, Laws and Gabor kernels, separable and nonseparable wavelets (including decomposed forms), and Riesz transformations. In the first phase, 15 teams used digital phantoms to establish 33 reference filtered images of 36 filter configurations. In phase 2, 11 teams used a chest CT image to derive reference values for 323 of 396 features computed from filtered images using 22 filter and image processing configurations. Reference filtered images and feature values for Riesz transformations were not established. Reproducibility of standardized convolutional filters was validated on a public data set of multimodal imaging (CT, fluorodeoxyglucose PET, and T1-weighted MRI) in 51 patients with soft-tissue sarcoma. At validation, reproducibility of 486 features computed from filtered images using nine configurations × three imaging modalities was assessed using the lower bounds of 95% CIs of intraclass correlation coefficients. Out of 486 features, 458 were found to be reproducible across nine teams with lower bounds of 95% CIs of intraclass correlation coefficients greater than 0.75. In conclusion, eight filter types were standardized with reference filtered images and reference feature values for verifying and calibrating radiomics software packages. A web-based tool is available for compliance checking

Detection of the rotator cuff tears using a novel convolutional neural network from magnetic resonance image (MRI)

The rotator cuff tear is a common situation for basketballers, handballers, or other athletes that strongly use their shoulders. This injury can be diagnosed precisely from a magnetic resonance (MR) image. In this paper, a novel deep learning-based framework is proposed to diagnose rotator cuff tear from MRI images of patients suspected of the rotator cuff tear. First, we collected 150 shoulders MRI images from two classes of rotator cuff tear patients and healthy ones with the same numbers. These images were observed by an orthopedic specialist and then tagged and used as input in the various configurations of the Convolutional Neural Network (CNN). At this stage, five different configurations of convolutional networks have been examined. Then, in the next step, the selected network with the highest accuracy is used to extract the deep features and classify the two classes of rotator cuff tear and healthy. Also, MRI images are feed to two quick pre-trained CNNs (MobileNetv2 and SqueezeNet) to compare with the proposed CNN. Finally, the evaluation is performed using the 5-fold cross-validation method. Also, a specific Graphical User Interface (GUI) was designed in the MATLAB environment for simplicity, which allows for testing by detecting the image class. The proposed CNN achieved higher accuracy than the two mentioned pre-trained CNNs. The average accuracy, precision, sensitivity, and specificity achieved by the best selected CNN configuration are equal to 92.67%, 91.13%, 91.75%, and 92.22%, respectively. The deep learning algorithm could accurately rule out significant rotator cuff tear based on shoulder MRI

AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging: Towards Radiophenomics

International audienc

AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging

International audienc

Assessing Privacy Leakage in Synthetic 3-D PET Imaging using Transversal GAN

Training computer-vision related algorithms on medical images for disease diagnosis or image segmentation is difficult in large part due to privacy concerns. For this reason, generative image models are highly sought after to facilitate data sharing. However, 3-D generative models are understudied, and investigation of their privacy leakage is needed. We introduce our 3-D generative model, Transversal GAN (TrGAN), using head & neck PET images which are conditioned on tumour masks as a case study. We define quantitative measures of image fidelity, utility and privacy for our model. These metrics are evaluated in the course of training to identify ideal fidelity, utility and privacy trade-offs and establish the relationships between these parameters. We show that the discriminator of the TrGAN is vulnerable to attack, and that an attacker can identify which samples were used in training with almost perfect accuracy (AUC = 0.99). We also show that an attacker with access to only the generator cannot reliably classify whether a sample had been used for training (AUC = 0.51). This suggests that TrGAN generators, but not discriminators, may be used for sharing synthetic 3-D PET data with minimal privacy risk while maintaining good utility and fidelity.Comment: arXiv admin note: text overlap with arXiv:2111.0186

Role of Artificial Intelligence in Theranostics: Toward Routine Personalized Radiopharmaceutical Therapies

The final publication is available at Elsevier via https://doi.org/10.1016/j.cpet.2021.06.002. © 2021. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/KEY POINTS AI has shown promising applications in quantitative imaging required for dosimetry. Segmentation of organs and tumors, the most time consuming task in the dosimetry workflow, can be automated using AI. Using the theranostic approach, AI models that predict absorbed dose and therapy outcomes might play a key role in personalizing RPTs. AI has significant potential to improve accuracy and reduce times for routine implementation of patient-specific dosimetry in RPTs.Natural Sciences and Engineering Research Council of Canada (NSERC), Discovery G4rant RGPIN-2019-06467 || NSERC, Discovery Grant RGPIN-2021-02965

Radiomics predictive modeling from dual-time-point FDG PET Ki parametric maps : application to chemotherapy response in lymphoma

Background: To investigate the use of dynamic radiomics features derived from dual-time-point (DTP-feature) [18F]FDG PET metabolic uptake rate Ki parametric maps to develop a predictive model for response to chemotherapy in lymphoma patients. Methods We analyzed 126 lesions from 45 lymphoma patients (responding n = 75 and non-responding n = 51) treated with chemotherapy from two different centers. Static and DTP radiomics features were extracted from baseline static PET images and DTP Ki parametric maps. Spearman’s rank correlations were calculated between static and DTP features to identify features with potential additional information. We first employed univariate analysis to determine correlations between individual features, and subsequently utilized multivariate analysis to derive predictive models utilizing DTP and static radiomics features before and after ComBat harmonization. For multivariate modeling, we utilized both the minimum redundancy maximum relevance feature selection technique and the XGBoost classifier. To evaluate our model, we partitioned the patient datasets into training/validation and testing sets using an 80/20% split. Different metrics for classification including area under the curve (AUC), sensitivity (SEN), specificity (SPE), and accuracy (ACC) were reported in test sets. Results Via Spearman’s rank correlations, there was negligible to moderate correlation between 32 out of 65 DTP features and some static features (ρ < 0.7); all the other 33 features showed high correlations (ρ ≥ 0.7). In univariate modeling, no significant difference between AUC of DTP and static features was observed. GLRLM_RLNU from static features demonstrated a strong correlation (AUC = 0.75, p value = 0.0001, q value = 0.0007) with therapy response. The most predictive DTP features were GLCM_Energy, GLCM_Entropy, and Uniformity, each with AUC = 0.73, p value = 0.0001, and q value < 0.0005. In multivariate analysis, the mean ranges of AUCs increased following harmonization. Use of harmonization plus combining DTP and static features was shown to provide significantly improved predictions (AUC = 0.97 ± 0.02, accuracy = 0.89 ± 0.05, sensitivity = 0.92 ± 0.09, and specificity = 0.88 ± 0.05). All models depicted significant performance in terms of AUC, ACC, SEN, and SPE (p < 0.05, Mann–Whitney test). Conclusions Our results demonstrate significant value in harmonization of radiomics features as well as combining DTP and static radiomics models for predicting response to chemotherapy in lymphoma patients.Medicine, Faculty ofScience, Faculty ofNon UBCPhysics and Astronomy, Department ofRadiology, Department ofReviewedFacultyResearche

Information fusion for fully automated segmentation of head and neck tumors from PET and CT images

Background: PET/CT images combining anatomic and metabolic data provide complementary information that can improve clinical task performance. PET image segmentation algorithms exploiting the multi-modal information available are still lacking. Purpose: Our study aimed to assess the performance of PET and CT image fusion for gross tumor volume (GTV) segmentations of head and neck cancers (HNCs) utilizing conventional, deep learning (DL), and output-level voting-based fusions. Methods: The current study is based on a total of 328 histologically confirmed HNCs from six different centers. The images were automatically cropped to a 200 × 200 head and neck region box, and CT and PET images were normalized for further processing. Eighteen conventional image-level fusions were implemented. In addition, a modified U2-Net architecture as DL fusion model baseline was used. Three different input, layer, and decision-level information fusions were used. Simultaneous truth and performance level estimation (STAPLE) and majority voting to merge different segmentation outputs (from PET and image-level and network-level fusions), that is, output-level information fusion (voting-based fusions) were employed. Different networks were trained in a 2D manner with a batch size of 64. Twenty percent of the dataset with stratification concerning the centers (20% in each center) were used for final result reporting. Different standard segmentation metrics and conventional PET metrics, such as SUV, were calculated. Results: In single modalities, PET had a reasonable performance with a Dice score of 0.77 ± 0.09, while CT did not perform acceptably and reached a Dice score of only 0.38 ± 0.22. Conventional fusion algorithms obtained a Dice score range of [0.76-0.81] with guided-filter-based context enhancement (GFCE) at the low-end, and anisotropic diffusion and Karhunen-Loeve transform fusion (ADF), multi-resolution singular value decomposition (MSVD), and multi-level image decomposition based on latent low-rank representation (MDLatLRR) at the high-end. All DL fusion models achieved Dice scores of 0.80. Output-level voting-based models outperformed all other models, achieving superior results with a Dice score of 0.84 for Majority_ImgFus, Majority_All, and Majority_Fast. A mean error of almost zero was achieved for all fusions using SUVpeak , SUVmean and SUVmedian . Conclusion: PET/CT information fusion adds significant value to segmentation tasks, considerably outperforming PET-only and CT-only methods. In addition, both conventional image-level and DL fusions achieve competitive results. Meanwhile, output-level voting-based fusion using majority voting of several algorithms results in statistically significant improvements in the segmentation of HNC.</p