Clinical utility of convolutional neural networks for treatment planning in radiotherapy for spinal metastases

Background and purpose: Spine delineation is essential for high quality radiotherapy treatment planning of spinal metastases. However, manual delineation is time-consuming and prone to interobserver variability. Automatic spine delineation, especially using deep learning, has shown promising results in healthy subjects. We aimed to evaluate the clinical utility of deep learning-based vertebral body delineations for radiotherapy planning purposes. Materials and methods: A multi-scale convolutional neural network (CNN) was used for automatic segmentation and labeling. Two approaches were tested: the combined approach using one CNN for both segmentation and labeling, and the sequential approach using separate CNN's for these tasks. Training and internal validation data included 580 vertebrae, external validation data included 202 vertebrae. For quantitative assessment, Dice similarity coefficient (DSC) and Hausdorff distance (HD) were used. Axial slices from external images were presented to radiation oncologists for subjective evaluation. Results: Both approaches performed comparably during the internal validation (DSC: 96.7%, HD: 3.6 mm), but the sequential approach proved more robust during the external validation (DSC: 94.5% vs 94.4%, p < 0.001, HD: 4.5 vs 7.1 mm, p < 0.001). Subsequently, subjective evaluation of this sequential approach showed that experienced radiation oncologists could distinguish automatic from human-made contours in 63% of cases. They rated automatic contours clinically acceptable in 77% of cases, compared to 88% of human-made contours. Conclusion: We present a feasible approach for automatic vertebral body delineation using two variants of a multi-scale CNN. This approach generates high quality automatic delineations, which can save time in a clinical radiotherapy workflow

Deep learning for automated contouring of neurovascular structures on magnetic resonance imaging for prostate cancer patients

Background and purpose: Manual contouring of neurovascular structures on prostate magnetic resonance imaging (MRI) is labor-intensive and prone to considerable interrater disagreement. Our aim is to contour neurovascular structures automatically on prostate MRI by deep learning (DL) to improve workflow and interrater agreement. Materials and methods: Segmentation of neurovascular structures was performed on pre-treatment 3.0 T MRI data of 131 prostate cancer patients (training [n = 105] and testing [n = 26]). The neurovascular structures include the penile bulb (PB), corpora cavernosa (CCs), internal pudendal arteries (IPAs), and neurovascular bundles (NVBs). Two DL networks, nnU-Net and DeepMedic, were trained for auto-contouring on prostate MRI and evaluated using volumetric Dice similarity coefficient (DSC), mean surface distances (MSD), Hausdorff distances, and surface DSC. Three radiation oncologists evaluated the DL-generated contours and performed corrections when necessary. Interrater agreement was assessed and the time required for manual correction was recorded. Results: nnU-Net achieved a median DSC of 0.92 (IQR: 0.90–0.93) for the PB, 0.90 (IQR: 0.86–0.92) for the CCs, 0.79 (IQR: 0.77–0.83) for the IPAs, and 0.77 (IQR: 0.72–0.81) for the NVBs, which outperformed DeepMedic for each structure (p < 0.03). nnU-Net showed a median MSD of 0.24 mm for the IPAs and 0.71 mm for the NVBs. The median interrater DSC ranged from 0.93 to 1.00, with the majority of cases (68.9%) requiring manual correction times under two minutes. Conclusions: DL enables reliable auto-contouring of neurovascular structures on pre-treatment MRI data, easing the clinical workflow in neurovascular-sparing MR-guided radiotherapy

CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation

In presence of inter-fractional anatomical changes, clinical benefits are anticipated from image-guided adaptive radiotherapy. Nowadays, cone-beam CT (CBCT) imaging is mostly utilized during pre-treatment imaging for position verification. Due to various artifacts, image quality is typically not sufficient for photon or proton dose calculation, thus demanding accurate CBCT correction, as potentially provided by deep learning techniques. This work aimed at investigating the feasibility of utilizing a cycle-consistent generative adversarial network (cycleGAN) for prostate CBCT correction using unpaired training. Thirty-three patients were included. The network was trained to translate uncorrected, original CBCT images (CBCT org) into planning CT equivalent images (CBCT cycleGAN). HU accuracy was determined by comparison to a previously validated CBCT correction technique (CBCT cor). Dosimetric accuracy was inferred for volumetric-modulated arc photon therapy (VMAT) and opposing single-field uniform dose (OSFUD) proton plans, optimized on CBCT cor and recalculated on CBCT cycleGAN. Single-sided SFUD proton plans were utilized to assess proton range accuracy. The mean HU error of CBCT cycleGAN with respect to CBCT cor decreased from 24 HU for CBCT org to -6 HU. Dose calculation accuracy was high for VMAT, with average pass-rates of 100%/89% for a 2%/1% dose difference criterion. For proton OSFUD plans, the average pass-rate for a 2% dose difference criterion was 80%. Using a (2%, 2 mm) gamma criterion, the pass-rate was 96%. 93% of all analyzed SFUD profiles had a range agreement better than 3 mm. CBCT correction time was reduced from 6-10 min for CBCT cor to 10 s for CBCT cycleGAN. Our study demonstrated the feasibility of utilizing a cycleGAN for CBCT correction, achieving high dose calculation accuracy for VMAT. For proton therapy, further improvements may be required. Due to unpaired training, the approach does not rely on anatomically consistent training data or potentially inaccurate deformable image registration. The substantial speed-up for CBCT correction renders the method particularly interesting for adaptive radiotherapy

CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation

In presence of inter-fractional anatomical changes, clinical benefits are anticipated from image-guided adaptive radiotherapy. Nowadays, cone-beam CT (CBCT) imaging is mostly utilized during pre-treatment imaging for position verification. Due to various artifacts, image quality is typically not sufficient for photon or proton dose calculation, thus demanding accurate CBCT correction, as potentially provided by deep learning techniques. This work aimed at investigating the feasibility of utilizing a cycle-consistent generative adversarial network (cycleGAN) for prostate CBCT correction using unpaired training. Thirty-three patients were included. The network was trained to translate uncorrected, original CBCT images (CBCT org) into planning CT equivalent images (CBCT cycleGAN). HU accuracy was determined by comparison to a previously validated CBCT correction technique (CBCT cor). Dosimetric accuracy was inferred for volumetric-modulated arc photon therapy (VMAT) and opposing single-field uniform dose (OSFUD) proton plans, optimized on CBCT cor and recalculated on CBCT cycleGAN. Single-sided SFUD proton plans were utilized to assess proton range accuracy. The mean HU error of CBCT cycleGAN with respect to CBCT cor decreased from 24 HU for CBCT org to -6 HU. Dose calculation accuracy was high for VMAT, with average pass-rates of 100%/89% for a 2%/1% dose difference criterion. For proton OSFUD plans, the average pass-rate for a 2% dose difference criterion was 80%. Using a (2%, 2 mm) gamma criterion, the pass-rate was 96%. 93% of all analyzed SFUD profiles had a range agreement better than 3 mm. CBCT correction time was reduced from 6-10 min for CBCT cor to 10 s for CBCT cycleGAN. Our study demonstrated the feasibility of utilizing a cycleGAN for CBCT correction, achieving high dose calculation accuracy for VMAT. For proton therapy, further improvements may be required. Due to unpaired training, the approach does not rely on anatomically consistent training data or potentially inaccurate deformable image registration. The substantial speed-up for CBCT correction renders the method particularly interesting for adaptive radiotherapy

Deep MR to CT synthesis using unpaired data

MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images

Deep learning-based reconstruction of in vivo pelvis conductivity with a 3D patch-based convolutional neural network trained on simulated MR data

Purpose: To demonstrate that mapping pelvis conductivity at 3T with deep learning (DL) is feasible. Methods: 210 dielectric pelvic models were generated based on CT scans of 42 cervical cancer patients. For all dielectric models, electromagnetic and MR simulations with realistic accuracy and precision were performed to obtain (Formula presented.) and transceive phase (ϕ ±). Simulated (Formula presented.) and ϕ ± served as input to a 3D patch-based convolutional neural network, which was trained in a supervised fashion to retrieve the conductivity. The same network architecture was retrained using only ϕ ± in input. Both network configurations were tested on simulated MR data and their conductivity reconstruction accuracy and precision were assessed. Furthermore, both network configurations were used to reconstruct conductivity maps from a healthy volunteer and two cervical cancer patients. DL-based conductivity was compared in vivo and in silico to Helmholtz-based (H-EPT) conductivity. Results: Conductivity maps obtained from both network configurations were comparable. Accuracy was assessed by mean error (ME) with respect to ground truth conductivity. On average, ME < 0.1 Sm −1 for all tissues. Maximum MEs were 0.2 Sm −1 for muscle and tumour, and 0.4 Sm −1 for bladder. Precision was indicated with the difference between 90 th and 10 th conductivity percentiles, and was below 0.1 Sm −1 for fat, bone and muscle, 0.2 Sm −1 for tumour and 0.3 Sm −1 for bladder. In vivo, DL-based conductivity had median values in agreement with H-EPT values, but a higher precision. Conclusion: Anatomically detailed, noise-robust 3D conductivity maps with good sensitivity to tissue conductivity variations were reconstructed in the pelvis with DL

Deep learning‐based reconstruction of in vivo pelvis conductivity with a 3D patch‐based convolutional neural network trained on simulated MR data

Purpose: To demonstrate that mapping pelvis conductivity at 3T with deep learning (DL) is feasible. Methods: 210 dielectric pelvic models were generated based on CT scans of 42 cervical cancer patients. For all dielectric models, electromagnetic and MR simulations with realistic accuracy and precision were performed to obtain (Formula presented.) and transceive phase (ϕ ±). Simulated (Formula presented.) and ϕ ± served as input to a 3D patch-based convolutional neural network, which was trained in a supervised fashion to retrieve the conductivity. The same network architecture was retrained using only ϕ ± in input. Both network configurations were tested on simulated MR data and their conductivity reconstruction accuracy and precision were assessed. Furthermore, both network configurations were used to reconstruct conductivity maps from a healthy volunteer and two cervical cancer patients. DL-based conductivity was compared in vivo and in silico to Helmholtz-based (H-EPT) conductivity. Results: Conductivity maps obtained from both network configurations were comparable. Accuracy was assessed by mean error (ME) with respect to ground truth conductivity. On average, ME < 0.1 Sm −1 for all tissues. Maximum MEs were 0.2 Sm −1 for muscle and tumour, and 0.4 Sm −1 for bladder. Precision was indicated with the difference between 90 th and 10 th conductivity percentiles, and was below 0.1 Sm −1 for fat, bone and muscle, 0.2 Sm −1 for tumour and 0.3 Sm −1 for bladder. In vivo, DL-based conductivity had median values in agreement with H-EPT values, but a higher precision. Conclusion: Anatomically detailed, noise-robust 3D conductivity maps with good sensitivity to tissue conductivity variations were reconstructed in the pelvis with DL

Dosimetric evaluation of synthetic CT for head and neck radiotherapy generated by a patch-based three-dimensional convolutional neural network

Purpose: To develop and evaluate a patch-based convolutional neural network (CNN) to generate synthetic computed tomography (sCT) images for magnetic resonance (MR)-only workflow for radiotherapy of head and neck tumors. A patch-based deep learning method was chosen to improve robustness to abnormal anatomies caused by large tumors, surgical excisions, or dental artifacts. In this study, we evaluate whether the generated sCT images enable accurate MR-based dose calculations in the head and neck region. Methods: We conducted a retrospective study on 34 patients with head and neck cancer who underwent both CT and MR imaging for radiotherapy treatment planning. To generate the sCTs, a large field-of-view T2-weighted Turbo Spin Echo MR sequence was used from the clinical protocol for multiple types of head and neck tumors. To align images as well as possible on a voxel-wise level, CT scans were nonrigidly registered to the MR (CTreg). The CNN was based on a U-net architecture and consisted of 14 layers with 3 × 3 × 3 filters. Patches of 48 × 48 × 48 were randomly extracted and fed into the training. sCTs were created for all patients using threefold cross validation. For each patient, the clinical CT-based treatment plan was recalculated on sCT using Monaco TPS (Elekta). We evaluated mean absolute error (MAE) and mean error (ME) within the body contours and dice scores in air and bone mask. Also, dose differences and gamma pass rates between CT- and sCT-based plans inside the body contours were calculated. Results: sCT generation took 4 min per patient. The MAE over the patient population of the sCT within the intersection of body contours was 75 ± 9 Hounsfield Units (HU) (±1 SD), and the ME was 9 ± 11 HU. Dice scores of the air and bone masks (CTreg vs sCT) were 0.79 ± 0.08 and 0.70 ± 0.07, respectively. Dosimetric analysis showed mean deviations of −0.03% ± 0.05% for dose within the body contours and −0.07% ± 0.22% inside the >90% dose volume. Dental artifacts obscuring the CT could be circumvented in the sCT by the CNN-based approach in combination with Turbo Spin Echo (TSE) magnetic resonance imaging (MRI) sequence that typically is less prone to susceptibility artifacts. Conclusions: The presented CNN generated sCTs from conventional MR images without adding scan time to the acquisition. Dosimetric evaluation suggests that dose calculations performed on the sCTs are accurate, and can therefore be used for MR-only radiotherapy treatment planning of the head and neck

Dosimetric evaluation of synthetic CT for head and neck radiotherapy generated by a patch-based three-dimensional convolutional neural network

Purpose: To develop and evaluate a patch-based convolutional neural network (CNN) to generate synthetic computed tomography (sCT) images for magnetic resonance (MR)-only workflow for radiotherapy of head and neck tumors. A patch-based deep learning method was chosen to improve robustness to abnormal anatomies caused by large tumors, surgical excisions, or dental artifacts. In this study, we evaluate whether the generated sCT images enable accurate MR-based dose calculations in the head and neck region. Methods: We conducted a retrospective study on 34 patients with head and neck cancer who underwent both CT and MR imaging for radiotherapy treatment planning. To generate the sCTs, a large field-of-view T2-weighted Turbo Spin Echo MR sequence was used from the clinical protocol for multiple types of head and neck tumors. To align images as well as possible on a voxel-wise level, CT scans were nonrigidly registered to the MR (CT reg). The CNN was based on a U-net architecture and consisted of 14 layers with 3 × 3 × 3 filters. Patches of 48 × 48 × 48 were randomly extracted and fed into the training. sCTs were created for all patients using threefold cross validation. For each patient, the clinical CT-based treatment plan was recalculated on sCT using Monaco TPS (Elekta). We evaluated mean absolute error (MAE) and mean error (ME) within the body contours and dice scores in air and bone mask. Also, dose differences and gamma pass rates between CT- and sCT-based plans inside the body contours were calculated. Results: sCT generation took 4 min per patient. The MAE over the patient population of the sCT within the intersection of body contours was 75 ± 9 Hounsfield Units (HU) (±1 SD), and the ME was 9 ± 11 HU. Dice scores of the air and bone masks (CT reg vs sCT) were 0.79 ± 0.08 and 0.70 ± 0.07, respectively. Dosimetric analysis showed mean deviations of −0.03% ± 0.05% for dose within the body contours and −0.07% ± 0.22% inside the >90% dose volume. Dental artifacts obscuring the CT could be circumvented in the sCT by the CNN-based approach in combination with Turbo Spin Echo (TSE) magnetic resonance imaging (MRI) sequence that typically is less prone to susceptibility artifacts. Conclusions: The presented CNN generated sCTs from conventional MR images without adding scan time to the acquisition. Dosimetric evaluation suggests that dose calculations performed on the sCTs are accurate, and can therefore be used for MR-only radiotherapy treatment planning of the head and neck