PyRaDiSe: A Python package for DICOM-RT-based auto-segmentation pipeline construction and DICOM-RT data conversion.

BACKGROUND AND OBJECTIVE Despite fast evolution cycles in deep learning methodologies for medical imaging in radiotherapy, auto-segmentation solutions rarely run in clinics due to the lack of open-source frameworks feasible for processing DICOM RT Structure Sets. Besides this shortage, available open-source DICOM RT Structure Set converters rely exclusively on 2D reconstruction approaches leading to pixelated contours with potentially low acceptance by healthcare professionals. PyRaDiSe, an open-source, deep learning framework independent Python package, addresses these issues by providing a framework for building auto-segmentation solutions feasible to operate directly on DICOM data. In addition, PyRaDiSe provides profound DICOM RT Structure Set conversion and processing capabilities; thus, it applies also to auto-segmentation-related tasks, such as dataset construction for deep learning model training. METHODS The PyRaDiSe package follows a holistic approach and provides DICOM data handling, deep learning model inference, pre-processing, and post-processing functionalities. The DICOM data handling allows for highly automated and flexible handling of DICOM image series, DICOM RT Structure Sets, and DICOM registrations, including 2D-based and 3D-based conversion from and to DICOM RT Structure Sets. For deep learning model inference, extending given skeleton classes is straightforwardly achieved, allowing for employing any deep learning framework. Furthermore, a profound set of pre-processing and post-processing routines is included that incorporate partial invertibility for restoring spatial properties, such as image origin or orientation. RESULTS The PyRaDiSe package, characterized by its flexibility and automated routines, allows for fast deployment and prototyping, reducing efforts for auto-segmentation pipeline implementation. Furthermore, while deep learning model inference is independent of the deep learning framework, it can easily be integrated into famous deep learning frameworks such as PyTorch or Tensorflow. The developed package has successfully demonstrated its capabilities in a research project at our institution for organs-at-risk segmentation in brain tumor patients. Furthermore, PyRaDiSe has shown its conversion performance for dataset construction. CONCLUSIONS The PyRaDiSe package closes the gap between data science and clinical radiotherapy by enabling deep learning segmentation models to be easily transferred into clinical research practice. PyRaDiSe is available on https://github.com/ubern-mia/pyradise and can be installed directly from the Python Package Index using pip install pyradise

The predictive value of segmentation metrics on dosimetry in organs at risk of the brain.

BACKGROUND Fully automatic medical image segmentation has been a long pursuit in radiotherapy (RT). Recent developments involving deep learning show promising results yielding consistent and time efficient contours. In order to train and validate these systems, several geometric based metrics, such as Dice Similarity Coefficient (DSC), Hausdorff, and other related metrics are currently the standard in automated medical image segmentation challenges. However, the relevance of these metrics in RT is questionable. The quality of automated segmentation results needs to reflect clinical relevant treatment outcomes, such as dosimetry and related tumor control and toxicity. In this study, we present results investigating the correlation between popular geometric segmentation metrics and dose parameters for Organs-At-Risk (OAR) in brain tumor patients, and investigate properties that might be predictive for dose changes in brain radiotherapy. METHODS A retrospective database of glioblastoma multiforme patients was stratified for planning difficulty, from which 12 cases were selected and reference sets of OARs and radiation targets were defined. In order to assess the relation between segmentation quality -as measured by standard segmentation assessment metrics- and quality of RT plans, clinically realistic, yet alternative contours for each OAR of the selected cases were obtained through three methods: (i) Manual contours by two additional human raters. (ii) Realistic manual manipulations of reference contours. (iii) Through deep learning based segmentation results. On the reference structure set a reference plan was generated that was re-optimized for each corresponding alternative contour set. The correlation between segmentation metrics, and dosimetric changes was obtained and analyzed for each OAR, by means of the mean dose and maximum dose to 1% of the volume (Dmax 1%). Furthermore, we conducted specific experiments to investigate the dosimetric effect of alternative OAR contours with respect to the proximity to the target, size, particular shape and relative location to the target. RESULTS We found a low correlation between the DSC, reflecting the alternative OAR contours, and dosimetric changes. The Pearson correlation coefficient between the mean OAR dose effect and the Dice was -0.11. For Dmax 1%, we found a correlation of -0.13. Similar low correlations were found for 22 other segmentation metrics. The organ based analysis showed that there is a better correlation for the larger OARs (i.e. brainstem and eyes) as for the smaller OARs (i.e. optic nerves and chiasm). Furthermore, we found that proximity to the target does not make contour variations more susceptible to the dose effect. However, the direction of the contour variation with respect to the relative location of the target seems to have a strong correlation with the dose effect. CONCLUSIONS This study shows a low correlation between segmentation metrics and dosimetric changes for OARs in brain tumor patients. Results suggest that the current metrics for image segmentation in RT, as well as deep learning systems employing such metrics, need to be revisited towards clinically oriented metrics that better reflect how segmentation quality affects dose distribution and related tumor control and toxicity

Deep learning-based simultaneous multi-phase deformable image registration of sparse 4D-CBCT

Purpose: Respiratory gated 4D-CBCT suffers from sparseness artefacts caused by the limited number of projections available for each respiratory phase/amplitude. These artefacts severely impact deformable image registration methods used to extract motion information. We use deep learning-based methods to predict displacement vector-fields (DVF) from sparse 4D-CBCT images to alleviate the impacts of sparseness artefacts. Methods: We trained U-Net-type convolutional neural network models to predict multiple (10) DVFs in a single forward pass given multiple sparse, gated CBCT and an optional artefact-free reference image as inputs. The predicted DVFs are used to warp the reference image to the different motion states, resulting in an artefact-free image for each state. The supervised training uses data generated by a motion simulation framework. The training dataset consists of 560 simulated 4D-CBCT images of 56 different patients; the generated data include fully sampled ground-truth images that are used to train the network. We compare the results of our method to pairwise image registration (reference image to single sparse image) using a) the deeds algorithm and b) VoxelMorph with image pair inputs. Results: We show that our method clearly outperforms pairwise registration using the deeds algorithm alone. PSNR improved from 25.8 to 46.4, SSIM from 0.9296 to 0.9999. In addition, the runtime of our learning-based method is orders of magnitude shorter (2 seconds instead of 10 minutes). Our results also indicate slightly improved performance compared to pairwise registration (delta-PSNR=1.2). We also trained a model that does not require the artefact-free reference image (which is usually not available) during inference demonstrating only marginally compromised results (delta-PSNR=-0.8). Conclusion: To the best of our knowledge, this is the first time CNNs are used to predict multi-phase DVFs in a single forward pass. This enables novel applications such as 4D-auto-segmentation, motion compensated image reconstruction, motion analyses, and patient motion modeling

Mitigation of motion-induced artifacts in cone beam computed tomography using deep convolutional neural networks

Background: Cone beam computed tomography (CBCT) is often employed on radiation therapy treatment devices (linear accelerators) used in image-guided radiation therapy (IGRT). For each treatment session, it is necessary to obtain the image of the day in order to accurately position the patient, and to enable adaptive treatment capabilities including auto-segmentation and dose calculation. Reconstructed CBCT images often suffer from artifacts, in particular those induced by patient motion. Deep-learning based approaches promise ways to mitigate such artifacts. Purpose: We propose a novel deep-learning based approach with the goal to reduce motion induced artifacts in CBCT images and improve image quality. It is based on supervised learning and includes neural network architectures employed as pre- and/or post-processing steps during CBCT reconstruction. Methods: Our approach is based on deep convolutional neural networks which complement the standard CBCT reconstruction, which is performed either with the analytical Feldkamp-Davis-Kress (FDK) method, or with an iterative algebraic reconstruction technique (SART-TV). The neural networks, which are based on refined U-net architectures, are trained end-to-end in a supervised learning setup. Labeled training data are obtained by means of a motion simulation, which uses the two extreme phases of 4D CT scans, their deformation vector fields, as well as time-dependent amplitude signals as input. The trained networks are validated against ground truth using quantitative metrics, as well as by using real patient CBCT scans for a qualitative evaluation by clinical experts. Results: The presented novel approach is able to generalize to unseen data and yields significant reductions in motion induced artifacts as well as improvements in image quality compared with existing state-of-the-art CBCT reconstruction algorithms (up to +6.3 dB and +0.19 improvements in peak signal-to-noise ratio, PSNR, and structural similarity index measure, SSIM, respectively), as evidenced by validation with an unseen test dataset, and confirmed by a clincal evaluation on real patient scans (up to 74% preference for motion artifact reduction over standard reconstruction). Conclusions: For the first time, it is demonstrated, also by means of clinical evaluation, that inserting deep neural networks as pre- and post-processing plugins in the existing 3D CBCT reconstruction and trained end-to-end yield significant improvements in image quality and reduction of motion artifacts

Abstracts from the 8th International Conference on cGMP Generators, Effectors and Therapeutic Implications

This work was supported by a restricted research grant of Bayer AG

Verifying tumor position during stereotactic body radiation therapy delivery using (limited-arc) cone beam computed tomography imaging

BACKGROUND AND PURPOSE: Proof of tumor position during stereotactic body radiotherapy (SBRT) delivery is desirable. We investigated if cone-beam CT (CBCT) scans reconstructed from (collimated) fluoroscopic kV images acquired during irradiation could show the dominant tumor position. MATERIALS AND METHODS: Full-arc CBCT scans were reconstructed using FDK filtered back projection from 38kV fluoroscopy datasets (16 patients) continuously acquired during volumetric modulated spine SBRT. CBCT-CT match values were compared to the average spine offset values found using template matching+triangulation of the individual kV images. Multiple limited-arc CBCTs were reconstructed from fluoroscopic images acquired during lung SBRT of an anthropomorphic thorax phantom using 20-180° arc lengths and for 3 breath-hold lung SBRT patients. RESULTS: Differences between 3D CBCT-CT match results and average spine offsets found using template matching+triangulation were 0.1±0.1mm for all directions (range: 0.0-0.5mm). For limited-arc CBCTs of the thorax phantom, the automatic 3D CBCT-CT match results for arc lengths of 80-180° were ≤1mm. 20° CBCT reconstruction still allowed for positional verification in 2D. CONCLUSIONS: (Limited-arc) CBCT reconstructions of kV images acquired during irradiation can identify the dominant position of the tumor during treatment delivery

Time dependent pre-treatment EPID dosimetry for standard and FFF VMAT

Methods to calibrate Megavoltage electronic portal imaging devices (EPIDs) for dosimetry have been previously documented for dynamic treatments such as intensity modulated radiotherapy (IMRT) using flattened beams and typically using integrated fields. While these methods verify the accumulated field shape and dose, the dose rate and differential fields remain unverified. The aim of this work is to provide an accurate calibration model for time dependent pre-treatment dose verification using amorphous silicon (a-Si) EPIDs in volumetric modulated arc therapy (VMAT) for both flattened and flattening filter free (FFF) beams. A general calibration model was created using a Varian TrueBeam accelerator, equipped with an aS1000 EPID, for each photon spectrum 6 MV, 10 MV, 6 MV-FFF, 10 MV-FFF. As planned VMAT treatments use control points (CPs) for optimization, measured images are separated into corresponding time intervals for direct comparison with predictions. The accuracy of the calibration model was determined for a range of treatment conditions. Measured and predicted CP dose images were compared using a time dependent gamma evaluation using criteria (3%, 3 mm, 0.5 sec). Time dependent pre-treatment dose verification is possible without an additional measurement device or phantom, using the on-board EPID. Sufficient data is present in trajectory log files and EPID frame headers to reliably synchronize and resample portal images. For the VMAT plans tested, significantly more deviation is observed when analysed in a time dependent manner for FFF and non-FFF plans than when analysed using only the integrated field. We show EPID-based pre-treatment dose verification can be performed on a CP basis for VMAT plans. This model can measure pre-treatment doses for both flattened and unflattened beams in a time dependent manner which highlights deviations that are missed in integrated field verifications

Patient positioning in radiotherapy based on surface imaging using time of flight cameras

International audienc

Detection of anatomical changes in lung cancer patients with 2D time-integrated, 2D time-resolved and 3D time-integrated portal dosimetry: a simulation study

The aim of this work is to assess the performance of 2D time-integrated (2D-TI), 2D time-resolved (2D-TR) and 3D time-integrated (3D-TI) portal dosimetry in detecting dose discrepancies between the planned and (simulated) delivered dose caused by simulated changes in the anatomy of lung cancer patients.For six lung cancer patients, tumor shift, tumor regression and pleural effusion are simulated by modifying their CT images. Based on the modified CT images, time-integrated (TI) and time-resolved (TR) portal dose images (PDIs) are simulated and 3D-TI doses are calculated. The modified and original PDIs and 3D doses are compared by a gamma analysis with various gamma criteria. Furthermore, the difference in the D-95% (Delta D-95%) of the GTV is calculated and used as a gold standard. The correlation between the gamma fail rate and the Delta D-95% is investigated, as well the sensitivity and specificity of all combinations of portal dosimetry method, gamma criteria and gamma fail rate threshold.On the individual patient level, there is a correlation between the gamma fail rate and the Delta D-95%, which cannot be found at the group level. The sensitivity and specificity analysis showed that there is not one combination of portal dosimetry method, gamma criteria and gamma fail rate threshold that can detect all simulated anatomical changes.This work shows that it will be more beneficial to relate portal dosimetry and DVH analysis on the patient level, rather than trying to quantify a relationship for a group of patients. With regards to optimizing sensitivity and specificity, different combinations of portal dosimetry method, gamma criteria and gamma fail rate should be used to optimally detect certain types of anatomical changes.</p