PViT-6D: Overclocking Vision Transformers for 6D Pose Estimation with Confidence-Level Prediction and Pose Tokens

In the current state of 6D pose estimation, top-performing techniques depend on complex intermediate correspondences, specialized architectures, and non-end-to-end algorithms. In contrast, our research reframes the problem as a straightforward regression task by exploring the capabilities of Vision Transformers for direct 6D pose estimation through a tailored use of classification tokens. We also introduce a simple method for determining pose confidence, which can be readily integrated into most 6D pose estimation frameworks. This involves modifying the transformer architecture by decreasing the number of query elements based on the network's assessment of the scene complexity. Our method that we call Pose Vision Transformer or PViT-6D provides the benefits of simple implementation and being end-to-end learnable while outperforming current state-of-the-art methods by +0.3% ADD(-S) on Linemod-Occlusion and +2.7% ADD(-S) on the YCB-V dataset. Moreover, our method enhances both the model's interpretability and the reliability of its performance during inference

OC-0549: Improving the clinical applicability of markerless lung tumour tracking with contrast-enhanced kV imaging

In-room kV imaging is widely applied for intrafraction motion compensation in image-guided radiation therapy (IGRT). The low contrast of lung tumours in kV images and the overlap of high-intensity surrounding structures, such as the mediastinum, may limit the applicability of IGRT techniques in lung cancer treatments. The aim of this study is to apply a CT-based contrast enhancement method to improve markerless lung tumour tracking in kV images, thus enhancing the potential of X-raybased image guidance in lung cancer patients

Contactless Sensing of Water Properties for Smart Monitoring of Pipelines

A key milestone for the pervasive diffusion of wireless sensing nodes for smart monitoring of water quality and quantity in distribution networks is the simplification of the installation of sensors. To address this aspect, we demonstrate how two basic contactless sensors, such as piezoelectric transducers and strip electrodes (in a longitudinal interdigitated configuration to sense impedance inside and outside of the pipe with potential for impedimetric leak detection), can be easily clamped on plastic pipes to enable the measurement of multiple parameters without contact with the fluid and, thus, preserving the integrity of the pipe. Here we report the measurement of water flow rate (up to 24 m(3)/s) and temperature with ultrasounds and of the pipe filling fraction (capacitance at 1 MHz with similar to cm(3) resolution) and ionic conductivity (resistance at 20 MHz from 700 to 1400 mu S/cm) by means of impedance. The equivalent impedance model of the sensor is discussed in detail. Numerical finite-element simulations, carried out to optimize the sensing parameters such as the sensing frequency, confirm the lumped models and are matched by experimental results. In fact, a 6 m long, 30 L demonstration hydraulic loop was built to validate the sensors in realistic conditions (water speed of 1 m/s) monitoring a pipe segment of 0.45 m length and 90 mm diameter (one of the largest ever reported in the literature). Tradeoffs in sensors accuracy, deployment, and fabrication, for instance, adopting single-sided flexible PCBs as electrodes protected by Kapton on the external side and experimentally validated, are discussed as well

Intra‐frame motion deterioration effects and deep‐learning‐based compensation in MR‐guided radiotherapy

Background Current commercially available hybrid magnetic resonance linear accelerators (MR-Linac) use 2D+t cine MR imaging to provide intra-fractional motion monitoring. However, given the limited temporal resolution of cine MR imaging, target intra-frame motion deterioration effects, resulting in effective time latency and motion artifacts in the image domain, can be appreciable, especially in the case of fast breathing. Purpose The aim of this work is to investigate intra-frame motion deterioration effects in MR-guided radiotherapy (MRgRT) by simulating the motion-corrupted image acquisition, and to explore the feasibility of deep-learning-based compensation approaches, relying on the intra-frame motion information which is spatially and temporally encoded in the raw data (k-space). Methods An intra-frame motion model was defined to simulate motion-corrupted MR images, with 4D anthropomorphic digital phantoms being exploited to provide ground truth 2D+t cine MR sequences. A total number of 10 digital phantoms were generated for lung cancer patients, with randomly selected eight patients for training or validation and the remaining two for testing. The simulation code served as the data generator, and a dedicated motion pattern perturbation scheme was proposed to build the intra-frame motion database, where three degrees of freedom were designed to guarantee the diversity of intra-frame motion trajectories, enabling a thorough exploration in the domain of the potential anatomical structure positions. U-Nets with three types of loss functions: L1 or L2 loss defined in image or Fourier domain, referred to as NNImgLoss-L1, NNFloss-L1 and NNL2-Loss were trained to extract information from the motion-corrupted image and used to estimate the ground truth final-position image, corresponding to the end of the acquisition. Images before and after compensation were evaluated in terms of (i) image mean-squared error (MSE) and mean absolute error (MAE), and (ii) accuracy of gross tumor volume (GTV) contouring, based on optical-flow image registration. Results Image degradation caused by intra-frame motion was observed: for a linearly and fully acquired Cartesian readout k-space trajectory, intra-frame motion resulted in an imaging latency of approximately 50% of the acquisition time; in comparison, the motion artifacts exhibited only a negligible contribution to the overall geometric errors. All three compensation models led to a decrease in image MSE/MAE and GTV position offset compared to the motion-corrupted image. In the investigated testing dataset for GTV contouring, the average dice similarity coefficients (DSC) improved from 88% to 96%, and the 95th percentile Hausdorff distance (HD95) dropped from 4.8 mm to 2.1 mm. Different models showed slight performance variations across different intra-frame motion amplitude categories: NNImgLoss-L1 excelled for small/medium amplitudes, whereas NNFloss-L1 demonstrated higher DSC median values at larger amplitudes. The saliency maps of the motion-corrupted image highlighted the major contribution of the later acquired k-space data, as well as the edges of the moving anatomical structures at their final positions, during the model inference stage. Conclusions Our results demonstrate the deep-learning-based approaches have the potential to compensate for intra-frame motion by utilizing the later acquired data to drive the convergence of the earlier acquired k-space components

integration of enhanced optical tracking techniques and imaging in igrt

Patient setup/Optical tracking/IGRT/Treatment surveillance. In external beam radiotherapy, modern technologies for dynamic dose delivery and beam conformation provide high selectivity in radiation dose administration to the pathological volume. A comparable accuracy level is needed in the 3-D localization of tumor and organs at risk (OARs), in order to accomplish the planned dose distribution in the reality of each irradiation session. In-room imaging techniques for patient setup verification and tumor targeting may benefit of the combined daily use of optical tracking technologies, supported by techniques for the detection and compensation of organ motion events. Multiple solutions to enhance the use of optical tracking for the on-line correction of target localization uncertainties are described, with specific emphasis on the compensation of setup errors, breathing movements and non-rigid deformations. The final goal is the implementation of customized protocols where appropriate external landmarks, to be tracked in real-time by means of noninvasive optical devices, are selected as a function of inner target localization. The presented methodology features high accuracy in patient setup optimization, also providing a valuable tool for on-line patient surveillance, taking into account both breathing and deformation effects. The methodic application of optical tracking is put forward to represent a reliable and low cost procedure for the reduction of safety margins, once the patient-specific correlation between external landmarks and inner structures has been established. Therefore, the integration of optical tracking with in-room imaging devices is proposed as a way to gain higher confidence in the framework of Image Guided Radiation Therapy (IGRT) treatments

X-ray CT adaptation based on a 2D–3D deformable image registration framework using simulated in-room proton radiographies

The aim of this work is to investigate in-room proton radiographies to compensate realistic rigid and non-rigid transformations in clinical-like scenarios based on 2D–3D deformable image registration (DIR) framework towards future clinical implementation of adaptive radiation therapy (ART). Monte Carlo simulations of proton radiographies (pRads) based on clinical x-ray CT of a head and neck, and a brain tumor patients are simulated for two different detector configurations (i.e. integration-mode and list-mode detectors) including high and low proton statistics. A realistic deformation, derived from cone beam CT of the patient, is applied to the treatment planning CT. Rigid inaccuracies in patient positioning are also applied and the effect of small, medium and large fields of view (FOVs) is investigated. A stopping criterion, as desirable in realistic scenarios devoid of ground truth proton CT (pCT), is proposed and investigated. Results show that rigid and non-rigid transformations can be compensated based on a limited number of low dose pRads. The root mean square error with respect to the pCT shows that the 2D–3D DIR of the treatment planning CT based on 10 pRads from integration-mode data and 2 pRads from list-mode data is capable of achieving comparable accuracy (∼90% and >90%, respectively) to conventional 3D–3D DIR. The dice similarity coefficient over the segmented regions of interest also verifies the improvement in accuracy prior to and after 2D–3D DIR. No relevant changes in accuracy are found between high and low proton statistics except for 2 pRads from integration-mode data. The impact of FOV size is negligible. The convergence of the metric adopted for the stopping criterion indicates the optimal convergence of the 2D–3D DIR. This work represents a further step towards the potential implementation of ART in proton therapy. Further computational optimization is however required to enable extensive clinical validation

Simultaneous object detection and segmentation for patient‐specific markerless lung tumor tracking in simulated radiographs with deep learning

Background Real-time tumor tracking is one motion management method to address motion-induced uncertainty. To date, fiducial markers are often required to reliably track lung tumors with X-ray imaging, which carries risks of complications and leads to prolonged treatment time. A markerless tracking approach is thus desirable. Deep learning-based approaches have shown promise for markerless tracking, but systematic evaluation and procedures to investigate applicability in individual cases are missing. Moreover, few efforts have been made to provide bounding box prediction and mask segmentation simultaneously, which could allow either rigid or deformable multi-leaf collimator tracking. Purpose The purpose of this study was to implement a deep learning-based markerless lung tumor tracking model exploiting patient-specific training which outputs both a bounding box and a mask segmentation simultaneously. We also aimed to compare the two kinds of predictions and to implement a specific procedure to understand the feasibility of markerless tracking on individual cases. Methods We first trained a Retina U-Net baseline model on digitally reconstructed radiographs (DRRs) generated from a public dataset containing 875 CT scans and corresponding lung nodule annotations. Afterwards, we used an independent cohort of 97 lung patients to develop a patient-specific refinement procedure. In order to determine the optimal hyperparameters for automatic patient-specific training, we selected 13 patients for validation where the baseline model predicted a bounding box on planning CT (PCT)-DRR with intersection over union (IoU) with the ground-truth higher than 0.7. The final test set contained the remaining 84 patients with varying PCT-DRR IoU. For each testing patient, the baseline model was refined on the PCT-DRR to generate a patient-specific model, which was then tested on a separate 10-phase 4DCT-DRR to mimic the intrafraction motion during treatment. A template matching algorithm served as benchmark model. The testing results were evaluated by four metrics: the center of mass (COM) error and the Dice similarity coefficient (DSC) for segmentation masks, and the center of box (COB) error and the DSC for bounding box detections. Performance was compared to the benchmark model including statistical testing for significance. Results A PCT-DRR IoU value of 0.2 was shown to be the threshold dividing inconsistent (68%) and consistent (100%) success (defined as mean bounding box DSC > 0.6) of PS models on 4DCT-DRRs. Thirty-seven out of the eighty-four testing cases had a PCT-DRR IoU above 0.2. For these 37 cases, the mean COM error was 2.6 mm, the mean segmentation DSC was 0.78, the mean COB error was 2.7 mm, and the mean box DSC was 0.83. Including the validation cases, the model was applicable to 50 out of 97 patients when using the PCT-DRR IoU threshold of 0.2. The inference time per frame was 170 ms. The model outperformed the benchmark model on all metrics, and the comparison was significant (p 0.2 cases, but not over the undifferentiated 84 testing cases. Conclusions The implemented patient-specific refinement approach based on a pre-trained baseline model was shown to be applicable to markerless tumor tracking in simulated radiographs for lung cases

On the robustness of multilateration of ionoacoustic signals for localization of the Bragg peak at pre-clinical proton beam energies in water

Objectives. The energy deposited in a medium by a pulsed proton beam results in the emission of thermoacoustic waves, also called ionoacoustics (IA). The proton beam stopping position (Bragg peak) can be retrieved from a time-of-flight analysis (ToF) of IA signals acquired at different sensor locations (multilateration). This work aimed to assess the robustness of multilateration methods in proton beams at pre-clinical energies for the development of a small animal irradiator. Approach. The accuracy of multilateration performed using different algorithms; namely, time of arrival and time difference of arrival, was investigated in-silico for ideal point sources in the presence of realistic uncertainties on the ToF estimation and ionoacoustic signals generated by a 20 MeV pulsed proton beam stopped in a homogeneous water phantom. The localisation accuracy was further investigated experimentally based on two different measurements with pulsed monoenergetic proton beams at energies of 20 and 22 MeV. Main results. It was found that the localisation accuracy mainly depends on the position of the acoustic detectors relative to the proton beam due to spatial variation of the error on the ToF estimation. By optimally positioning the sensors to reduce the ToF error, the Bragg peak could be located in-silico with an accuracy better than 90 μm (2 error). Localisation errors going up to 1 mm were observed experimentally due to inaccurate knowledge of the sensor positions and noisy ionoacoustic signals. Significance. This study gives a first overview of the implementation of different multilateration methods for ionoacoustics-based Bragg peak localisation in two- and three-dimensions at pre-clinical energies. Different sources of uncertainty were investigated, and their impact on the localisation accuracy was quantified in-silico and experimentally