A comparison of gantry-mounted x-ray-based real-time target tracking methods.

PURPOSE: Most modern radiotherapy machines are built with a 2D kV imaging system. Combining this imaging system with a 2D-3D inference method would allow for a ready-made option for real-time 3D tumor tracking. This work investigates and compares the accuracy of four existing 2D-3D inference methods using both motion traces inferred from external surrogates and measured internally from implanted beacons. METHOD: Tumor motion data from 160 fractions (46 thoracic/abdominal patients) of Synchrony traces (inferred traces), and 28 fractions (7 lung patients) of Calypso traces (internal traces) from the LIGHT SABR trial (NCT02514512) were used in this study. The motion traces were used as the ground truth. The ground truth trajectories were used in silico to generate 2D positions projected on the kV detector. These 2D traces were then passed to the 2D-3D inference methods: interdimensional correlation, Gaussian probability density function (PDF), arbitrary-shape PDF, and the Kalman filter. The inferred 3D positions were compared with the ground truth to determine tracking errors. The relationships between tracking error and motion magnitude, interdimensional correlation, and breathing periodicity index (BPI) were also investigated. RESULTS: Larger tracking errors were observed from the Calypso traces, with RMS and 95th percentile 3D errors of 0.84-1.25 mm and 1.72-2.64 mm, compared to 0.45-0.68 mm and 0.74-1.13 mm from the Synchrony traces. The Gaussian PDF method was found to be the most accurate, followed by the Kalman filter, the interdimensional correlation method, and the arbitrary-shape PDF method. Tracking error was found to strongly and positively correlate with motion magnitude for both the Synchrony and Calypso traces and for all four methods. Interdimensional correlation and BPI were found to negatively correlate with tracking error only for the Synchrony traces. The Synchrony traces exhibited higher interdimensional correlation than the Calypso traces especially in the anterior-posterior direction. CONCLUSION: Inferred traces often exhibit higher interdimensional correlation, which are not true representation of thoracic/abdominal motion and may underestimate kV-based tracking errors. The use of internal traces acquired from systems such as Calypso is advised for future kV-based tracking studies. The Gaussian PDF method is the most accurate 2D-3D inference method for tracking thoracic/abdominal targets. Motion magnitude has significant impact on 2D-3D inference error, and should be considered when estimating kV-based tracking error

Markerless Lung Tumor Trajectory Estimation from Rotating Cone Beam Computed Tomography Projections

Respiration introduces large tumor motion in the thoracic region which influences treatment outcome for lung cancer patients. Tumor motion management techniques require characterization of temporal tumor motions because tumor motion varies patient to patient, day to day and cycle to cycle. This work develops a markerless algorithm to estimate 3 dimensional (3D) lung-tumor trajectories on free breathing cone beam computed tomography (CBCT) projections, which are 2 dimensional (2D) sequential images rotating about an axis and are used to reconstruct 3D CBCT images. A gold standard tumor trajectory is required to guide the algorithm development and estimate the tumor detection accuracy for markerless tracking algorithms. However, a sufficient strategy to validate markerless tracking algorithms is lacking. A validation framework is developed based on fiducial markers. Markers are segmented and marker trajectories are xiv obtained. The displacement of the tumor to the marker is calculated and added to the segmented marker trajectory to generate reference tumor trajectory. Markerless tumor trajectory estimation (MLTM) algorithm is developed and improved to acquire tumor trajectory with clinical acceptable accuracy for locally advanced lung tumors. The development is separate into two parts. The first part considers none tumor deformation. It investigates shape and appearance of the template, moreover, a constraint method is introduced to narrow down the template matching searching region for more precise matching results. The second part is to accommodate tumor deformation near the end of the treatment. The accuracy of MLTM is calculated and compared against 4D CBCT, which is the current standard of care. In summary, a validation framework based on fiducial markers is successfully built. MLTM is successfully developed with or without the consideration of tumor deformation with promising accuracy. MLTM outperforms 4D CBCT in temporal tumor trajectory estimation

The Estimation and Correction of Rigid Motion in Helical Computed Tomography

X-ray CT is a tomographic imaging tool used in medicine and industry. Although technological developments have significantly improved the performance of CT systems, the accuracy of images produced by state-of-the-art scanners is still often limited by artefacts due to object motion. To tackle this problem, a number of motion estimation and compensation methods have been proposed. However, no methods with the demonstrated ability to correct for rigid motion in helical CT scans appear to exist. The primary aims of this thesis were to develop and evaluate effective methods for the estimation and correction of arbitrary six degree-of-freedom rigid motion in helical CT. As a first step, a method was developed to accurately estimate object motion during CT scanning with an optical tracking system, which provided sub-millimetre positional accuracy. Subsequently a motion correction method, which is analogous to a method previously developed for SPECT, was adapted to CT. The principle is to restore projection consistency by modifying the source-detector orbit in response to the measured object motion and reconstruct from the modified orbit with an iterative reconstruction algorithm. The feasibility of this method was demonstrated with a rapidly moving brain phantom, and the efficacy of correcting for a range of human head motions acquired from healthy volunteers was evaluated in simulations. The methods developed were found to provide accurate and artefact-free motion corrected images with most types of head motion likely to be encountered in clinical CT imaging, provided that the motion was accurately known. The method was also applied to CT data acquired on a hybrid PET/CT scanner demonstrating its versatility. Its clinical value may be significant by reducing the need for repeat scans (and repeat radiation doses), anesthesia and sedation in patient groups prone to motion, including young children

Spatial Programming for Industrial Robots through Task Demonstration

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks

Characterization of multiphase flows integrating X-ray imaging and virtual reality

Multiphase flows are used in a wide variety of industries, from energy production to pharmaceutical manufacturing. However, because of the complexity of the flows and difficulty measuring them, it is challenging to characterize the phenomena inside a multiphase flow. To help overcome this challenge, researchers have used numerous types of noninvasive measurement techniques to record the phenomena that occur inside the flow. One technique that has shown much success is X-ray imaging. While capable of high spatial resolutions, X-ray imaging generally has poor temporal resolution. This research improves the characterization of multiphase flows in three ways. First, an X-ray image intensifier is modified to use a high-speed camera to push the temporal limits of what is possible with current tube source X-ray imaging technology. Using this system, sample flows were imaged at 1000 frames per second without a reduction in spatial resolution. Next, the sensitivity of X-ray computed tomography (CT) measurements to changes in acquisition parameters is analyzed. While in theory CT measurements should be stable over a range of acquisition parameters, previous research has indicated otherwise. The analysis of this sensitivity shows that, while raw CT values are strongly affected by changes to acquisition parameters, if proper calibration techniques are used, acquisition parameters do not significantly influence the results for multiphase flow imaging. Finally, two algorithms are analyzed for their suitability to reconstruct an approximate tomographic slice from only two X-ray projections. These algorithms increase the spatial error in the measurement, as compared to traditional CT; however, they allow for very high temporal resolutions for 3D imaging. The only limit on the speed of this measurement technique is the image intensifier-camera setup, which was shown to be capable of imaging at a rate of at least 1000 FPS. While advances in measurement techniques for multiphase flows are one part of improving multiphase flow characterization, the challenge extends beyond measurement techniques. For improved measurement techniques to be useful, the data must be accessible to scientists in a way that maximizes the comprehension of the phenomena. To this end, this work also presents a system for using the Microsoft Kinect sensor to provide natural, non-contact interaction with multiphase flow data. Furthermore, this system is constructed so that it is trivial to add natural, non-contact interaction to immersive visualization applications. Therefore, multiple visualization applications can be built that are optimized to specific types of data, but all leverage the same natural interaction. Finally, the research is concluded by proposing a system that integrates the improved X-ray measurements, with the Kinect interaction system, and a CAVE automatic virtual environment (CAVE) to present scientists with the multiphase flow measurements in an intuitive and inherently three-dimensional manner

A Continuous Grasp Representation for the Imitation Learning of Grasps on Humanoid Robots

Models and methods are presented which enable a humanoid robot to learn reusable, adaptive grasping skills. Mechanisms and principles in human grasp behavior are studied. The findings are used to develop a grasp representation capable of retaining specific motion characteristics and of adapting to different objects and tasks. Based on the representation a framework is proposed which enables the robot to observe human grasping, learn grasp representations, and infer executable grasping actions

Surrogate-driven motion models from cone-beam CT for motion management in radiotherapy treatments

This thesis details a variety of methods to build a surrogate-driven motion model from a cone-beam CT (CBCT) scan. The methods are intended to form a key constituent of a tracked RT treatment system, by providing a markerless means of tracking tumour and organs at risk (OAR) positions in real-time. The beam can then be adjusted to account for the respiratory motion of the tumour, whilst ensuring no adverse e.ects on the OAR from the adjustment in the beam. An approach to describe an iterative method to markerlessly track the lung tumour region is presented. A motion model is built of the tumour region using the CBCT projections, which then gives tumour position information during treatment. For simulated data, the motion model was able to reduce the mean L2-norm error from 4.1 to 1.0 mm, relative to the mean position. The model was used to account for the motion of an object placed within a respiratory phantom. When used to perform a motion compensated reconstruction (MCR), measured dimensions of this object agreed to within the voxel size (1 mm cube) used for the reconstruction. The method was applied to 6 clinical datasets. Improvements in edge contrast of the tumour were seen, and compared to clinically-derived positions for the tumour centres, the mean absolute errors in superior-inferior directions was reduced to under 2.5 mm. The model is then subsequently extended to monitor both tumour and OAR regions during treatment. This extended approach uses both the planning 4DCT and CBCT scans, focusing on the strengths of each respective dataset. Results are presented on three simulated and three clinical datasets. For the simulated data, maximal L2-norm errors were reduced from 14.8 to 4.86 mm. Improvements in edge contrast in the diaphragm and lung regions were seen in the MCR for the clinical data. A final approach to building a model of the entire patient is then presented, utilising only the CBCT data. An optical-flow-based approach is taken, which is adapted to the unique nature of the CBCT data via some interesting conceptualisations. Results on a simulated case are presented, showing increased edge contrast in the MCR using the fitted motion model. Mean L2-norm errors in the tumour region were reduced from 4.2 to 2.6 mm. Future work is discussed, with a variety of extensions to the methods proposed. With further development, it is hoped that some of the ideas detailed could be translated into the clinic and have a direct impact on patient treatment

Intraoperative Endoscopic Augmented Reality in Third Ventriculostomy

In neurosurgery, as a result of the brain-shift, the preoperative patient models used as a intraoperative reference change. A meaningful use of the preoperative virtual models during the operation requires for a model update. The NEAR project, Neuroendoscopy towards Augmented Reality, describes a new camera calibration model for high distorted lenses and introduces the concept of active endoscopes endowed with with navigation, camera calibration, augmented reality and triangulation modules

IMAGE-BASED RESPIRATORY MOTION EXTRACTION AND RESPIRATION-CORRELATED CONE BEAM CT (4D-CBCT) RECONSTRUCTION

Accounting for respiration motion during imaging helps improve targeting precision in radiation therapy. Respiratory motion can be a major source of error in determining the position of thoracic and upper abdominal tumor targets during radiotherapy. Thus, extracting respiratory motion is a key task in radiation therapy planning. Respiration-correlated or four-dimensional CT (4DCT) imaging techniques have been recently integrated into imaging systems for verifying tumor position during treatment and managing respiration-induced tissue motion. The quality of the 4D reconstructed volumes is highly affected by the respiratory signal extracted and the phase sorting method used. This thesis is divided into two parts. In the first part, two image-based respiratory signal extraction methods are proposed and evaluated. Those methods are able to extract the respiratory signals from CBCT images without using external sources, implanted markers or even dependence on any structure in the images such as the diaphragm. The first method, called Local Intensity Feature Tracking (LIFT), extracts the respiratory signal depending on feature points extracted and tracked through the sequence of projections. The second method, called Intensity Flow Dimensionality Reduction (IFDR), detects the respiration signal by computing the optical flow motion of every pixel in each pair of adjacent projections. Then, the motion variance in the optical flow dataset is extracted using linear and non-linear dimensionality reduction techniques to represent a respiratory signal. Experiments conducted on clinical datasets showed that the respiratory signal was successfully extracted using both proposed methods and it correlates well with standard respiratory signals such as diaphragm position and the internal markers’ signal. In the second part of this thesis, 4D-CBCT reconstruction based on different phase sorting techniques is studied. The quality of the 4D reconstructed images is evaluated and compared for different phase sorting methods such as internal markers, external markers and image-based methods (LIFT and IFDR). Also, a method for generating additional projections to be used in 4D-CBCT reconstruction is proposed to reduce the artifacts that result when reconstructing from an insufficient number of projections. Experimental results showed that the feasibility of the proposed method in recovering the edges and reducing the streak artifacts