Online 4D ultrasound guidance for real-time motion compensation by MLC tracking

PURPOSE: With the trend in radiotherapy moving toward dose escalation and hypofractionation, the need for highly accurate targeting increases. While MLC tracking is already being successfully used for motion compensation of moving targets in the prostate, current real-time target localization methods rely on repeated x-ray imaging and implanted fiducial markers or electromagnetic transponders rather than direct target visualization. In contrast, ultrasound imaging can yield volumetric data in real-time (3D + time = 4D) without ionizing radiation. The authors report the first results of combining these promising techniques-online 4D ultrasound guidance and MLC tracking-in a phantom. METHODS: A software framework for real-time target localization was installed directly on a 4D ultrasound station and used to detect a 2 mm spherical lead marker inside a water tank. The lead marker was rigidly attached to a motion stage programmed to reproduce nine characteristic tumor trajectories chosen from large databases (five prostate, four lung). The 3D marker position detected by ultrasound was transferred to a computer program for MLC tracking at a rate of 21.3 Hz and used for real-time MLC aperture adaption on a conventional linear accelerator. The tracking system latency was measured using sinusoidal trajectories and compensated for by applying a kernel density prediction algorithm for the lung traces. To measure geometric accuracy, static anterior and lateral conformal fields as well as a 358° arc with a 10 cm circular aperture were delivered for each trajectory. The two-dimensional (2D) geometric tracking error was measured as the difference between marker position and MLC aperture center in continuously acquired portal images. For dosimetric evaluation, VMAT treatment plans with high and low modulation were delivered to a biplanar diode array dosimeter using the same trajectories. Dose measurements with and without MLC tracking were compared to a static reference dose using 3%/3 mm and 2%/2 mm γ-tests. RESULTS: The overall tracking system latency was 172 ms. The mean 2D root-mean-square tracking error was 1.03 mm (0.80 mm prostate, 1.31 mm lung). MLC tracking improved the dose delivery in all cases with an overall reduction in the γ-failure rate of 91.2% (3%/3 mm) and 89.9% (2%/2 mm) compared to no motion compensation. Low modulation VMAT plans had no (3%/3 mm) or minimal (2%/2 mm) residual γ-failures while tracking reduced the γ-failure rate from 17.4% to 2.8% (3%/3 mm) and from 33.9% to 6.5% (2%/2 mm) for plans with high modulation. CONCLUSIONS: Real-time 4D ultrasound tracking was successfully integrated with online MLC tracking for the first time. The developed framework showed an accuracy and latency comparable with other MLC tracking methods while holding the potential to measure and adapt to target motion, including rotation and deformation, noninvasively

A Novel Imaging System for Automatic Real-Time 3D Patient-Specific Knee Model Reconstruction Using Ultrasound RF Data

This dissertation introduces a novel imaging method and system for automatic real-time 3D patient-specific knee model reconstruction using ultrasound RF data. The developed method uses ultrasound to transcutaneously digitize a point cloud representing the bone’s surface. This point cloud is then used to reconstruct 3D bone model using deformable models method. In this work, three systems were developed for 3D knee bone model reconstruction using ultrasound RF data. The first system uses tracked single-element ultrasound transducer, and was experimented on 12 knee phantoms. An average reconstruction accuracy of 0.98 mm was obtained. The second system was developed using an ultrasound machine which provide real-time access to the ultrasound RF data, and was experimented on 2 cadaveric distal femurs, and proximal tibia. An average reconstruction accuracy of 0.976 mm was achieved. The third system was developed as an extension of the second system, and was used for clinical study of the developed system further assess its accuracy and repeatability. A knee scanning protocol was developed to scan the different articular surfaces of the knee bones to reconstruct 3D model of the bone without the need for bone-implanted motion tracking reference probes. The clinical study was performed on 6 volunteers’ knees. Average reconstruction accuracy of 0.88 mm was achieved with 93.5% repeatability. Three extensions to the developed system were investigated for future work. The first extension is 3D knee injection guidance system. A prototype for the 3D injection guidance system was developed to demonstrate the feasibility of the idea. The second extension in a knee kinematics tracking system using A-mode ultrasound. A simulation framework was developed to study the feasibility of the idea, and to find the best number of single-element ultrasound transducers and their spatial distribution that yield the highest kinematics tracking accuracy. The third extension is 3D cartilage model reconstruction. A preliminary method for cartilage echo detection from ultrasound RF data was developed, and experimented on the distal femur scans of one of the clinical study’s volunteers to reconstruct a 3D point cloud for the cartilage

An Inertial-Optical Tracking System for Quantitative, Freehand, 3D Ultrasound

Three dimensional (3D) ultrasound has become an increasingly popular medical imaging tool over the last decade. It offers significant advantages over Two Dimensional (2D) ultrasound, such as improved accuracy, the ability to display image planes that are physically impossible with 2D ultrasound, and reduced dependence on the skill of the sonographer. Among 3D medical imaging techniques, ultrasound is the only one portable enough to be used by first responders, on the battlefield, and in rural areas. There are three basic methods of acquiring 3D ultrasound images. In the first method, a 2D array transducer is used to capture a 3D volume directly, using electronic beam steering. This method is mainly used for echocardiography. In the second method, a linear array transducer is mechanically actuated, giving a slower and less expensive alternative to the 2D array. The third method uses a linear array transducer that is moved by hand. This method is known as freehand 3D ultrasound. Whether using a 2D array or a mechanically actuated linear array transducer, the position and orientation of each image is known ahead of time. This is not the case for freehand scanning. To reconstruct a 3D volume from a series of 2D ultrasound images, assumptions must be made about the position and orientation of each image, or a mechanism for detecting the position and orientation of each image must be employed. The most widely used method for freehand 3D imaging relies on the assumption that the probe moves along a straight path with constant orientation and speed. This method requires considerable skill on the part of the sonographer. Another technique uses features within the images themselves to form an estimate of each image\u27s relative location. However, these techniques are not well accepted for diagnostic use because they are not always reliable. The final method for acquiring position and orientation information is to use a six Degree-of-Freedom (6 DoF) tracking system. Commercially available 6 DoF tracking systems use magnetic fields, ultrasonic ranging, or optical tracking to measure the position and orientation of a target. Although accurate, all of these systems have fundamental limitations in that they are relatively expensive and they all require sensors or transmitters to be placed in fixed locations to provide a fixed frame of reference. The goal of the work presented here is to create a probe tracking system for freehand 3D ultrasound that does not rely on any fixed frame of reference. This system tracks the ultrasound probe using only sensors integrated into the probe itself. The advantages of such a system are that it requires no setup before it can be used, it is more portable because no extra equipment is required, it is immune from environmental interference, and it is less expensive than external tracking systems. An ideal tracking system for freehand 3D ultrasound would track in all 6 DoF. However, current sensor technology limits this system to five. Linear transducer motion along the skin surface is tracked optically and transducer orientation is tracked using MEMS gyroscopes. An optical tracking system was developed around an optical mouse sensor to provide linear position information by tracking the skin surface. Two versions were evaluated. One included an optical fiber bundle and the other did not. The purpose of the optical fiber is to allow the system to integrate more easily into existing probes by allowing the sensor and electronics to be mounted away from the scanning end of the probe. Each version was optimized to track features on the skin surface while providing adequate Depth Of Field (DOF) to accept variation in the height of the skin surface. Orientation information is acquired using a 3 axis MEMS gyroscope. The sensor was thoroughly characterized to quantify performance in terms of accuracy and drift. This data provided a basis for estimating the achievable 3D reconstruction accuracy of the complete system. Electrical and mechanical components were designed to attach the sensor to the ultrasound probe in such a way as to simulate its being embedded in the probe itself. An embedded system was developed to perform the processing necessary to translate the sensor data into probe position and orientation estimates in real time. The system utilizes a Microblaze soft core microprocessor and a set of peripheral devices implemented in a Xilinx Spartan 3E field programmable gate array. The Xilinx Microkernel real time operating system performs essential system management tasks and provides a stable software platform for implementation of the inertial tracking algorithm. Stradwin 3D ultrasound software was used to provide a user interface and perform the actual 3D volume reconstruction. Stradwin retrieves 2D ultrasound images from the Terason t3000 portable ultrasound system and communicates with the tracking system to gather position and orientation data. The 3D reconstruction is generated and displayed on the screen of the PC in real time. Stradwin also provides essential system features such as storage and retrieval of data, 3D data interaction, reslicing, manual 3D segmentation, and volume calculation for segmented regions. The 3D reconstruction performance of the system was evaluated by freehand scanning a cylindrical inclusion in a CIRS model 044 ultrasound phantom. Five different motion profiles were used and each profile was repeated 10 times. This entire test regimen was performed twice, once with the optical tracking system using the optical fiber bundle, and once with the optical tracking system without the optical fiber bundle. 3D reconstructions were performed with and without the position and orientation data to provide a basis for comparison. Volume error and surface error were used as the performance metrics. Volume error ranged from 1.3% to 5.3% with tracking information versus 15.6% to 21.9% without for the version of the system without the optical fiber bundle. Volume error ranged from 3.7% to 7.6% with tracking information versus 8.7% to 13.7% without for the version of the system with the optical fiber bundle. Surface error ranged from 0.319 mm RMS to 0.462 mm RMS with tracking information versus 0.678 mm RMS to 1.261 mm RMS without for the version of the system without the optical fiber bundle. Surface error ranged from 0.326 mm RMS to 0.774 mm RMS with tracking information versus 0.538 mm RMS to 1.657 mm RMS without for the version of the system with the optical fiber bundle. The prototype tracking system successfully demonstrated that accurate 3D ultrasound volumes can be generated from 2D freehand data using only sensors integrated into the ultrasound probe. One serious shortcoming of this system is that it only tracks 5 of the 6 degrees of freedom required to perform complete 3D reconstructions. The optical system provides information about linear movement but because it tracks a surface, it cannot measure vertical displacement. Overcoming this limitation is the most obvious candidate for future research using this system. The overall tracking platform, meaning the embedded tracking computer and the PC software, developed and integrated in this work, is ready to take advantage of vertical displacement data, should a method be developed for sensing it

Real-Time Ultrasound Image-Guidance and Tracking in External Beam Radiotherapy

Background and Purpose - To evaluate the accuracy of Clarity (clinical version) system by using ultrasound phantom and some probe position. - To evaluate the intrafraction motion of prostate by collecting and analyzing ultrasound monitoring data from some patients. - To evaluate the accuracy of Clarity (Anticosti) system by using 3D phantom programmed with sinusoidal and breathing movement patterns to simulate computer-controlled based breath-hold phases interspersed with spontaneous breathing. - To evaluate the clinical applicability of Clarity (Anticosti) system for liver cases in healthy volunteers. The tracking results of healthy volunteers were compared to surface marker. - To evaluate the intrafraction motion during breath-hold in liver case by collecting and analyzing US monitoring data from some patients. Material and Methods The accuracy of Clarity (clinical version) system was evaluated using ultrasound phantom and some probe position. Two different probes were used: transabdominal ultrasound (TAUS) and transperineal ultrasound (TPUS) probe. Two positions of the phantom were used for TPUS, the vertical and the horizontal position. Intrafraction motion assessment of the prostate was based on continuous position monitoring with a 4D US system along the three directions; left(+)-right (LR), anterior(+)-posterior (AP), and inferior(+)-superior (SI). 770 US monitoring sessions in 38 prostate cancer patients’ normo-fractionated VMAT treatment series were retrospectively evaluated. The overall mean values and standard deviations (SD) along with random and systematic SDs were computed. The tracking accuracy of the research 4D US system was evaluated using two motion phantoms programmed with sinusoidal and breathing patterns to simulate free breathing and DIBH. The clinical performance was evaluated with 5 healthy volunteers. US datasets were acquired in computer-controlled DIBH with varying angular scanning angles. Tracked structures were renal pelvis (spherical structure) and portal/liver vein branches (non-spherical structure). An external marker was attached to the surface of both phantoms and volunteers as a secondary tracked object by an infrared camera for comparison. Residual intrafractional motion of DIBH tracking target relative to beginning position in each breath-hold plateau region was analysed along three directions; superior-inferior (SI), left-right (LR) and anterior-posterior (AP). 12 PTVs of 11 patients with primary/secondary liver tumours or adrenal gland/spleen metastases of diverse primaries were irradiated with SBRT in DIBH. Real time tracking of target or neighbouring surrogate structures was performed additionally using 4D US system during CBCT acquisition after permission of local IRB. Results The geometric positioning tolerance for Clarity-Sim and Clarity-Guide is 1 mm according to the manufacturer’s specifications. The results showed that all phantom and probe combinations met this criterion. The mean duration of each prostate monitoring session was 254s. The mean (μ), the systematic error () and the random error (σ) of intrafraction prostate motion were μ=(0.01, -0.08, 0.15)mm, =(0.30, 0.34, 0.23)mm and σ=(0.59, 0.73, 0.64)mm in LR, AP and SI direction, respectively. The percentage of treatments for which prostate motion was 2mm was present in about 0.67% of the data. The percentage increased to 2.42%, 6.14%, and 9.35% at 120s, 180s and 240s, respectively. The phantom measurements using Clarity (Anticosti) system showed increasing accuracy of US tracking with decreasing scanning range. The probability of lost tracking was higher for small scanning ranges (43.09% (10°) and 13.54% (20°)).The tracking success rates in healthy volunteers during DIBH were 93.24% and 89.86% for renal pelvis and portal vein branches, respectively. There was a strong correlation between the motion of the marker and the US tracking for the majority of analyzed breath-holds. 84.06% and 88.41% of renal pelvis target results and 82.26% and 91.94% of liver vein target results in AP and SI direction, the Pearson correlation coefficient was between 0.71 and 0.99. For evaluation of the intrafraction motion during breath-hold, 680 individual BHs during 93 treatment fractions were analysed. On visual control of tracking movies, target was lost in 27.9% of tracking, leaving a total of 490 BHs with optimal tracking. During these BHs, mean(+SD) target displacement were 1.7(+0.8)mm, 0.9(+0.4)mm, 2.2(+1.0)mm and 3.2(+1.0)mm for SI, LR, AP and 3D vector, respectively. Most of target displacement was below 2mm with percentage of 64.6%, 88.1% and 60.5% for SI, LR and AP, respectively. Data percentage of large target displacement increased with added BH time. At 5s, 3D vector of target displacement >10mm could be observed in 0.1% of data. Percentage values increased to 0.2%, 0.6%, and 1.1% at 10s, 15s and 20s, respectively. Conclusions The 4D US system offers a non-invasive method for online organ motion monitoring without additional ionizing radiation dose to the patient. The magnitudes of intrafraction prostate motion along the SI and AP directions were comparable. On average, the smallest motion was in the LR direction and the largest in AP direction. Most of the prostate displacements were within a few millimeters. However, with increased treatment time, larger 3D vector prostate displacements up to 18.30 mm could be observed. Shortening the treatment time can reduce the intrafractional motion and its effects and US monitoring can help to maximize treatment precision particularly in hypofractionated treatment regimens. For organ monitoring during BH application, the 4D US system showed a good performance and tracking accuracy in a 4D motion phantom when tracking a target that moves in accordance to a simulating breathing pattern. A 30°scanning range turned out to be an optimal parameter to track along with respiratory motion considering the accuracy of tracking and the possible loss of the tracked structure. The ultrasound tracking system is also applicable to a clinical setup with the tested hardware solution. The tracking capability of surrogate structures for upper abdominal lesions in DIBH is promising but needs further investigation in a larger cohort of patients. Ultrasound motion data show a strong correlation with surface motion data for most of individual breath-holds. Further improvement of the tracking algorithm is suggested to improve accuracy along with respiratory motion if using larger scanning angles for detection of high-amplitude motion and non-linear transformations of the tracking target. The exact quantification of residual motion impact requires an in-depth analysis of time spent at every position, nevertheless mean residual motion during DIBH is low and predominant direction is SI and AP. Only infrequently larger displacements of 3D vector >1 cm were observed, for short periods. Beam interruption at predefined thresholds could take DIBH treatments close to perfection. Key words: Medical Physics, 4D ultrasound, IGRT (image-guided radiotherapy), prostate motion, stereotactic body radiotherapy (SBRT), deep inspiratory breath-hold (DIBH)

Quasiperiodic predictive filtering for robot-assisted beating heart surgery

Beating heart procedures promise significant health benefits to patients but the fast motion of the heart poses a serious challenge to the surgeon. Robotic motion synchronization to heart movements could facilitate these surgeries, although for intracardiac procedures this requires the development of a predictive filter to compensate for the measurement noise and time delay present in 3D ultrasound imaging. In this paper, we present a quasiperiodic cardiac motion model and apply the extended Kalman filter to estimation of its parameters in real-time. We experimentally demonstrate high accuracy robot tracking to heart motion using this filter.Engineering and Applied Science

Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation

In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is required for subsurface visualisation to characterise the state of the tissue. However, scanning of large tissue surfaces in the presence of deformation is a challenging task for the surgeon. Recently, robot-assisted local tissue scanning has been investigated for motion stabilisation of imaging probes to facilitate the capturing of good quality images and reduce the surgeon's cognitive load. Nonetheless, these approaches require the tissue surface to be static or deform with periodic motion. To eliminate these assumptions, we propose a visual servoing framework for autonomous tissue scanning, able to deal with free-form tissue deformation. The 3D structure of the surgical scene is recovered and a feature-based method is proposed to estimate the motion of the tissue in real-time. A desired scanning trajectory is manually defined on a reference frame and continuously updated using projective geometry to follow the tissue motion and control the movement of the robotic arm. The advantage of the proposed method is that it does not require the learning of the tissue motion prior to scanning and can deal with free-form deformation. We deployed this framework on the da Vinci surgical robot using the da Vinci Research Kit (dVRK) for Ultrasound tissue scanning. Since the framework does not rely on information from the Ultrasound data, it can be easily extended to other probe-based imaging modalities.Comment: 7 pages, 5 figures, ICRA 202

Respiratory organ motion in interventional MRI : tracking, guiding and modeling

Respiratory organ motion is one of the major challenges in interventional MRI, particularly in interventions with therapeutic ultrasound in the abdominal region. High-intensity focused ultrasound found an application in interventional MRI for noninvasive treatments of different abnormalities. In order to guide surgical and treatment interventions, organ motion imaging and modeling is commonly required before a treatment start. Accurate tracking of organ motion during various interventional MRI procedures is prerequisite for a successful outcome and safe therapy. In this thesis, an attempt has been made to develop approaches using focused ultrasound which could be used in future clinically for the treatment of abdominal organs, such as the liver and the kidney. Two distinct methods have been presented with its ex vivo and in vivo treatment results. In the first method, an MR-based pencil-beam navigator has been used to track organ motion and provide the motion information for acoustic focal point steering, while in the second approach a hybrid imaging using both ultrasound and magnetic resonance imaging was combined for advanced guiding capabilities. Organ motion modeling and four-dimensional imaging of organ motion is increasingly required before the surgical interventions. However, due to the current safety limitations and hardware restrictions, the MR acquisition of a time-resolved sequence of volumetric images is not possible with high temporal and spatial resolution. A novel multislice acquisition scheme that is based on a two-dimensional navigator, instead of a commonly used pencil-beam navigator, was devised to acquire the data slices and the corresponding navigator simultaneously using a CAIPIRINHA parallel imaging method. The acquisition duration for four-dimensional dataset sampling is reduced compared to the existing approaches, while the image contrast and quality are improved as well. Tracking respiratory organ motion is required in interventional procedures and during MR imaging of moving organs. An MR-based navigator is commonly used, however, it is usually associated with image artifacts, such as signal voids. Spectrally selective navigators can come in handy in cases where the imaging organ is surrounding with an adipose tissue, because it can provide an indirect measure of organ motion. A novel spectrally selective navigator based on a crossed-pair navigator has been developed. Experiments show the advantages of the application of this novel navigator for the volumetric imaging of the liver in vivo, where this navigator was used to gate the gradient-recalled echo sequence

First evaluation of the feasibility of MLC tracking using ultrasound motion estimation.

Purpose To quantify the performance of the Clarity ultrasound (US) imaging system (Elekta AB, Stockholm, Sweden) for real-time dynamic multileaf collimator (MLC) tracking.Methods The Clarity calibration and quality assurance phantom was mounted on a motion platform moving with a periodic sine wave trajectory. The detected position of a 30 mm hypoechogenic sphere within the phantom was continuously reported via Clarity's real-time streaming interface to an in-house tracking and delivery software and subsequently used to adapt the MLC aperture. A portal imager measured MV treatment field/MLC apertures and motion platform positions throughout each experiment to independently quantify system latency and geometric error. Based on the measured range of latency values, a prostate stereotactic body radiation therapy (SBRT) delivery was performed with three realistic motion trajectories. The dosimetric impact of system latency on MLC tracking was directly measured using a 3D dosimeter mounted on the motion platform.Results For 2D US imaging, the overall system latency, including all delay times from the imaging and delivery chain, ranged from 392 to 424 ms depending on the lateral sector size. For 3D US imaging, the latency ranged from 566 to 1031 ms depending on the elevational sweep. The latency-corrected geometric root-mean squared error was below 0.75 mm (2D US) and below 1.75 mm (3D US). For the prostate SBRT delivery, the impact of a range of system latencies (400-1000 ms) on the MLC tracking performance was minimal in terms of gamma failure rate.Conclusions Real-time MLC tracking based on a noninvasive US input is technologically feasible. Current system latencies are higher than those for x-ray imaging systems, but US can provide full volumetric image data and the impact of system latency was measured to be small for a prostate SBRT case when using a US-like motion input

EchoFusion: Tracking and Reconstruction of Objects in 4D Freehand Ultrasound Imaging without External Trackers

Ultrasound (US) is the most widely used fetal imaging technique. However, US images have limited capture range, and suffer from view dependent artefacts such as acoustic shadows. Compounding of overlapping 3D US acquisitions into a high-resolution volume can extend the field of view and remove image artefacts, which is useful for retrospective analysis including population based studies. However, such volume reconstructions require information about relative transformations between probe positions from which the individual volumes were acquired. In prenatal US scans, the fetus can move independently from the mother, making external trackers such as electromagnetic or optical tracking unable to track the motion between probe position and the moving fetus. We provide a novel methodology for image-based tracking and volume reconstruction by combining recent advances in deep learning and simultaneous localisation and mapping (SLAM). Tracking semantics are established through the use of a Residual 3D U-Net and the output is fed to the SLAM algorithm. As a proof of concept, experiments are conducted on US volumes taken from a whole body fetal phantom, and from the heads of real fetuses. For the fetal head segmentation, we also introduce a novel weak annotation approach to minimise the required manual effort for ground truth annotation. We evaluate our method qualitatively, and quantitatively with respect to tissue discrimination accuracy and tracking robustness.Comment: MICCAI Workshop on Perinatal, Preterm and Paediatric Image analysis (PIPPI), 201

Medical image computing and computer-aided medical interventions applied to soft tissues. Work in progress in urology

Until recently, Computer-Aided Medical Interventions (CAMI) and Medical Robotics have focused on rigid and non deformable anatomical structures. Nowadays, special attention is paid to soft tissues, raising complex issues due to their mobility and deformation. Mini-invasive digestive surgery was probably one of the first fields where soft tissues were handled through the development of simulators, tracking of anatomical structures and specific assistance robots. However, other clinical domains, for instance urology, are concerned. Indeed, laparoscopic surgery, new tumour destruction techniques (e.g. HIFU, radiofrequency, or cryoablation), increasingly early detection of cancer, and use of interventional and diagnostic imaging modalities, recently opened new challenges to the urologist and scientists involved in CAMI. This resulted in the last five years in a very significant increase of research and developments of computer-aided urology systems. In this paper, we propose a description of the main problems related to computer-aided diagnostic and therapy of soft tissues and give a survey of the different types of assistance offered to the urologist: robotization, image fusion, surgical navigation. Both research projects and operational industrial systems are discussed