Multimodal Noncontact Tracking of Surgical Instruments

For many procedures, open surgery is being replaced with minimally invasive surgical (MIS) techniques. The advantages of MIS include reduced operative trauma and fewer complications leading to faster patient recovery, better cosmetic results and shorter hospital stays. As the demand for MIS procedures increases, effective surgical training tools must be developed to improve procedure efficiency and patient safety. Motion tracking of laparoscopic instruments can provide objective skills assessment for novices and experienced users. The most common approaches to noncontact motion capture are optical and electromagnetic (EM) tracking systems, though each approach has operational limitations. Optical trackers are prone to occlusion and the performance of EM trackers degrades in the presence of magnetic and ferromagnetic material. The cost of these systems also limits their availability for surgical training and clinical environments. This thesis describes the development and validation of a novel, noncontact laparoscopic tracking system as an inexpensive alternative to current technology. This system is based on the fusion of inertial, magnetic and distance sensing to generate real-time, 6-DOF pose data. Orientation is estimated using a Kalman-filtered attitude-heading reference system (AHRS) and restricted motion at the trocar provides a datum from which position information can be recovered. The Inertial and Range-Enhanced Surgical (IRES) Tracker was prototyped, then validated using a MIS training box and by comparison to an EM tracking system. Results of IRES tracker testing showed similar performance to an EM tracker with position error as low as 1.25 mm RMS and orientation error \u3c0.58 degrees RMS along each axis. The IRES tracker also displayed greater precision and superior magnetic interference rejection capabilities. At a fraction of the cost of current laparoscopic tracking methods, the IRES tracking system would provide an excellent alternative for use in surgical training and skills assessment

Neurosurgical Ultrasound Pose Estimation Using Image-Based Registration and Sensor Fusion - A Feasibility Study

Modern neurosurgical procedures often rely on computer-assisted real-time guidance using multiple medical imaging modalities. State-of-the-art commercial products enable the fusion of pre-operative with intra-operative images (e.g., magnetic resonance [MR] with ultrasound [US] images), as well as the on-screen visualization of procedures in progress. In so doing, US images can be employed as a template to which pre-operative images can be registered, to correct for anatomical changes, to provide live-image feedback, and consequently to improve confidence when making resection margin decisions near eloquent regions during tumour surgery. In spite of the potential for tracked ultrasound to improve many neurosurgical procedures, it is not widely used. State-of-the-art systems are handicapped by optical tracking’s need for consistent line-of-sight, keeping tracked rigid bodies clean and rigidly fixed, and requiring a calibration workflow. The goal of this work is to improve the value offered by co-registered ultrasound images without the workflow drawbacks of conventional systems. The novel work in this thesis includes: the exploration and development of a GPU-enabled 2D-3D multi-modal registration algorithm based on the existing LC2 metric; and the use of this registration algorithm in the context of a sensor and image-fusion algorithm. The work presented here is a motivating step in a vision towards a heterogeneous tracking framework for image-guided interventions where the knowledge from intraoperative imaging, pre-operative imaging, and (potentially disjoint) wireless sensors in the surgical field are seamlessly integrated for the benefit of the surgeon. The technology described in this thesis, inspired by advances in robot localization demonstrate how inaccurate pose data from disjoint sources can produce a localization system greater than the sum of its parts

Quaternionic Attitude Estimation with Inertial Measuring Unit for Robotic and Human Body Motion Tracking using Sequential Monte Carlo Methods with Hyper-Dimensional Spherical Distributions

This dissertation examined the inertial tracking technology for robotics and human tracking applications. This is a multi-discipline research that builds on the embedded system engineering, Bayesian estimation theory, software engineering, directional statistics, and biomedical engineering. A discussion of the orientation tracking representations and fundamentals of attitude estimation are presented briefly to outline the some of the issues in each approach. In addition, a discussion regarding to inertial tracking sensors gives an insight to the basic science and limitations in each of the sensing components. An initial experiment was conducted with existing inertial tracker to study the feasibility of using this technology in human motion tracking. Several areas of improvement were made based on the results and analyses from the experiment. As the performance of the system relies on multiple factors from different disciplines, the only viable solution is to optimize the performance in each area. Hence, a top-down approach was used in developing this system. The implementations of the new generation of hardware system design and firmware structure are presented in this dissertation. The calibration of the system, which is one of the most important factors to minimize the estimation error to the system, is also discussed in details. A practical approach using sequential Monte Carlo method with hyper-dimensional statistical geometry is taken to develop the algorithm for recursive estimation with quaternions. An analysis conducted from a simulation study provides insights to the capability of the new algorithms. An extensive testing and experiments was conducted with robotic manipulator and free hand human motion to demonstrate the improvements with the new generation of inertial tracker and the accuracy and stability of the algorithm. In addition, the tracking unit is used to demonstrate the potential in multiple biomedical applications including kinematics tracking and diagnosis instrumentation. The inertial tracking technologies presented in this dissertation is aimed to use specifically for human motion tracking. The goal is to integrate this technology into the next generation of medical diagnostic system

Hip joint centre position estimation using a dual unscented Kalman filter for computer-assisted orthopaedic surgery

In computer-assisted knee surgery, the accuracy of the localization of the femur centre of rotation relative to the hip-bone (hip joint centre) is affected by the unavoidable and untracked pelvic movements because only the femoral pose is acquired during passive pivoting manoeuvres. We present a dual unscented Kalman filter algorithm that allows the estimation of the hip joint centre also using as input the position of a pelvic reference point that can be acquired with a skin marker placed on the hip, without increasing the invasiveness of the surgical procedure. A comparative assessment of the algorithm was carried out using data provided by in vitro experiments mimicking in vivo surgical conditions. Soft tissue artefacts were simulated and superimposed onto the position of a pelvic landmark. Femoral pivoting made of a sequence of star-like quasi-planar movements followed by a circumduction was performed. The dual unscented Kalman filter method proved to be less sensitive to pelvic displacements, which were shown to be larger during the manoeuvres in which the femur was more adducted. Comparable accuracy between all the analysed methods resulted for hip joint centre displacements smaller than 1 mm (error: 2.2 ± [0.2; 0.3] mm, median ± [inter-quartile range 25%; inter-quartile range 75%]) and between 1 and 6 mm (error: 4.8 ± [0.5; 0.8] mm) during planar movements. When the hip joint centre displacement exceeded 6 mm, the dual unscented Kalman filter proved to be more accurate than the other methods by 30% during multi-planar movements (error: 5.2 ± [1.2; 1] mm)

Constrained Stochastic State Estimation for 3D Shape Reconstruction of Catheters and Guidewires in Fluoroscopic Images

Minimally invasive fluoroscopy-based procedures are the gold standard for diagnosis and treatment of various pathologies of the cardiovascular system. This kind of procedures imply for the clinicians to infer the 3D shape of the device from 2D images, which is known to be an ill-posed problem. In this paper we present a method to reconstruct the 3D shape of the interventional device, with the aim of improving the navigation. The method combines a physics-based simulation with non-linear Bayesian filter. Whereas the physics-based model provides a prediction of the shape of the device navigating within the blood vessels (taking into account non-linear interactions between the catheter and the surrounding anatomy), an Unscented Kalman Filter is used to correct the navigation model using 2D image features as external observations. The proposed framework has been evaluated on both synthetic and real data, under different model parameterization, filter parameters tuning and external observations data-sets. Comparing the reconstructed 3D shape with a known ground truth, for the synthetic data-set, we obtained an average 3D Hausdorff distance of 0.07 ± 0.37 mm; the 3D distance at the tip equal to 0.021 ± 0.009 mm and the 3D mean distance at the distal segment of the catheter equal to 0.02 ± 0.008 mm. For the real data-set, the obtained average 3D Hausdorff Distance was of 0.95 ± 0.35 mm, the average 3D distance at the tip is equal to 0.7 ± 0.45 mm with an average 3D mean distance at the distal segment of 0.7 ± 0.46 mm. These results show the ability of our method to retrieve the 3D shape of the device, under a variety of filter parameterizations and challenging conditions: errors on the friction coefficient, ambiguous views and non-linear complex phenomena such as stick and slip motions

A Low-Cost Camera-based Transducer Tracking System for Freehand Three-Dimensional Ultrasound Imaging

Freehand three-dimensional ultrasound (3D US) imaging is commonly used for clinical diagnosis and therapy monitoring. In this technique, accurate tracking of the US transducer is a crucial requirement to develop high-quality 3D US volumes. However, current methods for transducer tracking are generally expensive and inconvenient. This thesis presents a low-cost camera-based system for tracking the US transducer with six degrees of freedom (DoF). In this system, two orthogonal cameras with non-overlapped views are mounted on the US transducer. During US scanning, the two cameras are employed to track artificial features attached to the skin of the patient. A 3D surface map is constructed based on the tracked features and the 3D poses of each camera with respect to the skin are extracted separately. The estimated poses of the two cameras are spatially combined to provide accurate and robust pose estimation of the US transducer. In particular, the fusion of the estimated poses by the two cameras is performed using Kalman filtering based technique, which is a popular optimization technique in motion guidance and tracking. The camera-based tracking of the US transducer has been applied to synthesize freehand 3D US volumes. The performance of the proposed system is evaluated by performing in-vitro 3D US imaging experiments and quantifying the synthesized US volumes. The results demonstrate that two points in the 3D US volume separated by a distance of 10 mm can be reconstructed with an average error of 0.35 mm and a 3D volume of a cylinder can be estimated within an error of 3.8%

Integration of Local Positioning System & Strapdown Inertial Navigation System for Hand-Held Tool Tracking

This research concerns the development of a smart sensory system for tracking a hand-held moving device to millimeter accuracy, for slow or nearly static applications over extended periods of time. Since different operators in different applications may use the system, the proposed design should provide the accurate position, orientation, and velocity of the object without relying on the knowledge of its operation and environment, and based purely on the motion that the object experiences. This thesis proposes the design of the integration a low-cost Local Positioning System (LPS) and a low-cost StrapDown Inertial Navigation System (SDINS) with the association of the modified EKF to determine 3D position and 3D orientation of a hand-held tool within a required accuracy. A hybrid LPS/SDINS combines and complements the best features of two different navigation systems, providing a unique solution to track and localize a moving object more precisely. SDINS provides continuous estimates of all components of a motion, but SDINS loses its accuracy over time because of inertial sensors drift and inherent noise. LPS has the advantage that it can possibly get absolute position and velocity independent of operation time; however, it is not highly robust, is computationally quite expensive, and exhibits low measurement rate. This research consists of three major parts: developing a multi-camera vision system as a reliable and cost-effective LPS, developing a SDINS for a hand-held tool, and developing a Kalman filter for sensor fusion. Developing the multi-camera vision system includes mounting the cameras around the workspace, calibrating the cameras, capturing images, applying image processing algorithms and features extraction for every single frame from each camera, and estimating the 3D position from 2D images. In this research, the specific configuration for setting up the multi-camera vision system is proposed to reduce the loss of line of sight as much as possible. The number of cameras, the position of the cameras with respect to each other, and the position and the orientation of the cameras with respect to the center of the world coordinate system are the crucial characteristics in this configuration. The proposed multi-camera vision system is implemented by employing four CCD cameras which are fixed in the navigation frame and their lenses placed on semicircle. All cameras are connected to a PC through the frame grabber, which includes four parallel video channels and is able to capture images from four cameras simultaneously. As a result of this arrangement, a wide circular field of view is initiated with less loss of line-of-sight. However, the calibration is more difficult than a monocular or stereo vision system. The calibration of the multi-camera vision system includes the precise camera modeling, single camera calibration for each camera, stereo camera calibration for each two neighboring cameras, defining a unique world coordinate system, and finding the transformation from each camera frame to the world coordinate system. Aside from the calibration procedure, digital image processing is required to be applied into the images captured by all four cameras in order to localize the tool tip. In this research, the digital image processing includes image enhancement, edge detection, boundary detection, and morphologic operations. After detecting the tool tip in each image captured by each camera, triangulation procedure and optimization algorithm are applied in order to find its 3D position with respect to the known navigation frame. In the SDINS, inertial sensors are mounted rigidly and directly to the body of the tracking object and the inertial measurements are transformed computationally to the known navigation frame. Usually, three gyros and three accelerometers, or a three-axis gyro and a three-axis accelerometer are used for implementing SDINS. The inertial sensors are typically integrated in an inertial measurement unit (IMU). IMUs commonly suffer from bias drift, scale-factor error owing to non-linearity and temperature changes, and misalignment as a result of minor manufacturing defects. Since all these errors lead to SDINS drift in position and orientation, a precise calibration procedure is required to compensate for these errors. The precision of the SDINS depends not only on the accuracy of calibration parameters but also on the common motion-dependent errors. The common motion-dependent errors refer to the errors caused by vibration, coning motion, sculling, and rotational motion. Since inertial sensors provide the full range of heading changes, turn rates, and applied forces that the object is experiencing along its movement, accurate 3D kinematics equations are developed to compensate for the common motion-dependent errors. Therefore, finding the complete knowledge of the motion and orientation of the tool tip requires significant computational complexity and challenges relating to resolution of specific forces, attitude computation, gravity compensation, and corrections for common motion-dependent errors. The Kalman filter technique is a powerful method for improving the output estimation and reducing the effect of the sensor drift. In this research, the modified EKF is proposed to reduce the error of position estimation. The proposed multi-camera vision system data with cooperation of the modified EKF assists the SDINS to deal with the drift problem. This configuration guarantees the real-time position and orientation tracking of the instrument. As a result of the proposed Kalman filter, the effect of the gravitational force in the state-space model will be removed and the error which results from inaccurate gravitational force is eliminated. In addition, the resulting position is smooth and ripple-free. The experimental results of the hybrid vision/SDINS design show that the position error of the tool tip in all directions is about one millimeter RMS. If the sampling rate of the vision system decreases from 20 fps to 5 fps, the errors are still acceptable for many applications

A Low Complexity 6DoF Magnetic Tracking System For Biomedical Applications

L'abstract è presente nell'allegato / the abstract is in the attachmen