Online model estimation and haptic characterization for robotic-assisted minimally invasive surgery

Online soft tissue characterization is important for robotic-assisted minimally invasive surgery (RAMIS) to achieve a precise and stable robotic control with haptic feedback. The traditional linear regression method (i.e. the recursive least square (RLS) method) is inappropriate to handle nonlinear Hunt-Crossley (H-C) model since its linearization process involves unacceptable errors. This thesis presents a new nonlinear estimation method for online soft tissue characterization. To deal with nonlinear and dynamic conditions involved in soft tissue characterization, the approach expands the nonlinearity and dynamics of the H-C model by treating parameter p as an independent variable. Based on this, an unscented Kalman filter (UKF) was adapted for online nonlinear soft tissue characterization. A comparison analysis of the UKF and RLS methods was conducted to validate the performance of the UKF-based method. The UKF-based method suffers from two major problems. The first one is that it requires prior noise statistics of the corresponding system to be precisely known. However, due to uncertainties in the dynamic environment of RAMIS, it is difficult to accurately describe noise characteristics. This leads to biased or even divergent UKF solutions. Therefore, in order to attain accurate estimation results from the UKF-based approach, it is necessary to estimate noise statistics online to restrain the disturbance of noise uncertainty. Secondly, the UKF performance depends on the pre-defined system and measurement models. If the models involve stochastic errors, the UKF-based solution will be unstable. In fact, the measurement model’s accuracy can be guaranteed by using high-precision measurement equipment together with a high volume of available measurement data. On the other hand, the system model is more often involved with the inaccuracy problem. In RAMIS, the system model is a theoretical approximation of the physical contact between robotic tool and biological soft tissue. The approximation is intended to fulfil the requirement of real-time performance in RAMIS. Therefore, it is essential to improve the UKF performance in the presence of system model (the contact model) uncertainty. To address the UKF problem for inaccurate noise statistics, this thesis further presents a new recursive adaptive UKF (RAUKF) method for online nonlinear soft tissue characterization. It was developed, based on the H-C model, to estimate system noise statistics in real-time with windowing approximation. The method was developed under the condition that system noises are of small variation. In order to account for the inherent relationship between the current and previous states of soft tissue deformation involved in RAMIS, a recursive formulation was further constructed by introducing a fading scaling factor. This factor was further modified to accommodate noise statistics of a large variation, which may be caused by rupture events or geometric discontinuities in RAMIS. Simulations and comparison analyses verified the performance of the proposed RAUKF. The second UKF limitation regarding the requirement of the accurate system model was also addressed. A random weighting strong tracking unscented Kalman filter (RWSTUKF) was developed based on the Hunt-Crossley model for online nonlinear soft tissue characterization. This RWSTUKF overcomes the problem of performance degradation in the UKF due to system model errors. It adopts a scaling factor in the predicted state covariance to compensate the inaccuracy of the system model. This scaling factor was derived by combining the orthogonality principle with the random weighting concept to prevent the cumbersome computation from Jacobian matrix and offer the reliable estimation for innovation covariances. Simulation and comparison analyses demonstrated that the proposed RWSTUKF can characterise soft tissue parameters in the presence of system model error for RAMIS in on online mode. Using the proposed methods, a master-slave robotic system has been developed with a nonlinear state observer for soft tissue characterization. Robotic indentation and needle insertion tests conducted to evaluate performances of the proposed methods. Further, a rupture detection approach was established based on the RWSTUKF. It was also integrated into the master-slave robotic system to detect rupture events occurred during needle insertion. The experiment results demonstrated that the RWSTUKF outperforms RLS, UKF and RAUKF for soft tissue characterization

Doctor of Philosophy

dissertationClosed-loop control of wireless capsule endoscopes is an active area of research because it would drastically improve screening of the gastrointestinal tract. Traditional endoscopic procedures are unable to view the entire gastrointestinal tract and current commercial wireless capsule endoscopes are limited in their effectiveness due to their passive nature. This dissertation advances the field of active capsule endoscopy by developing methods to localize the full six-degree-of-freedom (6-DOF) pose of a screw-type magnetic capsule while it is being propelled through a lumen (such as the small intestines) using an external rotating magnetic dipole. The same external magnetic dipole is utilized for both propulsion and localization. Hardware was designed and constructed to enable testing of the magnetic localization and propulsion methods, including a robotic end-effector used as the external actuator magnet, and a prototype capsule embedded with Hall-effect sensors. Due to the use of a rotating magnetic field for propulsion, at any given time, the capsule can be in one of three regimes: synchronously rotating with the applied field, in "step-out" where it is free to move but the external field is rotating too quickly for the capsule to remain synchronously rotating, or completely stationary. We show that it is only necessary to distinguish whether or not the capsule is synchronously rotating (i.e., a single localization method can be used for a capsule in either the step-out or stationary regimes). Two magnetic localization methods are developed. The first uses nonlinear least squares to estimate the capsule's pose when it has no (or approximately no) net motion (e.g., to find the initial capsule pose or when it is stuck in an intestinal fold). The second method estimates the 6-DOF capsule pose as it synchronously rotates with the applied magnetic field using a square-root variant of the Unscented Kalman filter. A simple process model is adopted that restricts the capsule's movement to translation along and rotation about its principle axis. The capsule is actively propelled forward or backward, but it is not actively steered, rather, steering is provided by the lumen. The propulsion parameters that transform magnetic force and torque to the capsule's spatial velocity and angular velocity are estimated with an additional square-root Unscented Kalman filter to enable the capsule to navigate heterogeneous environments such as the small intestines. An optimized localization-propulsion system is described using the two localization algorithms and prior work in screw-type magnetic capsule propulsion with a single rotating dipole field. The capsule's regime is determined and the corresponding localization method is employed. Based on the capsule's estimated pose and the current estimates of its propulsion parameters, the actuator magnet's pose relative to the capsule is optimized to maximize the capsule's forward propulsion. Using this system, our prototype magnetic capsule successfully completed U-shaped and S-shaped trajectories in fresh bovine intestines with an average forward velocity of 5.5mm/s and 3.5 mm/s, respectively. At this rate it would take approximately 18-30 minutes to traverse the 6 meters of a typical human small intestine

Planification de l’ablation radiofréquence des arythmies cardiaques en combinant modélisation et apprentissage automatique

Cardiac arrhythmias are heart rhythm disruptions which can lead to sudden cardiac death. They require a deeper understanding for appropriate treatment planning. In this thesis, we integrate personalized structural and functional data into a 3D tetrahedral mesh of the biventricular myocardium. Next, the Mitchell-Schaeffer (MS) simplified biophysical model is used to study the spatial heterogeneity of electrophysiological (EP) tissue properties and their role in arrhythmogenesis. Radiofrequency ablation (RFA) with the elimination of local abnormal ventricular activities (LAVA) has recently arisen as a potentially curative treatment for ventricular tachycardia but the EP studies required to locate LAVA are lengthy and invasive. LAVA are commonly found within the heterogeneous scar, which can be imaged non-invasively with 3D delayed enhanced magnetic resonance imaging (DE-MRI). We evaluate the use of advanced image features in a random forest machine learning framework to identify areas of LAVA-inducing tissue. Furthermore, we detail the dataset’s inherent error sources and their formal integration in the training process. Finally, we construct MRI-based structural patient-specific heart models and couple them with the MS model. We model a recording catheter using a dipole approach and generate distinct normal and LAVA-like electrograms at locations where they have been found in clinics. This enriches our predictions of the locations of LAVA-inducing tissue obtained through image-based learning. Confidence maps can be generated and analyzed prior to RFA to guide the intervention. These contributions have led to promising results and proofs of concepts.Les arythmies sont des perturbations du rythme cardiaque qui peuvent entrainer la mort subite et requièrent une meilleure compréhension pour planifier leur traitement. Dans cette thèse, nous intégrons des données structurelles et fonctionnelles à un maillage 3D tétraédrique biventriculaire. Le modèle biophysique simplifié de Mitchell-Schaeffer (MS) est utilisé pour étudier l’hétérogénéité des propriétés électrophysiologiques (EP) du tissu et leur rôle sur l’arythmogénèse. L’ablation par radiofréquence (ARF) en éliminant les activités ventriculaires anormales locales (LAVA) est un traitement potentiellement curatif pour la tachycardie ventriculaire, mais les études EP requises pour localiser les LAVA sont longues et invasives. Les LAVA se trouvent autour de cicatrices hétérogènes qui peuvent être imagées de façon non-invasive par IRM à rehaussement tardif. Nous utilisons des caractéristiques d’image dans un contexte d’apprentissage automatique avec des forêts aléatoires pour identifier des aires de tissu qui induisent des LAVA. Nous détaillons les sources d’erreur inhérentes aux données et leur intégration dans le processus d’apprentissage. Finalement, nous couplons le modèle MS avec des géométries du coeur spécifiques aux patients et nous modélisons le cathéter avec une approche par un dipôle pour générer des électrogrammes normaux et des LAVA aux endroits où ils ont été localisés en clinique. Cela améliore la prédiction de localisation du tissu induisant des LAVA obtenue par apprentissage sur l’image. Des cartes de confiance sont générées et peuvent être utilisées avant une ARF pour guider l’intervention. Les contributions de cette thèse ont conduit à des résultats et des preuves de concepts prometteurs

Combinatorial optimisation for arterial image segmentation.

Cardiovascular disease is one of the leading causes of the mortality in the western world. Many imaging modalities have been used to diagnose cardiovascular diseases. However, each has different forms of noise and artifacts that make the medical image analysis field important and challenging. This thesis is concerned with developing fully automatic segmentation methods for cross-sectional coronary arterial imaging in particular, intra-vascular ultrasound and optical coherence tomography, by incorporating prior and tracking information without any user intervention, to effectively overcome various image artifacts and occlusions. Combinatorial optimisation methods are proposed to solve the segmentation problem in polynomial time. A node-weighted directed graph is constructed so that the vessel border delineation is considered as computing a minimum closed set. A set of complementary edge and texture features is extracted. Single and double interface segmentation methods are introduced. Novel optimisation of the boundary energy function is proposed based on a supervised classification method. Shape prior model is incorporated into the segmentation framework based on global and local information through the energy function design and graph construction. A combination of cross-sectional segmentation and longitudinal tracking is proposed using the Kalman filter and the hidden Markov model. The border is parameterised using the radial basis functions. The Kalman filter is used to adapt the inter-frame constraints between every two consecutive frames to obtain coherent temporal segmentation. An HMM-based border tracking method is also proposed in which the emission probability is derived from both the classification-based cost function and the shape prior model. The optimal sequence of the hidden states is computed using the Viterbi algorithm. Both qualitative and quantitative results on thousands of images show superior performance of the proposed methods compared to a number of state-of-the-art segmentation methods

Directional Estimation for Robotic Beating Heart Surgery

In robotic beating heart surgery, a remote-controlled robot can be used to carry out the operation while automatically canceling out the heart motion. The surgeon controlling the robot is shown a stabilized view of the heart. First, we consider the use of directional statistics for estimation of the phase of the heartbeat. Second, we deal with reconstruction of a moving and deformable surface. Third, we address the question of obtaining a stabilized image of the heart

Motion Tracking for Medical Applications using Hierarchical Filter Models

A medical intervention often requires relating treatment to the situation, which it was planned on. In order to circumvent undesirable effects of motion during the intervention, positional differences must be detected in real-time. To this end, in this thesis a hierarchical Particle Filter based tracking algorithm is developed in three stages. Initially, a model description of the individual nodes in the aspired hierarchical tree is presented. Using different approaches, properties of such a node are derived and approximated, leading to a parametrization scheme. Secondly, transformations and appearance of the data are described by a fixed hierarchical tree. A sparse description for typical landmarks in medical image data is presented. A static tree model with two levels is developed and investigated. Finally, the notion of 'association' between landmarks and nodes is introduced in order to allow for dynamic adaptation to the underlying structure of the data. Processes for tree maintenance using clustering and sequential reinforcement are implemented. The function of the full algorithm is demonstrated on data of abdominal breathing motion

Directional Estimation for Robotic Beating Heart Surgery

In robotic beating heart surgery, a remote-controlled robot can be used to carry out the operation while automatically canceling out the heart motion. The surgeon controlling the robot is shown a stabilized view of the heart. First, we consider the use of directional statistics for estimation of the phase of the heartbeat. Second, we deal with reconstruction of a moving and deformable surface. Third, we address the question of obtaining a stabilized image of the heart

Fast dynamic magnetic resonance imaging using sparse recovery methods and novel signal encoding formulations

Magnetic resonance imaging (MRI) is a non-invasive imaging modality that provides excellent soft tissue contrast without using ionizing radiations. These qualities/properties make MRI the preferred imaging modality for critical organs like heart and brain. Over the past decade, the advancement in hardware and image reconstruction algorithms has led to substantial improvements in MRI in terms of imaging speeds, quality and reliability. However, MRI speeds need to be further improved while retaining/maintaining the image quality given that the emerging medical diagnostic procedures are increasingly relying on detailed characterization of physiological functions that evolve on time scales too fast to be captured using conventional MRI methods. This dissertation starts with presenting a sparse signal recovery based fast MRI method. This method synergistically combines a data redundancy scheme for high frequency details with a novel and physically realizable MR signal encoding formulation. The new signal encoding formulation uses clinically deployed tagging radio frequency pulses to mix information in the spatial frequency domain prior to acquisition. Thus, the new formulation leads to a more uniform coverage of spatial frequency information even at high accelerations. The synergistic combination of image-detail redundancy encoding with tagging based signal encoding allows recovery of edges and fine structures with unprecedented quality. Next, this dissertation evaluates the use of fast spiral trajectories for high spatial resolution functional imaging of human superior colliculus. Gradient efficient and motion-robust spiral trajectories are used to keep fMRI scan durations short. . However, high resolution imaging of human subcortical structures using these trajectories is limited due to the weak functional responses of SC structures and also low signal-to-noise ratio associated with small voxels. To improve the functional sensitivity of spiral trajectories, dual echo variants are used. Combination of two echoes of the dual-echo variants reduces noise and thereby improves the functional sensitivity of high resolution fMRI. Lastly, this dissertation presents a novel formulation for fast dynamic MRI which combines the generic linear dynamical system model with sparse recovery techniques. Specifically, the formulation uses a known prior spatio-temporal model to predict the underlying image and uses sparse recovery techniques to recover the residual image. The spatio-temporal evolution model inherently encodes for coupled data redundancies in the spatial- and temporal-dimensions. Also, the generalizability of the formulation in choosing the evolution model allows it to be applicable to various physiological functions.Electrical and Computer Engineerin

State Estimation for Distributed Systems with Stochastic and Set-membership Uncertainties

State estimation techniques for centralized, distributed, and decentralized systems are studied. An easy-to-implement state estimation concept is introduced that generalizes and combines basic principles of Kalman filter theory and ellipsoidal calculus. By means of this method, stochastic and set-membership uncertainties can be taken into consideration simultaneously. Different solutions for implementing these estimation algorithms in distributed networked systems are presented