Medical SLAM in an autonomous robotic system

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This thesis addresses the ambitious goal of achieving surgical autonomy, through the study of the anatomical environment by Initially studying the technology present and what is needed to analyze the scene: vision sensors. A novel endoscope for autonomous surgical task execution is presented in the first part of this thesis. Which combines a standard stereo camera with a depth sensor. This solution introduces several key advantages, such as the possibility of reconstructing the 3D at a greater distance than traditional endoscopes. Then the problem of hand-eye calibration is tackled, which unites the vision system and the robot in a single reference system. Increasing the accuracy in the surgical work plan. In the second part of the thesis the problem of the 3D reconstruction and the algorithms currently in use were addressed. In MIS, simultaneous localization and mapping (SLAM) can be used to localize the pose of the endoscopic camera and build ta 3D model of the tissue surface. Another key element for MIS is to have real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy. Starting from the ORB-SLAM algorithm we have modified the architecture to make it usable in an anatomical environment by adding the registration of the pre-operative information of the intervention to the map obtained from the SLAM. Once it has been proven that the slam algorithm is usable in an anatomical environment, it has been improved by adding semantic segmentation to be able to distinguish dynamic features from static ones. All the results in this thesis are validated on training setups, which mimics some of the challenges of real surgery and on setups that simulate the human body within Autonomous Robotic Surgery (ARS) and Smart Autonomous Robotic Assistant Surgeon (SARAS) projects

Medical SLAM in an autonomous robotic system

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This thesis addresses the ambitious goal of achieving surgical autonomy, through the study of the anatomical environment by Initially studying the technology present and what is needed to analyze the scene: vision sensors. A novel endoscope for autonomous surgical task execution is presented in the first part of this thesis. Which combines a standard stereo camera with a depth sensor. This solution introduces several key advantages, such as the possibility of reconstructing the 3D at a greater distance than traditional endoscopes. Then the problem of hand-eye calibration is tackled, which unites the vision system and the robot in a single reference system. Increasing the accuracy in the surgical work plan. In the second part of the thesis the problem of the 3D reconstruction and the algorithms currently in use were addressed. In MIS, simultaneous localization and mapping (SLAM) can be used to localize the pose of the endoscopic camera and build ta 3D model of the tissue surface. Another key element for MIS is to have real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy. Starting from the ORB-SLAM algorithm we have modified the architecture to make it usable in an anatomical environment by adding the registration of the pre-operative information of the intervention to the map obtained from the SLAM. Once it has been proven that the slam algorithm is usable in an anatomical environment, it has been improved by adding semantic segmentation to be able to distinguish dynamic features from static ones. All the results in this thesis are validated on training setups, which mimics some of the challenges of real surgery and on setups that simulate the human body within Autonomous Robotic Surgery (ARS) and Smart Autonomous Robotic Assistant Surgeon (SARAS) projects

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

SCEM+: Real-Time Robust Simultaneous Catheter and Environment Modeling for Endovascular Navigation

© 2016 IEEE. Endovascular procedures are characterised by significant challenges mainly due to the complexity in catheter control and navigation. Real-time recovery of the 3-D structure of the vasculature is necessary to visualise the interaction between the catheter and its surrounding environment to facilitate catheter manipulations. State-of-the-art intraoperative vessel reconstruction approaches are increasingly relying on nonionising imaging techniques such as optical coherence tomography (OCT) and intravascular ultrasound (IVUS). To enable accurate recovery of vessel structures and to deal with sensing errors and abrupt catheter motions, this letter presents a robust and real-time vessel reconstruction scheme for endovascular navigation based on IVUS and electromagnetic (EM) tracking. It is formulated as a nonlinear optimisation problem, which considers the uncertainty in both the IVUS contour and the EM pose, as well as vessel morphology provided by preoperative data. Detailed phantom validation is performed and the results demonstrate the potential clinical value of the technique

3D Guidewire Shape Reconstruction from Monoplane Fluoroscopic Images

Endovascular navigation, essential for diagnosing and treating endovascular diseases, predominantly hinges on fluoroscopic images due to the constraints in sensory feedback. Current shape reconstruction techniques for endovascular intervention often rely on either a priori information or specialized equipment, potentially subjecting patients to heightened radiation exposure. While deep learning holds potential, it typically demands extensive data. In this paper, we propose a new method to reconstruct the 3D guidewire by utilizing CathSim, a state-of-the-art endovascular simulator, and a 3D Fluoroscopy Guidewire Reconstruction Network (3D-FGRN). Our 3D-FGRN delivers results on par with conventional triangulation from simulated monoplane fluoroscopic images. Our experiments accentuate the efficiency of the proposed network, demonstrating it as a promising alternative to traditional methods.Comment: 11 page

Modeling and Control of Steerable Ablation Catheters

Catheters are long, flexible tubes that are extensively used in vascular and cardiac interventions, e.g., cardiac ablation, coronary angiography and mitral valve annuloplasty. Catheter-based cardiac ablation is a well-accepted treatment for atrial fibrillation, a common type of cardiac arrhythmia. During this procedure, a steerable ablation catheter is guided through the vasculature to the left atrium to correct the signal pathways inside the heart and restore normal heart rhythm. The outcome of the ablation procedure depends mainly on the correct positioning of the catheter tip at the target location inside the heart and also on maintaining a consistent contact between the catheter tip and cardiac tissue. In the presence of cardiac and respiratory motions, achieving these goals during the ablation procedure is very challenging without proper 3D visualization, dexterous control of the flexible catheter and an estimate of the catheter tip/tissue contact force. This research project provides the required basis for developing a robotics-assisted catheter manipulation system with contact force control for use in cardiac ablation procedures. The behavior of the catheter is studied in free space as well in contact with the environment to develop mathematical models of the catheter tip that are well suited for developing control systems. The validity of the proposed modeling approaches and the performance of the suggested control techniques are evaluated experimentally. As the first step, the static force-deflection relationship for ablation catheters is described with a large-deflection beam model and an optimized pseudo-rigid-body 3R model. The proposed static model is then used in developing a control system for controlling the contact force when the catheter tip is interacting with a static environment. Our studies also showed that it is possible to estimate the tip/tissue contact force by analyzing the shape of the catheter without installing a force sensor on the catheter. During cardiac ablation, the catheter tip is in contact with a relatively fast moving environment (cardiac tissue). Robotic manipulation of the catheter has the potential to improve the quality of contact between the catheter tip and cardiac tissue. To this end, the frequency response of the catheter is investigated and a control technique is proposed to compensate for the cardiac motion and to maintain a constant tip/tissue contact force. Our study on developing a motion compensated robotics-assisted catheter manipulation system suggests that redesigning the actuation mechanism of current ablation catheters would provide a major improvement in using these catheters in robotics-assisted cardiac ablation procedures

Dynamic modeling of soft continuum manipulators using lie group variational integration

This paper presents the derivation and experimental validation of algorithms for modeling and estimation of soft continuum manipulators using Lie group variational integration. Existing approaches are generally limited to static and quasi-static analyses, and are not sufficiently validated for dynamic motion. However, in several applications, models need to consider the dynamical behavior of the continuum manipulators. The proposed modeling and estimation formulation is obtained from a discrete variational principle, and therefore grants outstanding conservation properties to the continuum mechanical model. The main contribution of this article is the experimental validation of the dynamic model of soft continuum manipulators, including external torques and forces (e.g., generated by magnetic fields, friction, and the gravity), by carrying out different experiments with metal rods and polymer-based soft rods. To consider dissipative forces in the validation process, distributed estimation filters are proposed. The experimental and numerical tests also illustrate the algorithm's performance on a magnetically-actuated soft continuum manipulator. The model demonstrates good agreement with dynamic experiments in estimating the tip position of a Polydimethylsiloxane (PDMS) rod. The experimental results show an average absolute error and maximum error in tip position estimation of 0.13 mm and 0.58 mm, respectively, for a manipulator length of 60.55 mm

A Survey on the Current Status and Future Challenges Towards Objective Skills Assessment in Endovascular Surgery

Minimally-invasive endovascular interventions have evolved rapidly over the past decade, facilitated by breakthroughs in medical imaging and sensing, instrumentation and most recently robotics. Catheter based operations are potentially safer and applicable to a wider patient population due to the reduced comorbidity. As a result endovascular surgery has become the preferred treatment option for conditions previously treated with open surgery and as such the number of patients undergoing endovascular interventions is increasing every year. This fact coupled with a proclivity for reduced working hours, results in a requirement for efficient training and assessment of new surgeons, that deviates from the “see one, do one, teach one” model introduced by William Halsted, so that trainees obtain operational expertise in a shorter period. Developing more objective assessment tools based on quantitative metrics is now a recognised need in interventional training and this manuscript reports the current literature for endovascular skills assessment and the associated emerging technologies. A systematic search was performed on PubMed (MEDLINE), Google Scholar, IEEXplore and known journals using the keywords, “endovascular surgery”, “surgical skills”, “endovascular skills”, “surgical training endovascular” and “catheter skills”. Focusing explicitly on endovascular surgical skills, we group related works into three categories based on the metrics used; structured scales and checklists, simulation-based and motion-based metrics. This review highlights the key findings in each category and also provides suggestions for new research opportunities towards fully objective and automated surgical assessment solutions

Constrained Stochastic State Estimation of Deformable 1D Objects: Application to Single-view 3D Reconstruction of Catheters with Radio-opaque Markers

International audienceMinimally invasive fluoroscopy-based procedures are the gold standard for diagnosis and treatment of various pathologies of the cardiovascular system. This kind of procedures imply for the clinicians to infer the 3D shape of the device from 2D images, which is known to be an ill-posed 10 problem. In this paper we present a method to reconstruct the 3D shape of the interventional device, with the aim of improving the navigation. The method combines a physics-based simulation with non-linear Bayesian filter. Whereas the physics-based model provides a prediction of the shape of the device navigating within the blood vessels (taking into account non-linear interactions be-15 tween the catheter and the surrounding anatomy), an Unscented Kalman Filter is used to correct the navigation model using 2D image features as external observations. The proposed framework has been evaluated on both synthetic and real data, under different model parameterizations, filter parameters tuning and external observations data-sets. Comparing the reconstructed 3D shape with a known ground truth, for the synthetic data-set, we obtained average values for 3D Hausdorff Distance of $0.81 ± 0.53 mm$, for the 3D mean distance at the segment of $0.37 ± 0.17$ mm and an average 3D tip error of $0.24 ± 0.13 mm$. For the real data-set,we obtained an average 3D Hausdorff distance of $1.74 ± 0.77 mm$, a average 3D mean distance at the distal segment of 0.91 ± 0.14 mm, an average 3D error on the tip of $0.53 ± 0.09 mm$. These results show the ability of our method to retrieve the 3D shape of the device, under a variety of filter parameterizations and challenging conditions: uncertainties on model parameterization, ambiguous views and non-linear complex phenomena such as stick and slip motions