Robotics-Assisted Needle Steering for Percutaneous Interventions: Modeling and Experiments

Needle insertion and guidance plays an important role in medical procedures such as brachytherapy and biopsy. Flexible needles have the potential to facilitate precise targeting and avoid collisions during medical interventions while reducing trauma to the patient and post-puncture issues. Nevertheless, error introduced during guidance degrades the effectiveness of the planned therapy or diagnosis. Although steering using flexible bevel-tip needles provides great mobility and dexterity, a major barrier is the complexity of needle-tissue interaction that does not lend itself to intuitive control. To overcome this problem, a robotic system can be employed to perform trajectory planning and tracking by manipulation of the needle base. This research project focuses on a control-theoretic approach and draws on the rich literature from control and systems theory to model needle-tissue interaction and needle flexion and then design a robotics-based strategy for needle insertion/steering. The resulting solutions will directly benefit a wide range of needle-based interventions. The outcome of this computer-assisted approach will not only enable us to perform efficient preoperative trajectory planning, but will also provide more insight into needle-tissue interaction that will be helpful in developing advanced intraoperative algorithms for needle steering. Experimental validation of the proposed methodologies was carried out on a state of-the-art 5-DOF robotic system designed and constructed in-house primarily for prostate brachytherapy. The system is equipped with a Nano43 6-DOF force/torque sensor (ATI Industrial Automation) to measure forces and torques acting on the needle shaft. In our setup, an Aurora electromagnetic tracker (Northern Digital Inc.) is the sensing device used for measuring needle deflection. A multi-threaded application for control, sensor readings, data logging and communication over the ethernet was developed using Microsoft Visual C 2005, MATLAB 2007 and the QuaRC Toolbox (Quanser Inc.). Various artificial phantoms were developed so as to create a realistic medium in terms of elasticity and insertion force ranges; however, they simulated a uniform environment without exhibiting complexities of organic tissues. Experiments were also conducted on beef liver and fresh chicken breast, beef, and ham, to investigate the behavior of a variety biological tissues

Survey on Current State-of-the-Art in Needle Insertion Robots: Open Challenges for Application in Real Surgery

AbstractMinimally invasive percutaneous treatment robots have become a popular area in medical robotics. Minimally invasive treatments are an important part of modern surgery; however percutaneous treatments are a difficult procedure for surgeons. They must carry out a procedure that has limited visibility, tool maneuverability and where the target and tissue surrounding it move because of the tool. Robot technology can overcome those limitations and increase the success of minimally invasive percutaneous treatment. In this paper we will present a review of the current state-of-the-art in robotic insertion needle for minimally invasive treatments, focusing on the limitations and challenges still open for their use in clinical application

Image-Guided Robot-Assisted Techniques with Applications in Minimally Invasive Therapy and Cell Biology

There are several situations where tasks can be performed better robotically rather than manually. Among these are situations (a) where high accuracy and robustness are required, (b) where difficult or hazardous working conditions exist, and (c) where very large or very small motions or forces are involved. Recent advances in technology have resulted in smaller size robots with higher accuracy and reliability. As a result, robotics is fi nding more and more applications in Biomedical Engineering. Medical Robotics and Cell Micro-Manipulation are two of these applications involving interaction with delicate living organs at very di fferent scales.Availability of a wide range of imaging modalities from ultrasound and X-ray fluoroscopy to high magni cation optical microscopes, makes it possible to use imaging as a powerful means to guide and control robot manipulators. This thesis includes three parts focusing on three applications of Image-Guided Robotics in biomedical engineering, including: Vascular Catheterization: a robotic system was developed to insert a catheter through the vasculature and guide it to a desired point via visual servoing. The system provides shared control with the operator to perform a task semi-automatically or through master-slave control. The system provides control of a catheter tip with high accuracy while reducing X-ray exposure to the clinicians and providing a more ergonomic situation for the cardiologists. Cardiac Catheterization: a master-slave robotic system was developed to perform accurate control of a steerable catheter to touch and ablate faulty regions on the inner walls of a beating heart in order to treat arrhythmia. The system facilitates touching and making contact with a target point in a beating heart chamber through master-slave control with coordinated visual feedback. Live Neuron Micro-Manipulation: a microscope image-guided robotic system was developed to provide shared control over multiple micro-manipulators to touch cell membranes in order to perform patch clamp electrophysiology. Image-guided robot-assisted techniques with master-slave control were implemented for each case to provide shared control between a human operator and a robot. The results show increased accuracy and reduced operation time in all three cases

Avancées du framework de simulation biomécanique inverse pour le pilotage automatique d’aiguilles robotisé

International audienc

On the Application of Mechanical Vibration in Robotics-Assisted Soft Tissue Intervention

Mechanical vibration as a way of transmitting energy has been an interesting subject to study. While cyclic oscillation is usually associated with fatigue effect, and hence a detrimental factor in failure of structures and machineries, by controlled transmission of vibration, energy can be transferred from the source to the target. In this thesis, the application of such mechanical vibration in a few surgical procedures is demonstrated. Three challenges associated with lung cancer diagnosis and treatment are chosen for this purpose, namely, Motion Compensation, tumor targeting in lung Needle Insertion and Soft Tissue Dissection: A robotic solution is proposed for compensating for the undesirable oscillatory motion of soft tissue (caused by heart beat and respiration) during needle insertion in the lung. An impedance control strategy based on a mechanical vibratory system is implemented to minimize the tissue deformation during needle insertion. A prototype was built to evaluate the proposed approach using: 1) two Mitsubishi PA10-7C robots, one for manipulating the macro part and the other for mimicking the tissue motion, 2) one motorized linear stage to handle the micro part, and 3) a Phantom Omni haptic device for remote manipulation. Experimental results are given to demonstrate the performance of the motion compensation system. A vibration-assisted needle insertion technique has been proposed in order to reduce needle–tissue friction. The LuGre friction model is employed as a basis for the study and the model is extended and analyzed to include the impact of high-frequency vibration on translational friction. Experiments are conducted to evaluate the role of insertion speed as well as vibration frequency on frictional effects. In the experiments conducted, an 18 GA brachytherapy needle was vibrated and inserted into an ex-vivo soft tissue sample using a pair of amplified piezoelectric actuators. Analysis demonstrates that the translational friction can be reduced by introducing a vibratory low-amplitude motion onto a regular insertion profile, which is usually performed at a constant rate. A robotics-assisted articulating ultrasonic surgical scalpel for minimally invasive soft tissue cutting and coagulation is designed and developed. For this purpose, the optimal design of a Langevin transducer with stepped horn profile is presented for internal-body applications. The modeling, optimization and design of the ultrasonic scalpel are performed through equivalent circuit theory and verified by finite element analysis. Moreover, a novel surgical wrist, compatible with the da Vinci® surgical system, with decoupled two degrees-of-freedom (DOFs) is developed that eliminates the strain of pulling cables and electrical wires. The developed instrument is then driven using the dVRK (da Vinci® research kit) and the Classic da Vinci® surgical system

Dynamic analysis of a needle insertion for soft materials: Arbitrary Lagrangian-Eulerian-based three-dimensional finite element analysis

Background: Our goal was to develop a three-dimensional finite element model that enables dynamic analysis of needle insertion for soft materials. To demonstrate large deformation and fracture, we used the arbitrary Lagrangian-Eulerian (ALE) method for fluid analysis. We performed ALE-based finite element analysis for 3% agar gel and three types of copper needle with bevel tips. Methods: To evaluate simulation results, we compared the needle deflection and insertion force with corresponding experimental results acquired with a uniaxial manipulator. We studied the shear stress distribution of agar gel on various time scales. Results: For 30°, 45°, and 60°, differences in deflections of each needle between both sets of results were 2.424, 2.981, and 3.737. mm, respectively. For the insertion force, there was no significant difference for mismatching area error (p<0.05) between simulation and experimental results. Conclusions: Our results have the potential to be a stepping stone to develop pre-operative surgical planning to estimate an optimal needle insertion path for MR image-guided microwave coagulation therapy and for analyzing large deformation and fracture in biological tissues. © 2014 Elsevier Ltd.Yamaguchi S., Tsutsui K., Satake K., et al. Dynamic analysis of a needle insertion for soft materials: Arbitrary Lagrangian-Eulerian-based three-dimensional finite element analysis. Computers in Biology and Medicine 53, 42 (2014); https://doi.org/10.1016/j.compbiomed.2014.07.012

Development of a cognitive robotic system for simple surgical tasks

The introduction of robotic surgery within the operating rooms has significantly improved the quality of many surgical procedures. Recently, the research on medical robotic systems focused on increasing the level of autonomy in order to give them the possibility to carry out simple surgical actions autonomously. This paper reports on the development of technologies for introducing automation within the surgical workflow. The results have been obtained during the ongoing FP7 European funded project Intelligent Surgical Robotics (I-SUR). The main goal of the project is to demonstrate that autonomous robotic surgical systems can carry out simple surgical tasks effectively and without major intervention by surgeons. To fulfil this goal, we have developed innovative solutions (both in terms of technologies and algorithms) for the following aspects: fabrication of soft organ models starting from CT images, surgical planning and execution of movement of robot arms in contact with a deformable environment, designing a surgical interface minimizing the cognitive load of the surgeon supervising the actions, intra-operative sensing and reasoning to detect normal transitions and unexpected events. All these technologies have been integrated using a component-based software architecture to control a novel robot designed to perform the surgical actions under study. In this work we provide an overview of our system and report on preliminary results of the automatic execution of needle insertion for the cryoablation of kidney tumours

Development of a Novel Handheld Device for Active Compensation of Physiological Tremor

In microsurgery, the human hand imposes certain limitations in accurately positioning the tip of a device such as scalpel. Any errors in the motion of the hand make microsurgical procedures difficult and involuntary motions such as hand tremors can make some procedures significantly difficult to perform. This is particularly true in the case of vitreoretinal microsurgery. The most familiar source of involuntary motion is physiological tremor. Real-time compensation of tremor is, therefore, necessary to assist surgeons to precisely position and manipulate the tool-tip to accurately perform a microsurgery. In this thesis, a novel handheld device (AID) is described for compensation of physiological tremor in the hand. MEMS-based accelerometers and gyroscopes have been used for sensing the motion of the hand in six degrees of freedom (DOF). An augmented state complementary Kalman filter is used to calculate 2 DOF orientation. An adaptive filtering algorithm, band-limited Multiple Fourier linear combiner (BMFLC), is used to calculate the tremor component in the hand in real-time. Ionic Polymer Metallic Composites (IPMCs) have been used as actuators for deflecting the tool-tip to compensate for the tremor

Vision-based methods for state estimation and control of robotic systems with application to mobile and surgical robots

For autonomous systems that need to perceive the surrounding environment for the accomplishment of a given task, vision is a highly informative exteroceptive sensory source. When gathering information from the available sensors, in fact, the richness of visual data allows to provide a complete description of the environment, collecting geometrical and semantic information (e.g., object pose, distances, shapes, colors, lights). The huge amount of collected data allows to consider both methods exploiting the totality of the data (dense approaches), or a reduced set obtained from feature extraction procedures (sparse approaches). This manuscript presents dense and sparse vision-based methods for control and sensing of robotic systems. First, a safe navigation scheme for mobile robots, moving in unknown environments populated by obstacles, is presented. For this task, dense visual information is used to perceive the environment (i.e., detect ground plane and obstacles) and, in combination with other sensory sources, provide an estimation of the robot motion with a linear observer. On the other hand, sparse visual data are extrapolated in terms of geometric primitives, in order to implement a visual servoing control scheme satisfying proper navigation behaviours. This controller relies on visual estimated information and is designed in order to guarantee safety during navigation. In addition, redundant structures are taken into account to re-arrange the internal configuration of the robot and reduce its encumbrance when the workspace is highly cluttered. Vision-based estimation methods are relevant also in other contexts. In the field of surgical robotics, having reliable data about unmeasurable quantities is of great importance and critical at the same time. In this manuscript, we present a Kalman-based observer to estimate the 3D pose of a suturing needle held by a surgical manipulator for robot-assisted suturing. The method exploits images acquired by the endoscope of the robot platform to extrapolate relevant geometrical information and get projected measurements of the tool pose. This method has also been validated with a novel simulator designed for the da Vinci robotic platform, with the purpose to ease interfacing and employment in ideal conditions for testing and validation. The Kalman-based observers mentioned above are classical passive estimators, whose system inputs used to produce the proper estimation are theoretically arbitrary. This does not provide any possibility to actively adapt input trajectories in order to optimize specific requirements on the performance of the estimation. For this purpose, active estimation paradigm is introduced and some related strategies are presented. More specifically, a novel active sensing algorithm employing visual dense information is described for a typical Structure-from-Motion (SfM) problem. The algorithm generates an optimal estimation of a scene observed by a moving camera, while minimizing the maximum uncertainty of the estimation. This approach can be applied to any robotic platforms and has been validated with a manipulator arm equipped with a monocular camera

Online model estimation and haptic characterization for robotic-assisted minimally invasive surgery

Online soft tissue characterization is important for robotic-assisted minimally invasive surgery (RAMIS) to achieve a precise and stable robotic control with haptic feedback. The traditional linear regression method (i.e. the recursive least square (RLS) method) is inappropriate to handle nonlinear Hunt-Crossley (H-C) model since its linearization process involves unacceptable errors. This thesis presents a new nonlinear estimation method for online soft tissue characterization. To deal with nonlinear and dynamic conditions involved in soft tissue characterization, the approach expands the nonlinearity and dynamics of the H-C model by treating parameter p as an independent variable. Based on this, an unscented Kalman filter (UKF) was adapted for online nonlinear soft tissue characterization. A comparison analysis of the UKF and RLS methods was conducted to validate the performance of the UKF-based method. The UKF-based method suffers from two major problems. The first one is that it requires prior noise statistics of the corresponding system to be precisely known. However, due to uncertainties in the dynamic environment of RAMIS, it is difficult to accurately describe noise characteristics. This leads to biased or even divergent UKF solutions. Therefore, in order to attain accurate estimation results from the UKF-based approach, it is necessary to estimate noise statistics online to restrain the disturbance of noise uncertainty. Secondly, the UKF performance depends on the pre-defined system and measurement models. If the models involve stochastic errors, the UKF-based solution will be unstable. In fact, the measurement model&amp;rsquo;s accuracy can be guaranteed by using high-precision measurement equipment together with a high volume of available measurement data. On the other hand, the system model is more often involved with the inaccuracy problem. In RAMIS, the system model is a theoretical approximation of the physical contact between robotic tool and biological soft tissue. The approximation is intended to fulfil the requirement of real-time performance in RAMIS. Therefore, it is essential to improve the UKF performance in the presence of system model (the contact model) uncertainty. To address the UKF problem for inaccurate noise statistics, this thesis further presents a new recursive adaptive UKF (RAUKF) method for online nonlinear soft tissue characterization. It was developed, based on the H-C model, to estimate system noise statistics in real-time with windowing approximation. The method was developed under the condition that system noises are of small variation. In order to account for the inherent relationship between the current and previous states of soft tissue deformation involved in RAMIS, a recursive formulation was further constructed by introducing a fading scaling factor. This factor was further modified to accommodate noise statistics of a large variation, which may be caused by rupture events or geometric discontinuities in RAMIS. Simulations and comparison analyses verified the performance of the proposed RAUKF. The second UKF limitation regarding the requirement of the accurate system model was also addressed. A random weighting strong tracking unscented Kalman filter (RWSTUKF) was developed based on the Hunt-Crossley model for online nonlinear soft tissue characterization. This RWSTUKF overcomes the problem of performance degradation in the UKF due to system model errors. It adopts a scaling factor in the predicted state covariance to compensate the inaccuracy of the system model. This scaling factor was derived by combining the orthogonality principle with the random weighting concept to prevent the cumbersome computation from Jacobian matrix and offer the reliable estimation for innovation covariances. Simulation and comparison analyses demonstrated that the proposed RWSTUKF can characterise soft tissue parameters in the presence of system model error for RAMIS in on online mode. Using the proposed methods, a master-slave robotic system has been developed with a nonlinear state observer for soft tissue characterization. Robotic indentation and needle insertion tests conducted to evaluate performances of the proposed methods. Further, a rupture detection approach was established based on the RWSTUKF. It was also integrated into the master-slave robotic system to detect rupture events occurred during needle insertion. The experiment results demonstrated that the RWSTUKF outperforms RLS, UKF and RAUKF for soft tissue characterization