Semi-Automated Needle Steering in Biological Tissue Using an Ultrasound-Based Deflection Predictor

The performance of needle-based interventions depends on the accuracy of needle tip positioning. Here, a novel needle steering strategy is proposed that enhances accuracy of needle steering. In our approach the surgeon is in charge of needle insertion to ensure the safety of operation, while the needle tip bevel location is robotically controlled to minimize the targeting error. The system has two main components: (1) a real-time predictor for estimating future needle deflection as it is steered inside soft tissue, and (2) an online motion planner that calculates control decisions and steers the needle toward the target by iterative optimization of the needle deflection predictions. The predictor uses the ultrasound-based curvature information to estimate the needle deflection. Given the specification of anatomical obstacles and a target from preoperative images, the motion planner uses the deflection predictions to estimate control actions, i.e., the depth(s) at which the needle should be rotated to reach the target. Ex-vivo needle insertions are performed with and without obstacle to validate our approach. The results demonstrate the needle steering strategy guides the needle to the targets with a maximum error of 1.22 mm

Survey on Current State-of-the-Art in Needle Insertion Robots: Open Challenges for Application in Real Surgery

AbstractMinimally invasive percutaneous treatment robots have become a popular area in medical robotics. Minimally invasive treatments are an important part of modern surgery; however percutaneous treatments are a difficult procedure for surgeons. They must carry out a procedure that has limited visibility, tool maneuverability and where the target and tissue surrounding it move because of the tool. Robot technology can overcome those limitations and increase the success of minimally invasive percutaneous treatment. In this paper we will present a review of the current state-of-the-art in robotic insertion needle for minimally invasive treatments, focusing on the limitations and challenges still open for their use in clinical application

Modeling, Sensorization and Control of Concentric-Tube Robots

Since the concept of the Concentric-Tube Robot (CTR) was proposed in 2006, CTRs have been a popular research topic in the field of surgical robotics. The unique mechanical design of this robot allows it to navigate through narrow channels in the human anatomy and operate in highly constrained environments. It is therefore likely to become the next generation of surgical robots to overcome the challenges that cannot be addressed by current technologies. In CSTAR, we have had ongoing work over the past several years aimed at developing novel techniques and technologies for CTRs. This thesis describes the contributions made in this context, focusing primarily on topics such as modeling, sensorization, and control of CTRs. Prior to this work, one of the main challenges in CTRs was to develop a kinematic model that achieves a balance between the numerical accuracy and computational efficiency for surgical applications. In this thesis, a fast kinematic model of CTRs is proposed, which can be solved at a comparatively fast rate (0.2 ms) with minimal loss of accuracy (0.1 mm) for a 3-tube CTR. A Jacobian matrix is derived based on this model, leading to the development of a real-time trajectory tracking controller for CTRs. For tissue-robot interactions, a force-rejection controller is proposed for position control of CTRs under time-varying force disturbances. In contrast to rigid-link robots, instability of position control could be caused by non-unique solutions to the forward kinematics of CTRs. This phenomenon is modeled and analyzed, resulting in design criteria that can ensure kinematic stability of a CTR in its entire workspace. Force sensing is another major difficulty for CTRs. To address this issue, commercial force/torque sensors (Nano43, ATI Industrial Automation, United States) are integrated into one of our CTR prototypes. These force/torque sensors are replaced by Fiber-Bragg Grating (FBG) sensors that are helically-wrapped and embedded in CTRs. A strain-force calculation algorithm is proposed, to convert the reflected wavelength of FBGs into force measurements with 0.1 N force resolution at 100 Hz sampling rate. In addition, this thesis reports on our innovations in prototyping drive units for CTRs. Three designs of CTR prototypes are proposed, the latest one being significantly more compact and cost efficient in comparison with most designs in the literature. All of these contributions have brought this technology a few steps closer to being used in operating rooms. Some of the techniques and technologies mentioned above are not merely limited to CTRs, but are also suitable for problems arising in other types of surgical robots, for example, for sensorizing da Vinci surgical instruments for force sensing (see Appendix A)

Needle Steering in 3-D Via Rapid Replanning

Steerable needles have the potential to improve the effectiveness of needle-based clinical procedures such as biopsy and drug delivery by improving targeting accuracy and reaching previously inaccessible targets that are behind sensitive or impenetrable anatomical regions. We present a new needle steering system capable of automatically reaching targets in 3-D environments while avoiding obstacles and compensating for real-world uncertainties. Given a specification of anatomical obstacles and a clinical target (e.g., from preoperative medical images), our system plans and controls needle motion in a closed-loop fashion under sensory feedback to optimize a clinical metric. We unify planning and control using a new fast algorithm that continuously replans the needle motion. Our rapid replanning approach is enabled by an efficient sampling-based rapidly exploring random tree (RRT) planner that achieves orders-of-magnitude reduction in computation time compared with prior 3-D approaches by incorporating variable curvature kinematics and a novel distance metric for planning. Our system uses an electromagnetic tracking system to sense the state of the needle tip during the procedure. We experimentally evaluate our needle steering system using tissue phantoms and animal tissue ex vivo. We demonstrate that our rapid replanning strategy successfully guides the needle around obstacles to desired 3-D targets with an average error of less than 3 mm

下腹部を対象とした極細針によるCTガイド下高正確度穿刺プランニング

早大学位記番号:新8149早稲田大

SMART IMAGE-GUIDED NEEDLE INSERTION FOR TISSUE BIOPSY

M.S

Sensorisation of a novel biologically inspired flexible needle

Percutaneous interventions are commonly performed during minimally invasive brain surgery, where a straight rigid instrument is inserted through a small incision to access a deep lesion in the brain. Puncturing a vessel during this procedure can be a life-threatening complication. Embedding a forward-looking sensor in a rigid needle has been proposed to tackle this problem; however, using a rigid needle, the procedure needs to be interrupted if a vessel is detected. Steerable needle technology could be used to avoid obstacles, such as blood vessels, due to its ability to follow curvilinear paths, but research to date was lacking in this respect. This thesis aims to investigate the deployment of forward-looking sensors for vessel detection in a steerable needle. The needle itself is based on a bioinspired programmable bevel-tip needle (PBN), a multi-segment design featuring four hollow working channels. In this thesis, laser Doppler flowmetry (LDF) is initially characterised to ensure that the sensor fulfils the minimum requirements for it to be used in conjunction with the needle. Subsequently, vessel reconstruction algorithms are proposed. To determine the axial and off-axis position of the vessel with respect to the probe, successive measurements of the LDF sensor are used. Ideally, full knowledge of the vessel orientation is required to execute an avoidance strategy. Using two LDF probes and a novel signal processing method described in this thesis, the predicted possible vessel orientations can be reduced to four, a setup which is explored here to demonstrate viable obstacle detection with only partial sensor information. Relative measurements from four LDF sensors are also explored to classify possible vessel orientations in full and without ambiguity, but under the assumption that the vessel is perpendicular to the needle insertion axis. Experimental results on a synthetic grey matter phantom are presented, which confirm these findings. To release the perpendicularity assumption, the thesis concludes with the description of a machine learning technique based on a Long Short-term memory network, which enables a vessel's spatial position, cross-sectional diameter and full pose to be predicted with sub-millimetre accuracy. Simulated and in-vitro examinations of vessel detection with this approach are used to demonstrate effective predictive ability. Collectively, these results demonstrate that the proposed steerable needle sensorisation is viable and could lead to improved safety during robotic assisted needle steering interventions.Open Acces

Biomechanics of a parasitic wasp ovipositor : Probing for answers

Insects such as mosquitoes, true bugs, and parasitic wasps, probe for resources hidden in various substrates. The resources are often, located deep within the substrate and can only be reached with long and thin (slender) probes. Such probes can, however, easily bend or break (buckle) when pushed inside the substrate, which makes probing a challenging task. Nevertheless, the mentioned insects use their probes repeatedly throughout their lifetime without apparent damage. Furthermore, the probes are also used for sensing the targets, can be steered during insertion, and can transport both fluids (e.g. blood, phloem sap) and eggs. Insect probes seem highly versatile structures that satisfy many functional requirements, including buckling avoidance, steering, sensing, and transport. Similar requirements also hold for minimally invasive medical procedures, where slender tools are used to minimize damage to the patient. Understanding the probing process in insects can bring insights in the insect ecology and evolution and it may also help in the development of novel surgical tools. In this thesis, I focus on the mechanical and motor adaptations of insect probing, while other aspects are only briefly discussed. In chapter 2, we review the literature on the probing structures and their operating principles across mosquitoes, parasitic wasps, and hemipterans. Probes are either modified mouthparts (mosquitoes, true bugs) or special tubular outgrowths of the abdomen (parasitic wasps). Despite having different developmental origins, the probes share three major morphological characteristics, which may reflect the shared functional requirements of buckling avoidance and steering: (i) the probes consist of multiple, interconnected elements that can slide along each other, (ii) the probe diameters are very small, which leaves no space for internal musculature, and (iii) the distal ends (tips) of the probe elements are asymmetric and often bear various serrations, hooks, bulges, or notches. How such slender multi-element probes avoid buckling during insertion has been hypothesized in the so-called push–pull mechanism. According to this mechanism, the probe is inserted into the substrate by reciprocal movements of the elements. The insects therefore simultaneously push on some of the probe elements, while pulling on the others. The tip serrations are directed such, that they primarily increase the friction upon pulling of the elements. This puts the pulled elements under tension and makes them effectively stiffer in bending (like when pulling a rope). The elements under tension can serve as guides along which the other elements are pushed inside the substrate without the risk of buckling. The insect alternates the pushing and pulling between the elements to incrementally insert the probe in the substrate. This mechanism has, however, never been quantified in insects and it was hitherto unknown whether the animals rely on it during probing. The probe tip asymmetry presumably facilitates steering. The asymmetric tip geometry leads to asymmetric reaction forces from the substrate on the tip during insertion, which push the probe tip sideways into a curved path. Controlling the tip geometry therefore allows for control of probing direction. Although offsetting the elements by sliding already changes the shape of the probe tip, these changes might be too small to induce the necessary change of probing direction. A number of mechanisms that enhance the tip asymmetry during the sliding of the elements have been suggested. However, few mechanisms have been observed or studied in vivo, so it is not completely clear how insects steer with their probes. Additionally, the effect of the substrate on both the steering and insertion mechanisms is unknown. To understand the biomechanics of insect probing, we investigated the probing behaviour of the braconid parasitic wasp Diachasmimorpha longicaudata. This is an ideal species for studying the buckling avoidance and steering, because it: (i) possess a slender ovipositor several millimetres in length, (ii) probes into solid material (e.g. citrus fruits), and (iii) attack fruit-fly larvae that are freely moving within the substrate (i.e. steering can be expected). The ovipositor of D. longicaudata is similar to other hymenopterans and consists of three interconnected elements (valves), one dorsal and two ventral ones. The interconnection is a tongue-and-groove mechanism, which allows for sliding of the valves, but prevents their separation. The ovipositor has an asymmetric tip—the distal end of the dorsal valve is enlarged (bulge), while the ventral valve tips have harpoon-like serrations. Additionally, just proximal to the bulge of the dorsal valve, the ovipositor is characteristically bent in an S-shape. This seems to be a feature present only in D. longicaudata and closely related species. The wasps also possess a pair of sheaths that envelop the ovipositor at rest and throughout most of the probing process, but do not penetrate into the substrate. In chapter 3, we studied the kinematics of ovipositor insertion into translucent, artificial substrates of various stiffnesses. Ovipositor insertion was filmed in a three camera setup, which allowed us to reconstruct the ovipositor insertion in 3D, while also monitoring the orientation of the insect’s body. We discovered that the wasps can explore a wide range of the substrate by probing in any direction with respect to their body orientation from a single puncture point. Probing range and speed decreased with increasing substrate stiffness. Wasps used two strategies of ovipositor insertion. In soft substrates, all ovipositor valves were pushed inside the substrate at the same time. In stiff substrates, wasps always moved the valves alternatively, presumably employing the hypothesized push–pull mechanism. We observed that ovipositors can follow curved trajectories inside the substrate. Detailed kinematic analysis revealed that the ovipositors followed a curved path during probing with protracted ventral valve(s). In contrast, probing with protracted dorsal valve resulted in straight trajectories. We linked the changes in the probing direction to the shape changes in the ovipositor tip. When the ventral valves were protracted, they curved towards the dorsal valve, resulting in an enhanced bevel which presumably caused a change in insertion direction. In chapter 4, we investigated the above described steering mechanism by quantifying the bending stiffness (three point bend test) and the geometry (high-resolution computer tomography) of the ovipositor in D. longicaudata. Additionally, we qualitatively assessed the material composition of the valves using fluorescence imaging. The thick dorsal valve bulge might be stiff and could straighten the S-shaped region of the ovipositor during the valve offset, causing bending of the tip. We discovered that the S-shaped region of the ovipositor is significantly softer than its neighbouring regions, which is mostly due to the presence of resilin in the S-shaped region of the ventral valve. Resilin is a rubber-like protein and reduces the stiffness of the otherwise heavily sclerotized valves. Additionally, we showed that the ventral valves have a higher bending stiffness than the dorsal valve along most of their length. The exception is presumably the bulge on the dorsal valve—although we could not directly measure its bending stiffness, its geometrical properties show that it is the thickest (and therefore stiffest) region in the distal end of the ovipositor. Outside the substrate, offsetting of the valves in any direction (i.e. pro- or retraction of the ventral valves) caused a straightening of the S-shaped region of the ovipositor and a curving towards the dorsal side. However, during probing in a substrate, such curving was only observed upon protraction of the ventral valves. We hypothesize this is due to the interaction of the ovipositor with the substrate. Namely, the bevelled ventral valve tips generate substrate reaction forces that promote dorsal curving, while the bevelled tip of the dorsal valve generates substrate forces that promote ventral bending. The interaction between the ventral and dorsal valves straightens the S-shaped region of the ovipositor and enhances dorsal curving. This therefore facilitates strong shape changes of the tip only upon protraction of the ventral valves, while counteracting the ventral curving of the dorsal valve. These opposing mechanisms presumably result in an approximately straight protraction of the dorsal valve. In chapters 2 and 3 we describe how the wasps use the reciprocal valve movements when probing in stiff substrates. As such substrates presumably require strong forces during insertion, the reciprocal valve movements may indeed serve to avoid buckling. However, how the valves are actuated or the forces generated during probing have never been quantified. In chapter 5, we therefore investigated the ovipositor base and the muscles driving the movements of the valves. At the base, the valves attach to plate-like structures that are interconnected with a series of linkages. The muscles attach to these plates and can move them with respect to each other. Such movements also result in the movements of the valves. To analyse the mechanics of this linked system, we performed high-resolution computer tomography scans of wasps in different stages of the probing cycle. This allowed us to compare the configurational changes of the basal plates to the valve offset, and measure the muscle cross-sections and attachment sites. We also calculated the muscle moment arms and estimated the forces and moments of the most relevant musculature actuating the ovipositor movements, by assuming a tensile muscle stress previously reported for insect muscles. For the ventral valves only, we also calculated the forces the valves can exert onto the substrate. The dorsal valve can only be moved by moving the base that is linked inside the abdomen, and therefore force estimation could not be made. The displacement magnitude of the basal plates corresponded to the valve offset, indicating that the valves are indeed moved due to the changes in the arrangement of the basal plates. We also showed that the ventral valve plates move most during the probing cycle, while the magnitude of the dorsal valve plate movements is much smaller. This suggests that the ventral valves move along the dorsal valve, while the dorsal valve moves together with the abdomen during probing. Additionally, in the situation where the animal keeps its abdomen stationary, we estimated the maximal forces actuating the ventral valves. The estimated maximal pushing forces can be higher than the estimated buckling load of the unsupported ovipositor outside the substrate. Assuming the maximal pushing forces are required during probing, antibuckling mechanisms are needed to avoid damaging the ovipositor. Buckling can be limited (prevented) by either supporting the ovipositor outside the substrate with additional sheaths, employing the push–pull mechanism, or both. Subtracting the maximal estimated pushing and pulling forces on the ventral valves, results in a net pushing force that is very close to the buckling threshold of the ovipositor, albeit still slightly higher. The sheaths, although being flexible, might provide the additional support if needed. In this thesis, I show that multi-element probes are inserted into the substrate using reciprocal movements of the individual elements. These movements appear to be necessary in stiff substrates, which presumably require high pushing forces on a single element during probing. This is in accordance with the hypothesis that reciprocal valve movements serve as an anti-buckling mechanism. Additionally, such valve movements are also important for steering of the probe during insertion. The valve offset controls the shape of the probe tip and therefore the net substrate reaction forces that result in bending of the probe. Wasps evolved special structures that enhance the shape changes of their ovipositor tips and facilitate steering. Our findings may be interesting for a broad range of audiences. Entomologists, evolutionary biologists, and ecologists may find them useful when studying the diversification of probing insects, their evolutionary success, or their ecological interactions (e.g. insect–plant, parasite–host). The anti-buckling and steering mechanisms may be helpful when developing novel, man-made probes. These mechanisms allow for minimization of the probe thickness and accurate steering control, which minimizes substrate damage during probing. Our findings may be particularly useful in the development of slender, steerable needles for minimally invasive surgery.</p

Design, Modeling and Control of Micro-scale and Meso-scale Tendon-Driven Surgical Robots

Manual manipulation of passive surgical tools is time consuming with uncertain results in cases of navigating tortuous anatomy, avoiding critical anatomical landmarks, and reaching targets not located in the linear range of these tools. For example, in many cardiovascular procedures, manual navigation of a micro-scale passive guidewire results in increased procedure times and radiation exposure. This thesis introduces the design of two steerable guidewires: 1) A two degree-of-freedom (2-DoF) robotic guidewire with orthogonally oriented joints to access points in a three dimensional workspace, and 2) a micro-scale coaxially aligned steerable (COAST) guidewire robot that demonstrates variable and independently controlled bending length and curvature of the distal end. The 2-DoF guidewire features two micromachined joints from a tube of superelastic nitinol of outer diameter 0.78 mm. Each joint is actuated with two nitinol tendons. The joints that are used in this robot are called bidirectional asymmetric notch (BAN) joints, and the advantages of these joints are explored and analyzed. The design of the COAST robotic guidewire involves three coaxially aligned tubes with a single tendon running centrally through the length of the robot. The outer tubes are made from micromachined nitinol allowing for tendon-driven bending of the robot at variable bending curvatures, while an inner stainless steel tube controls the bending length of the robot. By varying the lengths of the tubes as well as the tendon, and by insertion and retraction of the entire assembly, various joint lengths and curvatures may be achieved. Kinematic and static models, a compact actuation system, and a controller for this robot are presented. The capability of the robot to accurately navigate through phantom anatomical bifurcations and tortuous angles is also demonstrated in three dimensional phantom vasculature. At the meso-scale, manual navigation of passive pediatric neuroendoscopes for endoscopic third ventriculostomy may not reach target locations in the patient's ventricle. This work introduces the design, analysis and control of a meso-scale two degree-of-freedom robotic bipolar electrocautery tool that increases the workspace of the neurosurgeon. A static model is proposed for the robot joints that avoids problems arising from pure kinematic control. Using this model, a control system is developed that comprises of a disturbance observer to provide precise force control and compensate for joint hysteresis. A handheld controller is developed and demonstrated in this thesis. To allow the clinician to estimate the shape of the steerable tools within the anatomy for both micro-scale and meso-scale tools, a miniature tendon force sensor and a high deflection shape sensor are proposed and demonstrated. The force sensor features a compact design consisting of a single LED, dual-phototransistor, and a dual-screen arrangement to increase the linear range of sensor output and compensate for external disturbances, thereby allowing force measurement of up to 21 N with 99.58 % accuracy. The shape sensor uses fiber Bragg grating based optical cable mounted on a micromachined tube and is capable of measuring curvatures as high as 145 /m. These sensors were incorporated and tested in the guidewire and the neuroendoscope tool robots and can provide robust feedback for closed-loop control of these devices in the future.Ph.D