Combining Differential Kinematics and Optical Flow for Automatic Labeling of Continuum Robots in Minimally Invasive Surgery

International audienceThe segmentation of continuum robots in medical images can be of interest for analyzing surgical procedures or for controlling them. However, the automatic segmentation of continuous and flexible shapes is not an easy task. On one hand conventional approaches are not adapted to the specificities of these instruments, such as imprecise kinematic models, and on the other hand techniques based on deep-learning showed interesting capabilities but need many manually labeled images. In this article we propose a novel approach for segmenting continuum robots on endoscopic images, which requires no prior on the instrument visual appearance and no manual annotation of images. The method relies on the use of the combination of kinematic models and differential kinematic models of the robot and the analysis of optical flow in the images. A cost function aggregating information from the acquired image, from optical flow and from robot encoders is optimized using particle swarm optimization and provides estimated parameters of the pose of the continuum instrument and a mask defining the instrument in the image. In addition a temporal consistency is assessed in order to improve stochastic optimization and reject outliers. The proposed approach has been tested for the robotic instruments of a flexible endoscopy platform both for benchtop acquisitions and an in vivo video. The results show the ability of the technique to correctly segment the instruments without a prior, and in challenging conditions. The obtained segmentation can be used for several applications, for instance for providing automatic labels for machine learning techniques

Design of a Surgical Manipulator System for Lumbar Discectomy Procedures

A 3D printed surgical master-slave manipulator system for minimally invasive lumbar discectomy procedure is proposed. Discectomy is the surgery to remove the herniated disc material that is pressing on a nerve root or spinal cord. This surgery is performed to relieve pain or numbness caused by the pressure on the nerve. The workspace is limited (\u3c 27 cm3) and the manipulator has to go through a 3.175mm (0.125”) diameter channel. The proposed system is comprised of a family of manipulators that can work alone or co-operatively to perform tasks required in the surgery. In the proposed system, the manipulator is 3D printed with multiple materials, with flexible links acting as joints of the mechanism. These flexible links are actuated by cables which provide sufficient forces for actuation in the surgical workspace. In this thesis, existing surgical techniques are investigated and a new surgical system is proposed. Various design ideas are presented and evaluated for manufacturing and assembly. Finally, the proposed mechanisms are modeled and tested for their capability to assist the surgeon to perform tasks required for the surgery

Intelligent Information-Guided Robotic Surgery

Laparoscopic surgery is minimally invasive, providing various benefits for patients. On the other hand, it is technically demanding for physicians due to limited dexterity of tools, limited vision. In order to cope with those limitations, recent various engineering technologies are trying to help surgeon. Robotics is one of the major technologies in this field. Until today, da Vinci has been only one such robot. But recently, many other robotic systems are under development. Those new robots are introduced in this chapter first. Other than robotics, or in conjunction with robotics, navigation technologies are getting popularity in clinical use. Navigation is a technology that provides useful information such as preoperative images or distance between tool and lesion, etc. to surgeon. Our experience in clinical use of navigation system in robotic surgery is introduced. Finally, technologies applied for the training of surgeon are introduced and described

Soft Robot-Assisted Minimally Invasive Surgery and Interventions: Advances and Outlook

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry's appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery

Cable-driven parallel mechanisms for minimally invasive robotic surgery

Minimally invasive surgery (MIS) has revolutionised surgery by providing faster recovery times, less post-operative complications, improved cosmesis and reduced pain for the patient. Surgical robotics are used to further decrease the invasiveness of procedures, by using yet smaller and fewer incisions or using natural orifices as entry point. However, many robotic systems still suffer from technical challenges such as sufficient instrument dexterity and payloads, leading to limited adoption in clinical practice. Cable-driven parallel mechanisms (CDPMs) have unique properties, which can be used to overcome existing challenges in surgical robotics. These beneficial properties include high end-effector payloads, efficient force transmission and a large configurable instrument workspace. However, the use of CDPMs in MIS is largely unexplored. This research presents the first structured exploration of CDPMs for MIS and demonstrates the potential of this type of mechanism through the development of multiple prototypes: the ESD CYCLOPS, CDAQS, SIMPLE, neuroCYCLOPS and microCYCLOPS. One key challenge for MIS is the access method used to introduce CDPMs into the body. Three different access methods are presented by the prototypes. By focusing on the minimally invasive access method in which CDPMs are introduced into the body, the thesis provides a framework, which can be used by researchers, engineers and clinicians to identify future opportunities of CDPMs in MIS. Additionally, through user studies and pre-clinical studies, these prototypes demonstrate that this type of mechanism has several key advantages for surgical applications in which haptic feedback, safe automation or a high payload are required. These advantages, combined with the different access methods, demonstrate that CDPMs can have a key role in the advancement of MIS technology.Open Acces

Shape Memory Alloy Actuators and Sensors for Applications in Minimally Invasive Interventions

Reduced access size in minimally invasive surgery and therapy (MIST) poses several restriction on the design of the dexterous robotic instruments. The instruments should be developed that are slender enough to pass through the small sized incisions and able to effectively operate in a compact workspace. Most existing robotic instruments are operated by big actuators, located outside the patient’s body, that transfer forces to the end effector via cables or magnetically controlled actuation mechanism. These instruments are certainly far from optimal in terms of their cost and the space they require in operating room. The lack of adequate sensing technologies make it very challenging to measure bending of the flexible instruments, and to measure tool-tissue contact forces of the both flexible and rigid instruments during MIST. Therefore, it requires the development of the cost effective miniature actuators and strain/force sensors. Having several unique features such as bio-compatibility, low cost, light weight, large actuation forces and electrical resistivity variations, the shape memory alloys (SMAs) show promising applications both as the actuators and strain sensors in MIST. However, highly nonlinear hysteretic behavior of the SMAs hinders their use as actuators. To overcome this problem, an adaptive artificial neural network (ANN) based Preisach model and a model predictive controller have been developed in this thesis to precisely control the output of the SMA actuators. A novel ultra thin strain sensor is also designed using a superelastic SMA wire, which can be used to measure strain and forces for many surgical and intervention instruments. A da Vinci surgical instrument is sensorized with these sensors in order to validate their force sensing capability

Modeling, Analysis, Force Sensing and Control of Continuum Robots for Minimally Invasive Surgery

This dissertation describes design, modeling and application of continuum robotics for surgical applications, specifically parallel continuum robots (PCRs) and concentric tube manipulators (CTMs). The introduction of robotics into surgical applications has allowed for a greater degree of precision, less invasive access to more remote surgical sites, and user-intuitive interfaces with enhanced vision systems. The most recent developments have been in the space of continuum robots, whose exible structure create an inherent safety factor when in contact with fragile tissues. The design challenges that exist involve balancing size and strength of the manipulators, controlling the manipulators over long transmission pathways, and incorporating force sensing and feedback from the manipulators to the user. Contributions presented in this work include: (1) prototyping, design, force sensing, and force control investigations of PCRs, and (2) prototyping of a concentric tube manipulator for use in a standard colonoscope. A general kinetostatic model is presented for PCRs along with identification of multiple physical constraints encountered in design and construction. Design considerations and manipulator capabilities are examined in the form of matrix metrics and ellipsoid representations. Finally, force sensing and control are explored and experimental results are provided showing the accuracy of force estimates based on actuation force measurements and control capabilities. An overview of the design requirements, manipulator construction, analysis and experimental results are provided for a CTM used as a tool manipulator in a traditional colonoscope. Currently, tools used in colonoscopic procedures are straight and exit the front of the scope with 1 DOF of operation (jaws of a grasper, tightening of a loop, etc.). This research shows that with a CTM deployed, the dexterity of these tools can be increased dramatically, increasing accuracy of tool operation, ease of use and safety of the overall procedure. The prototype investigated in this work allows for multiple tools to be used during a single procedure. Experimental results show the feasibility and advantages of the newly-designed manipulators