A surgical system for automatic registration, stiffness mapping and dynamic image overlay

In this paper we develop a surgical system using the da Vinci research kit (dVRK) that is capable of autonomously searching for tumors and dynamically displaying the tumor location using augmented reality. Such a system has the potential to quickly reveal the location and shape of tumors and visually overlay that information to reduce the cognitive overload of the surgeon. We believe that our approach is one of the first to incorporate state-of-the-art methods in registration, force sensing and tumor localization into a unified surgical system. First, the preoperative model is registered to the intra-operative scene using a Bingham distribution-based filtering approach. An active level set estimation is then used to find the location and the shape of the tumors. We use a recently developed miniature force sensor to perform the palpation. The estimated stiffness map is then dynamically overlaid onto the registered preoperative model of the organ. We demonstrate the efficacy of our system by performing experiments on phantom prostate models with embedded stiff inclusions.Comment: International Symposium on Medical Robotics (ISMR 2018

V-ANFIS for Dealing with Visual Uncertainty for Force Estimation in Robotic Surgery

Accurate and robust estimation of applied forces in Robotic-Assisted Minimally Invasive Surgery is a very challenging task. Many vision-based solutions attempt to estimate the force by measuring the surface deformation after contacting the surgical tool. However, visual uncertainty, due to tool occlusion, is a major concern and can highly affect the results' precision. In this paper, a novel design of an adaptive neuro-fuzzy inference strategy with a voting step (V-ANFIS) is used to accommodate with this loss of information. Experimental results show a significant accuracy improvement from 50% to 77% with respect to other proposals.Peer ReviewedPostprint (published version

Planning and Real Time Control of a Minimally Invasive Robotic Surgery System

This paper introduces the planning and control software of a teleoperating robotic system for minimally invasive surgery. It addresses the problem of how to organize a complex system with 41 degrees of freedom including robot setup planning, force feedback control and nullspace handling with three robotic arms. The planning software is separated into sequentially executed planning and registration procedures. An optimal setup is first planned in virtual reality and then adapted to variations in the operating room. The real time control system is composed of hierarchical layers. The design is flexible and expandable without losing performance. Structure, functionality and implementation of planning and control are described. The robotic system provides the surgeon with an intuitive hand-eye-coordination and force feedback in teleoperation for both hands

Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.Peer ReviewedPostprint (author's final draft

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Shape sensing of miniature snake-like robots using optical fibers

Snake like continuum robots are increasingly used for minimally invasive surgery. Most robotic devices of this sort that have been reported to date are controlled in an open loop manner. Using shape sensing to provide closed loop feedback would allow for more accurate control of the robot's position and, hence, more precise surgery. Fiber Bragg Gratings, magnetic sensors and optical reflectance sensors have all been reported for this purpose but are often limited by their cost, size, stiffness or complexity of fabrication. To address this issue, we designed, manufactured and tested a prototype two-link robot with a built-in fiber-optic shape sensor that can deliver and control the position of a CO 2 -laser fiber for soft tissue ablation. The shape sensing is based on optical reflectance, and the device (which has a 4 mm outer diameter) is fabricated using 3D printing. Here we present proof-of-concept results demonstrating successful shape sensing - i.e. measurement of the angular displacement of the upper link of the robot relative to the lower link - in real time with a mean measurement error of only 0.7°