V-ANFIS for Dealing with Visual Uncertainty for Force Estimation in Robotic Surgery

Accurate and robust estimation of applied forces in Robotic-Assisted Minimally Invasive Surgery is a very challenging task. Many vision-based solutions attempt to estimate the force by measuring the surface deformation after contacting the surgical tool. However, visual uncertainty, due to tool occlusion, is a major concern and can highly affect the results' precision. In this paper, a novel design of an adaptive neuro-fuzzy inference strategy with a voting step (V-ANFIS) is used to accommodate with this loss of information. Experimental results show a significant accuracy improvement from 50% to 77% with respect to other proposals.Peer ReviewedPostprint (published version

Force measurement capability for robotic assisted minimally invasive surgery systems

An automated laparoscopic instrument capable of non-invasive measurement of tip/tissue interaction forces for direct application in robotic assisted minimally invasive surgery systems_ is introduced in this paper. It has the capability to measure normal grasping forces as well as lateral interaction forces without any sensor mounted on the tip jaws. Further to non-invasive actuation of the tip, the proposed instrument is also able to change the grasping direction during surgical operation. Modular design of the instrument allows conversion between surgical modalities (e.g., grasping, cutting, and dissecting). The main focus of this paper is on evaluation of the grasping force capability of the proposed instrument. The mathematical formulation of fenestrated insert is presented and its non-linear behaviour is studied. In order to measure the stiffness of soft tissues, a device was developed that is also described in this paper. Tissue characterisation experiments were conducted and results are presented and analysed here. The experimental results verify the capability of the proposed instrument in accurately measuring grasping forces and in characterising artificial tissue samples of varying stiffness.<br /

Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.Peer ReviewedPostprint (author's final draft

Wireless Tissue Palpation for Intraoperative Detection of Lumps in the Soft Tissue

In an open surgery, identification of precise margins for curative tissue resection is performed by manual palpation. This is not the case for minimally invasive and robotic procedures, where tactile feedback is either distorted or not available. In this paper, we introduce the concept of intraoperative wireless tissue palpation. The wireless palpation probe (WPP) is a cylindrical device (15 mm in diameter, 60 mm in length) that can be deployed through a trocar incision and directly controlled by the surgeon to create a volumetric stiffness distribution map of the region of interest. This map can then be used to guide the tissue resection to minimize healthy tissue loss. The wireless operation prevents the need for a dedicated port and reduces the chance of instrument clashing in the operating field. The WPP is able to measure in real time the indentation pressure with a sensitivity of 34 Pa, the indentation depth with an accuracy of 0.68 mm, and the probe position with a maximum error of 11.3 mm in a tridimensional workspace. The WPP was assessed on the benchtop in detecting the local stiffness of two different silicone tissue simulators (elastic modulus ranging from 45 to 220 kPa), showing a maximum relative error below 5%. Then, in vivo trials were aimed to identify an agar-gel lump injected into a porcine liver and to assess the device usability within the frame of a laparoscopic procedure. The stiffness map created intraoperatively by the WPP was compared with a map generated ex vivo by a standard uniaxial material tester, showing less than 8% local stiffness error at the site of the lump

Robot Autonomy for Surgery

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots

Sight to touch: 3D diffeomorphic deformation recovery with mixture components for perceiving forces in robotic-assisted surgery

Robotic-assisted minimally invasive surgical sys-tems suffer from one major limitation which is the lack of interaction forces feedback. The restricted sense of touch hinders the surgeons’ performance and reduces their dexterity and precision during a procedure. In this work, we present a sensory substitution approach that relies on visual stimuli to transmit the tool-tissue interaction forces to the operating surgeon. Our approach combines a 3D diffeomorphic defor-mation mapping with a generative model to precisely label the force level. The main highlights of our approach are that the use of diffeomorphic transformation ensures anatomical structure preservation and the label assignment is based on a parametric form of several mixture elements. We performed experimentations on both ex-vivo and in-vivo datasets and offer careful numerical results evaluating our approach. The results show that our solution has an error measure less than 1mm in all directions and an average labeling error of 2.05%. It can also be applicable to other scenarios that require force feedback such as microsurgery, knot tying or needle-based procedures.Peer ReviewedPostprint (author's final draft

Microfabricated tactile sensors for biomedical applications: a review

During the last decades, tactile sensors based on different sensing principles have been developed due to the growing interest in robotics and, mainly, in medical applications. Several technological solutions have been employed to design tactile sensors; in particular, solutions based on microfabrication present several attractive features. Microfabrication technologies allow for developing miniaturized sensors with good performance in terms of metrological properties (e.g., accuracy, sensitivity, low power consumption, and frequency response). Small size and good metrological properties heighten the potential role of tactile sensors in medicine, making them especially attractive to be integrated in smart interfaces and microsurgical tools. This paper provides an overview of microfabricated tactile sensors, focusing on the mean principles of sensing, i.e., piezoresistive, piezoelectric and capacitive sensors. These sensors are employed for measuring contact properties, in particular force and pressure, in three main medical fields, i.e., prosthetics and artificial skin, minimal access surgery and smart interfaces for biomechanical analysis. The working principles and the metrological properties of the most promising tactile, microfabricated sensors are analyzed, together with their application in medicine. Finally, the new emerging technologies in these fields are briefly described

Force Measurement Methods in Telerobotic Surgery: Implications for End-Effector Manufacture

Haptic feedback in telesurgical applications refers to the relaying of position and force information from a remote surgical site to the surgeon in real-time during a surgical procedure. This feedback, coupled with visual information via microscopic cameras, has the potential to provide the surgeon with additional ‘feel’ for the manipulations being performed at the instrument-biological tissue interface. This increased sensitivity has many associated benefits which include, but are not limited to; minimal tissue damage, reduced recuperation periods, and less patient trauma. The inclusion of haptic feedback leads to reduction in surgeon fatigue which contributes to enhanced performance during operation. Commercially available Minimally Invasive Robotic Surgical (MIRS) systems are being widely used, the best-known examples being from the daVinci® by Intuitive Surgical Inc. However, currently these systems do not possess force feedback capability which therefore restricts their use during many delicate and complex procedures. The ideal system would consist of a multi-degree-of-freedom framework which includes end-effector instruments with embedded force sensing included. A force sensing characterisation platform has been developed by this group which facilitates the evaluation of force sensing technologies. Surgical scissors have been chosen as the instrument and biological tissue phantom specimens have been used during testing. This test-bed provides accurate, repeatable measurements of the forces produced at the interface between the tissue and the scissor blades during cutting using conventional sensing technologies. The primary focus of this paper is to provide a review of the traditional and developing force sensing technologies with a view to establishing the most appropriate solution for this application. The impact that an appropriate sensing technology has on the manufacturability of the instrument end-effector is considered. Particular attention is given to the issues of embedding the force sensing transducer into the instrument tip