Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

Medical Robotics

The first generation of surgical robots are already being installed in a number of operating rooms around the world. Robotics is being introduced to medicine because it allows for unprecedented control and precision of surgical instruments in minimally invasive procedures. So far, robots have been used to position an endoscope, perform gallbladder surgery and correct gastroesophogeal reflux and heartburn. The ultimate goal of the robotic surgery field is to design a robot that can be used to perform closed-chest, beating-heart surgery. The use of robotics in surgery will expand over the next decades without any doubt. Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. The aim of this book is to provide an overview of the state-of-art, to present new ideas, original results and practical experiences in this expanding area. Nevertheless, many chapters in the book concern advanced research on this growing area. The book provides critical analysis of clinical trials, assessment of the benefits and risks of the application of these technologies. This book is certainly a small sample of the research activity on Medical Robotics going on around the globe as you read it, but it surely covers a good deal of what has been done in the field recently, and as such it works as a valuable source for researchers interested in the involved subjects, whether they are currently “medical roboticists” or not

A Variable Stiffness Robotic Probe for Soft Tissue Palpation

During abdominal palpation diagnosis, a medical practitioner would change the stiffness of their fingers in order to improve the detection of hard nodules or abnormalities in soft tissue to maximize the haptic information gain via tendons. Our recent experiments using a controllable stiffness robotic probe representing a human finger also confirmed that such stiffness control in the finger can enhance the accuracy of detecting hard nodules in soft tissue. However, the limited range of stiffness achieved by the antagonistic springs variable stiffness joint subject to size constraints made it unsuitable for a wide range of physical examination scenarios spanning from breast to abdominal examination. In this letter, we present a new robotic probe based on a variable lever mechanism able to achieve stiffness ranging from 0.64 to 1.06 N ⋅m/rad that extends the maximum stiffness by around 16 times and the stiffness range by 33 times. This letter presents the mechanical model of the novel probe, the finite element simulation as well as experimental characterization of the stiffness response for lever actuation

A Variable Stiffness Robotic Probe for Soft Tissue Palpation

During abdominal palpation diagnosis, a medical practitioner would change the stiffness of their fingers in order to improve the detection of hard nodules or abnormalities in soft tissue to maximize the haptic information gain via tendons. Our recent experiments using a controllable stiffness robotic probe representing a human finger also confirmed that such stiffness control in the finger can enhance the accuracy of detecting hard nodules in soft tissue. However, the limited range of stiffness achieved by the antagonistic springs variable stiffness joint subject to size constraints made it unsuitable for a wide range of physical examination scenarios spanning from breast to abdominal examination. In this letter, we present a new robotic probe based on a variable lever mechanism able to achieve stiffness ranging from 0.64 to 1.06 N·m/rad that extends the maximum stiffness by around 16 times and the stiffness range by 33 times. This letter presents the mechanical model of the novel probe, the finite element simulation as well as experimental characterization of the stiffness response for lever actuation.This work was supported by The United Kingdom Engineering and Physical Sciences Research Council under MOTION Grant EP/N03211X/2

Tactile Sensing System for Lung Tumour Localization during Minimally Invasive Surgery

Video-assisted thoracoscopie surgery (VATS) is becoming a prevalent method for lung cancer treatment. However, VATS suffers from the inability to accurately relay haptic information to the surgeon, often making tumour localization difficult. This limitation was addressed by the design of a tactile sensing system (TSS) consisting of a probe with a tactile sensor and interfacing visualization software. In this thesis, TSS performance was tested to determine the feasibility of implementing the system in VATS. This was accomplished through a series of ex vivo experiments in which the tactile sensor was calibrated and the visualization software was modified to provide haptic information visually to the user, and TSS performance was compared using human and robot palpation methods, and conventional VATS instruments. It was concluded that the device offers the possibility of providing to the surgeon the haptic information lost during surgery, thereby mitigating one of the current limitations of VATS

Haptische Mensch-Maschine-Schnittstelle für ein laparoskopisches Chirurgie-System

Für eine Vielzahl von Operationen im Bauchraum ist heute die Laparoskopie Stand der Technik, so z.B. die Cholezytektomie zur Entfernung der Gallenblase. Hierbei handelt es sich um ein minimalinvasives Verfahren bei dem der Zugang zum Operationsgebiet durch kleinste Schnitte in der Bauchdecke des Patienten erfolgt. Bei der Operation kommen lange starre Instrumente zu Einsatz. Im Gegensatz zu einer offenen Operation haben die Hände des Chirurgen keinen direkten Zugang zum operierten Gewebe. Ein Abtasten des Gewebes ist nicht möglich, der haptische Sinn zur Diagnose und Navigation im Operationsgebiet steht dem Operateur folglich nicht zur Verfügung. Diese Einschränkung erhöht die Komplexität laparoskopischer Eingriffe erheblich. Auch die Beweglichkeit im Operationsfeld ist stark eingeschränkt. Eine technische Antwort auf diese Einschränkungen sind haptische Telemanipulationssysteme. Sie bestehen aus einer angetriebenen Instrumentenspitze sowie einem haptischen Bedienelement, das die Kontaktkräfte zwischen Instrumentenspitze und Gewebe an den Bediener meldet. Hierzu erfasst ein Kraftsensor an der Instrumentenspitze die auftretenden Kontaktkräfte. Antriebe im Bedienelement erzeugen daraus eine Kraftinformation und leiten sie über einen Mechanismus an den Bediener weiter. Die vorliegende Arbeit befasst sich mit der Erweiterung der Entwurfsmethodik für haptische Bedienelemente und der Realisierung eines neuartigen Bedienelements. Basis ist eine Analyse des chirurgischen Szenarios in der minimalinvasiven Leberchirurgie. Daraus leitet sich das Entwurfsziel eines haptischen Bedienelementes mit drei kartesischen Freiheitsgraden ab. Auf Grund ihrer guten dynamischen Eigenschaften sind besonders parallelkinematische Mechanismen zur Übertragung haptischer Informationen geeignet. Sie zeichnen sich durch eine große Struktursteifigkeit und geringe bewegte Massen aus. Ihr kinematisches Übertragungsverhalten ist hingegen meist komplex. Aus der Analyse der kinematischen Bedingungen für ein rein kartesisches Ausgangsverhalten ergibt sich ein möglicher Lösungsraum geeigneter Topologien. Alle bestehen aus drei Beinen mit je 5 Gelenkfreiheitsgraden, einer Basis-Plattform und einer Tool-Centre-Point-Plattform zur Ausgabe der haptischen Information. Für den vorliegenden Fall ist eine RUU- bzw. DELTA-Struktur geeignet. Diese Struktur übersetzt drei Antriebsmomente in eine rein kartesische Ausgabe. Basierend auf der Analyse der kinematischen Entwurfsziele für haptische Mechanismen erfolgte eine Auslegung des Mechanismus im Hinblick auf isotropes, d.h. richtungsunabhängiges Übertragungsverhalten. Charakteristisches Maß ist die globale Konditionszahl. Entscheidend für die Qualität der haptischen Rückmeldung ist das dynamische Übertragungsverhalten haptischer Bedienelemente. Für eindimensionale Systeme ist in der Literatur zur Modellierung der Zwei-Tor Ansatz basierend auf der elektromechanischen Netzwerktheorie eingeführt. Im Rahmen dieser Arbeit erfolgt erstmalig die Erweiterung auch für den mehrdimensionalen Fall. Damit ist es möglich, auch die dynamischen Eigenschaften mehrdimensionaler Mechanismen mit dem Zwei-Tor Ansatz abzubilden. Dies erlaubt Anwendung des Entwurfsverfahrens der "Transparenz" für mehrdimensionale Systeme. Zur Analyse der mechanischen Eigenschaften des operierten Gewebes entstehen zwei Messplätze für die Frequenzbereiche f = 10...10^4 Hz (taktile Wahrnehmung) und f=DC...50 Hz (kinästhetische Wahrnehmung). Sie ermöglichen die messtechnische Charakterisierung der mechanischen Impedanz und die Ableitung mechanischer Schaltungen. Damit lässt sich die Impedanz des Gewebes rechnerisch im Gütekriterium der Transparenz zur Bewertung eines haptischen Telemanipulationssystems einsetzen. Die Realisierung eines haptischen Bedienelements erfolgt für ein neuartiges, tragbares Teleoperationssystem. Das Bedienkonzept ist an Hand eines ergonomischen Funktionsmusters im Tierversuch evaluiert. Kernkomponente ist ein haptisches Joystick mit drei kartesischen Freiheitsgraden durch einen RUU-Mechanismus. Der Arbeitsraum beträgt 743,5 cm³. Das Bedienelement ist mit einer Impedanz-gesteuerten Systemstruktur entworfen und feinwerktechnisch umgesetzt. Als Antriebe kommen drei EC-Motoren zum Einsatz. Mit einem maximalen Moment von 0,2 Nm erzeugen sie eine haptische Rückmeldung von 5N in 82% des Volumens im Arbeitsraum. Die zum Betrieb erforderlichen kinematischen Berechnungen sind auf einem Steuerrechner implementiert. Zusammen mit der Leistungselektronik ist dieser in einem mobilen Rack integriert. Der Nachweis der Funktionsfähigkeit erfolgt an einem experimentellen Telemanipulationssystem im Laborbetrieb

A Computational Image-Based Guidance System for Precision Laparoscopy

This dissertation presents our progress toward the goal of building a computational image-based guidance system for precision laparoscopy; in particular, laparoscopic liver resection. As we aim to keep our working goal as simple as possible, we have focused on the most important questions of laparoscopy - predicting the new location of tumors and resection plane after a liver maneuver during surgery. Our approach was to build a mechanical model of the organ based on pre-operative images and register it to intra-operative data. We proposed several practical and cost-effective methods to obtain the intra-operative data in the real procedure. We integrated all of them into a framework on which we could develop new techniques without redoing everything. To test the system, we did an experiment with a porcine liver in a controlled setup: a wooden lever was used to elevate a part of the liver to access the posterior of the liver. We were able to confirm that our model has decent accuracy for tumor location (approximately 2 mm error) and resection plane (1% difference in remaining liver volume after resection). However, the overall shape of the liver and the fiducial markers still left a lot to be desired. For further corrections to the model, we also developed an algorithm to reconstruct the 3D surface of the liver utilizing Smart Trocars, a new surgical instrument recognition system. The algorithm had been verified by an experiment on a plastic model using the laparoscopic camera as a mean to obtain surface images. This method had millimetric accuracy provided the angle between two endoscope views is not too small. In an effort to transit our research from porcine livers to human livers, in-vivo experiments had been conducted on cadavers. From those studies, we found a new method that used a high-frequency ventilator to eliminate respiratory motion. The framework showed the potential to work on real organs in clinical settings. Hence, the studies on cadavers needed to be continued to improve those techniques and complete the guidance system.Computer Science, Department o

Realistic tool-tissue interaction models for surgical simulation and planning

Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in pre- and intra-operative surgical planning. Realistic modeling of medical interventions involving tool-tissue interactions has been considered to be a key requirement in the development of high-fidelity simulators and planners. The soft-tissue constitutive laws, organ geometry and boundary conditions imposed by the connective tissues surrounding the organ, and the shape of the surgical tool interacting with the organ are some of the factors that govern the accuracy of medical intervention planning.\ud \ud This thesis is divided into three parts. First, we compare the accuracy of linear and nonlinear constitutive laws for tissue. An important consequence of nonlinear models is the Poynting effect, in which shearing of tissue results in normal force; this effect is not seen in a linear elastic model. The magnitude of the normal force for myocardial tissue is shown to be larger than the human contact force discrimination threshold. Further, in order to investigate and quantify the role of the Poynting effect on material discrimination, we perform a multidimensional scaling study. Second, we consider the effects of organ geometry and boundary constraints in needle path planning. Using medical images and tissue mechanical properties, we develop a model of the prostate and surrounding organs. We show that, for needle procedures such as biopsy or brachytherapy, organ geometry and boundary constraints have more impact on target motion than tissue material parameters. Finally, we investigate the effects surgical tool shape on the accuracy of medical intervention planning. We consider the specific case of robotic needle steering, in which asymmetry of a bevel-tip needle results in the needle naturally bending when it is inserted into soft tissue. We present an analytical and finite element (FE) model for the loads developed at the bevel tip during needle-tissue interaction. The analytical model explains trends observed in the experiments. We incorporated physical parameters (rupture toughness and nonlinear material elasticity) into the FE model that included both contact and cohesive zone models to simulate tissue cleavage. The model shows that the tip forces are sensitive to the rupture toughness. In order to model the mechanics of deflection of the needle, we use an energy-based formulation that incorporates tissue-specific parameters such as rupture toughness, nonlinear material elasticity, and interaction stiffness, and needle geometric and material properties. Simulation results follow similar trends (deflection and radius of curvature) to those observed in macroscopic experimental studies of a robot-driven needle interacting with gels

A Sensorized Instrument for Minimally Invasive Surgery for the Measurement of Forces during Training and Surgery: Development and Applications

The reduced access conditions present in Minimally Invasive Surgery (MIS) affect the feel of interaction forces between the instruments and the tissue being treated. This loss of haptic information compromises the safety of the procedure and must be overcome through training. Haptics in MIS is the subject of extensive research, focused on establishing force feedback mechanisms and developing appropriate sensors. This latter task is complicated by the need to place the sensors as close as possible to the instrument tip, as the measurement of forces outside of the patient\u27s body does not represent the true tool--tissue interaction. Many force sensors have been proposed, but none are yet available for surgery. The objectives of this thesis were to develop a set of instruments capable of measuring tool--tissue force information in MIS, and to evaluate the usefulness of force information during surgery and for training and skills assessment. To address these objectives, a set of laparoscopic instruments was developed that can measure instrument position and tool--tissue interaction forces in multiple degrees of freedom. Different design iterations and the work performed towards the development of a sterilizable instrument are presented. Several experiments were performed using these instruments to establish the usefulness of force information in surgery and training. The results showed that the combination of force and position information can be used in the development of realistic tissue models or haptic interfaces specifically designed for MIS. This information is also valuable in order to create tactile maps to assist in the identification of areas of different stiffness. The real-time measurement of forces allows visual force feedback to be presented to the surgeon. When applied to training scenarios, the results show that experience level correlates better with force-based metrics than those currently used in training simulators. The proposed metrics can be automatically computed, are completely objective, and measure important aspects of performance. The primary contribution of this thesis is the design and development of highly versatile instruments capable of measuring force and position during surgery. A second contribution establishes the importance and usefulness of force data during skills assessment, training and surgery