Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

The Next-Generation Surgical Robots

The chronicle of surgical robots is short but remarkable. Within 20 years since the regulatory approval of the first surgical robot, more than 3,000 units were installed worldwide, and more than half a million robotic surgical procedures were carried out in the past year alone. The exceptionally high speeds of market penetration and expansion to new surgical areas had raised technical, clinical, and ethical concerns. However, from a technological perspective, surgical robots today are far from perfect, with a list of improvements expected for the next-generation systems. On the other hand, robotic technologies are flourishing at ever-faster paces. Without the inherent conservation and safety requirements in medicine, general robotic research could be substantially more agile and explorative. As a result, various technical innovations in robotics developed in recent years could potentially be grafted into surgical applications and ignite the next major advancement in robotic surgery. In this article, the current generation of surgical robots is reviewed from a technological point of view, including three of possibly the most debated technical topics in surgical robotics: vision, haptics, and accessibility. Further to that, several emerging robotic technologies are highlighted for their potential applications in next-generation robotic surgery

Haptics in Robot-Assisted Surgery: Challenges and Benefits

Robotic surgery is transforming the current surgical practice, not only by improving the conventional surgical methods but also by introducing innovative robot-enhanced approaches that broaden the capabilities of clinicians. Being mainly of man-machine collaborative type, surgical robots are seen as media that transfer pre- and intra-operative information to the operator and reproduce his/her motion, with appropriate filtering, scaling, or limitation, to physically interact with the patient. The field, however, is far from maturity and, more critically, is still a subject of controversy in medical communities. Limited or absent haptic feedback is reputed to be among reasons that impede further spread of surgical robots. In this paper objectives and challenges of deploying haptic technologies in surgical robotics is discussed and a systematic review is performed on works that have studied the effects of providing haptic information to the users in major branches of robotic surgery. It has been tried to encompass both classical works and the state of the art approaches, aiming at delivering a comprehensive and balanced survey both for researchers starting their work in this field and for the experts

A haptic training environment for the heart myoblast cell injection procedure

The heart muscle of a cardiac arrest victim continues to accumulate damage throughout its lifetime. This reduces the heart\u27s ability to pump sufficient oxygen and nutrient blood to meet the body\u27s needs. Medical researchers have shown that direct injection of pre-harvested skeletal myoblast cells into the heart can restore some muscle function [1]. This operative procedure usually necessitates the surgeon to open a patient\u27s chest. The open chest procedure is usually a lengthy process and often extends the recovery time of the patient. Alternatively, a high accuracy surgical aid robotic system can be used to assist the thoracoscopic surgery [2][3]. While the robotic surgical method aids faster patient recovery, a less experienced surgeon can potentially cause damage to surrounding tissue. This paper presents a study into the development of a virtual haptically-enabled heart myoblast injection simulation environment, which can be used to train new surgeons to get hands on experience with the process. The paper also discusses the development of a generic constraint motion technique for needle insertion. Experiments on human performance measures and efficacy, while interacting with haptic feedback training models, are also presented. The experiment involved 10 operators, with each person repeating the needle insertion and injection 10 times. A notable improvement in the task execution time with the number of repetitions was observed. Operators improved their time by up to 300% compared to their first training attempt for a static heart scenario. Under a dynamic heart motion, operator\u27s performance was slightly lower, with the successful rate of completing the experiment reduced from 84% to 75%

Artificial intelligence surgery: how do we get to autonomous actions in surgery?

Most surgeons are skeptical as to the feasibility of autonomous actions in surgery. Interestingly, many examples of autonomous actions already exist and have been around for years. Since the beginning of this millennium, the field of artificial intelligence (AI) has grown exponentially with the development of machine learning (ML), deep learning (DL), computer vision (CV) and natural language processing (NLP). All of these facets of AI will be fundamental to the development of more autonomous actions in surgery, unfortunately, only a limited number of surgeons have or seek expertise in this rapidly evolving field. As opposed to AI in medicine, AI surgery (AIS) involves autonomous movements. Fortuitously, as the field of robotics in surgery has improved, more surgeons are becoming interested in technology and the potential of autonomous actions in procedures such as interventional radiology, endoscopy and surgery. The lack of haptics, or the sensation of touch, has hindered the wider adoption of robotics by many surgeons; however, now that the true potential of robotics can be comprehended, the embracing of AI by the surgical community is more important than ever before. Although current complete surgical systems are mainly only examples of tele-manipulation, for surgeons to get to more autonomously functioning robots, haptics is perhaps not the most important aspect. If the goal is for robots to ultimately become more and more independent, perhaps research should not focus on the concept of haptics as it is perceived by humans, and the focus should be on haptics as it is perceived by robots/computers. This article will discuss aspects of ML, DL, CV and NLP as they pertain to the modern practice of surgery, with a focus on current AI issues and advances that will enable us to get to more autonomous actions in surgery. Ultimately, there may be a paradigm shift that needs to occur in the surgical community as more surgeons with expertise in AI may be needed to fully unlock the potential of AIS in a safe, efficacious and timely manner

Modular instrument for a haptically-enabled robotic surgical system (HeroSurg)

To restore the sense of touch in robotic surgical systems, a modular force feedback-enabled laparoscopic instrument is developed and employed in a robotic-assisted minimally invasive surgical system (HeroSurg). Strain gauge technology is incorporated into the instrument to measure tip/tissue lateral interaction forces. The modularity feature of the proposed instrument makes it interchangeable between various tip types of different functionalities, e.g., cutter, grasper, and dissector, without losing force sensing capability. Series of experiments are conducted and results are reported to evaluate force sensing capability of the instrument. The results reveal mean errors of 1.32 g and 1.98° in the measurements of tip/tissue load magnitude and direction across all experiments, respectively

Multimodality with Eye tracking and Haptics: A New Horizon for Serious Games?

The goal of this review is to illustrate the emerging use of multimodal virtual reality that can benefit learning-based games. The review begins with an introduction to multimodal virtual reality in serious games and we provide a brief discussion of why cognitive processes involved in learning and training are enhanced under immersive virtual environments. We initially outline studies that have used eye tracking and haptic feedback independently in serious games, and then review some innovative applications that have already combined eye tracking and haptic devices in order to provide applicable multimodal frameworks for learning-based games. Finally, some general conclusions are identified and clarified in order to advance current understanding in multimodal serious game production as well as exploring possible areas for new applications

Haptic assessment of tissue stiffness in locating and identifying gynaecological cancer in human tissue

Gynaecological surgeons are not able to gather adequate tissue feedback during minimal access surgery for cancer treatment. This can result in failure to locate tumour boundaries and to ensure these are completely resected within tumour-free resection margins. Surgeons achieve significantly better surgical and oncological outcomes if they can identify the precise location of a gynaecological tumour. Indeed, the true nature of tumour, whether benign or cancerous, is often not known prior to surgery. If more details were available in relation to the characteristics that differentiate gynaecological cancer in tumours, this would enable more accurate diagnosis and help in the planning of surgery. HYPOTHESIS: Haptic technology has the potential to enhance the surgeon’s degree of perception during minimal access surgery. Alteration in tissue stiffness in gynaecological tumours, thought to be associated with the accelerated multiplication of cancer cells, should allow their location to be identified and help in determining the likelihood of malignancy. METHOD: Setting: (i) Guy's & St Thomas' Hospital (ii) Dept of Informatics (King's College London).Permission from the National Research Ethics Committee and Research & Development (R&D) approval were sought from the National Health Service. The Phantom Omni, capable of 3D motion tracking, attached to a nano-17 force sensor, was used to capture real-time position data and force data. Uniaxial indentation palpation behaviour was used. The indentation depth was calculated using the displacement of the probe from the surface to the deepest point for each contact. The tissue stiffness (TS) was then calculated.The haptic probe was tested first on silicone models with embedded nodules mimicking tumour(s). This was followed by assessing TS ex-vivo using a haptic probe on fresh human gynaecological organs that had been removed in surgery. Tissue stiffness maps were generated in real time using the haptic device by converting stiffness values into RGB values. Surgeons also manually palpated and recorded the site of the tumour. Histology was used as the gold standard for location and cancer diagnosis. Manual palpation and haptic data were compared for accuracy on tumour location. The tissue stiffness calculated by the haptic probe was compared in cancer and control specimens. Several data analysis techniques were applied to derive results.CONTRIBUTIONS: Haptic indentation probe was tested for the first time on fresh human gynaecological organs to locate cancer in a clinical setting. We are the first one to evaluate the accuracy of cancer diagnosis in human gynaecological organs with a force sensing haptic indentation probe measuring tissue stiffness

Haptics-Enabled Teleoperation for Robotics-Assisted Minimally Invasive Surgery

The lack of force feedback (haptics) in robotic surgery can be considered to be a safety risk leading to accidental tissue damage and puncturing of blood vessels due to excessive forces being applied to tissue and vessels or causing inefficient control over the instruments because of insufficient applied force. This project focuses on providing a satisfactory solution for introducing haptic feedback in robotics-assisted minimally invasive surgical (RAMIS) systems. The research addresses several key issues associated with the incorporation of haptics in a master-slave (teleoperated) robotic environment for minimally invasive surgery (MIS). In this project, we designed a haptics-enabled dual-arm (two masters - two slaves) robotic MIS testbed to investigate and validate various single-arm as well as dual-arm teleoperation scenarios. The most important feature of this setup is the capability of providing haptic feedback in all 7 degrees of freedom (DOF) required for RAMIS (3 translations, 3 rotations and pinch motion of the laparoscopic tool). The setup also enables the evaluation of the effect of replacing haptic feedback by other sensory cues such as visual representation of haptic information (sensory substitution) and the hypothesis that surgical outcomes may be improved by substituting or augmenting haptic feedback by such sensory cues

Safe Haptics-enabled Patient-Robot Interaction for Robotic and Telerobotic Rehabilitation of Neuromuscular Disorders: Control Design and Analysis

Motivation: Current statistics show that the population of seniors and the incidence rate of age-related neuromuscular disorders are rapidly increasing worldwide. Improving medical care is likely to increase the survival rate but will result in even more patients in need of Assistive, Rehabilitation and Assessment (ARA) services for extended periods which will place a significant burden on the world\u27s healthcare systems. In many cases, the only alternative is limited and often delayed outpatient therapy. The situation will be worse for patients in remote areas. One potential solution is to develop technologies that provide efficient and safe means of in-hospital and in-home kinesthetic rehabilitation. In this regard, Haptics-enabled Interactive Robotic Neurorehabilitation (HIRN) systems have been developed. Existing Challenges: Although there are specific advantages with the use of HIRN technologies, there still exist several technical and control challenges, e.g., (a) absence of direct interactive physical interaction between therapists and patients; (b) questionable adaptability and flexibility considering the sensorimotor needs of patients; (c) limited accessibility in remote areas; and (d) guaranteeing patient-robot interaction safety while maximizing system transparency, especially when high control effort is needed for severely disabled patients, when the robot is to be used in a patient\u27s home or when the patient experiences involuntary movements. These challenges have provided the motivation for this research. Research Statement: In this project, a novel haptics-enabled telerobotic rehabilitation framework is designed, analyzed and implemented that can be used as a new paradigm for delivering motor therapy which gives therapists direct kinesthetic supervision over the robotic rehabilitation procedure. The system also allows for kinesthetic remote and ultimately in-home rehabilitation. To guarantee interaction safety while maximizing the performance of the system, a new framework for designing stabilizing controllers is developed initially based on small-gain theory and then completed using strong passivity theory. The proposed control framework takes into account knowledge about the variable biomechanical capabilities of the patient\u27s limb(s) in absorbing interaction forces and mechanical energy. The technique is generalized for use for classical rehabilitation robotic systems to realize patient-robot interaction safety while enhancing performance. In the next step, the proposed telerobotic system is studied as a modality of training for classical HIRN systems. The goal is to first model and then regenerate the prescribed kinesthetic supervision of an expert therapist. To broaden the population of patients who can use the technology and HIRN systems, a new control strategy is designed for patients experiencing involuntary movements. As the last step, the outcomes of the proposed theoretical and technological developments are translated to designing assistive mechatronic tools for patients with force and motion control deficits. This study shows that proper augmentation of haptic inputs can not only enhance the transparency and safety of robotic and telerobotic rehabilitation systems, but it can also assist patients with force and motion control deficiencies