Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

Touch and deformation perception of soft manipulators with capacitive e-skins and deep learning

Tactile sensing in soft robots remains particularly challenging because of the coupling between contact and deformation information which the sensor is subject to during actuation and interaction with the environment. This often results in severe interference and makes disentangling tactile sensing and geometric deformation difficult. To address this problem, this paper proposes a soft capacitive e-skin with a sparse electrode distribution and deep learning for information decoupling. Our approach successfully separates tactile sensing from geometric deformation, enabling touch recognition on a soft pneumatic actuator subject to both internal (actuation) and external (manual handling) forces. Using a multi-layer perceptron, the proposed e-skin achieves 99.88\% accuracy in touch recognition across a range of deformations. When complemented with prior knowledge, a transformer-based architecture effectively tracks the deformation of the soft actuator. The average distance error in positional reconstruction of the manipulator is as low as 2.905$\pm$2.207 mm, even under operative conditions with different inflation states and physical contacts which lead to additional signal variations and consequently interfere with deformation tracking. These findings represent a tangible way forward in the development of e-skins that can endow soft robots with proprioception and exteroception

Design, Modeling and Control of a 3D Printed Monolithic Soft Robotic Finger with Embedded Pneumatic Sensing Chambers

IEEE This paper presents a directly 3D printed soft monolithic robotic finger with embedded soft pneumatic sensing chambers (PSC) as position and touch sensors. The monolithic finger was fabricated using a low-cost and open-source fused deposition modeling (FDM) 3D printer that employs an off-the-shelf soft and flexible commercially available thermoplastic polyurethane (TPU). A single soft hinge with an embedded PSC was optimized using finite element modeling (FEM) and a hyperelastic material model to obtain a linear relationship between the internal change in the volume of its PSC and the corresponding input mechanical modality, to minimize its bending stiffness and to maximize its internal volume. The soft hinges with embedded PSCs have several advantages, such as fast response to very small changes in their internal volume (~0.0026ml/°), linearity, negligible hysteresis, repeatability, reliability, long lifetime and low power consumption. Also, the flexion of the soft robotic finger was predicted using a geometric model for use in real-time control. The real-time position and pressure/force control of the soft robotic finger were achieved using feedback signals from the soft hinges and the touch PSC embedded in the tip of the finger. This study contributes to the development of seamlessly embedding optimized sensing elements in the monolithic topology of a soft robotic system and controlling the robotic system using the feedback data provided by the sensing elements to validate their performance

Haptics in Robot-Assisted Surgery: Challenges and Benefits

Robotic surgery is transforming the current surgical practice, not only by improving the conventional surgical methods but also by introducing innovative robot-enhanced approaches that broaden the capabilities of clinicians. Being mainly of man-machine collaborative type, surgical robots are seen as media that transfer pre- and intra-operative information to the operator and reproduce his/her motion, with appropriate filtering, scaling, or limitation, to physically interact with the patient. The field, however, is far from maturity and, more critically, is still a subject of controversy in medical communities. Limited or absent haptic feedback is reputed to be among reasons that impede further spread of surgical robots. In this paper objectives and challenges of deploying haptic technologies in surgical robotics is discussed and a systematic review is performed on works that have studied the effects of providing haptic information to the users in major branches of robotic surgery. It has been tried to encompass both classical works and the state of the art approaches, aiming at delivering a comprehensive and balanced survey both for researchers starting their work in this field and for the experts

Soft Robot-Assisted Minimally Invasive Surgery and Interventions: Advances and Outlook

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry's appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery

3D printed pneumatic soft actuators and sensors: their modeling, performance quantification, control and applications in soft robotic systems

Continued technological progress in robotic systems has led to more applications where robots and humans operate in close proximity and even physical contact in some cases. Soft robots, which are primarily made of highly compliant and deformable materials, provide inherently safe features, unlike conventional robots that are made of stiff and rigid components. These robots are ideal for interacting safely with humans and operating in highly dynamic environments. Soft robotics is a rapidly developing field exploiting biomimetic design principles, novel sensor and actuation concepts, and advanced manufacturing techniques. This work presents novel soft pneumatic actuators and sensors that are directly 3D printed in one manufacturing step without requiring postprocessing and support materials using low-cost and open-source fused deposition modeling (FDM) 3D printers that employ an off-the-shelf commercially available soft thermoplastic poly(urethane) (TPU). The performance of the soft actuators and sensors developed is optimized and predicted using finite element modeling (FEM) analytical models in some cases. A hyperelastic material model is developed for the TPU based on its experimental stress-strain data for use in FEM analysis. The novel soft vacuum bending (SOVA) and linear (LSOVA) actuators reported can be used in diverse robotic applications including locomotion robots, adaptive grippers, parallel manipulators, artificial muscles, modular robots, prosthetic hands, and prosthetic fingers. Also, the novel soft pneumatic sensing chambers (SPSC) developed can be used in diverse interactive human-machine interfaces including wearable gloves for virtual reality applications and controllers for soft adaptive grippers, soft push buttons for science, technology, engineering, and mathematics (STEM) education platforms, haptic feedback devices for rehabilitation, game controllers and throttle controllers for gaming and bending sensors for soft prosthetic hands. These SPSCs are directly 3D printed and embedded in a monolithic soft robotic finger as position and touch sensors for real-time position and force control. One of the aims of soft robotics is to design and fabricate robotic systems with a monolithic topology embedded with its actuators and sensors such that they can safely interact with their immediate physical environment. The results and conclusions of this thesis have significantly contributed to the realization of this aim

A Review of Cooperative Actuator and Sensor Systems Based on Dielectric Elastomer Transducers

This paper presents an overview of cooperative actuator and sensor systems based on dielectric elastomer (DE) transducers. A DE consists of a flexible capacitor made of a thin layer of soft dielectric material (e.g., acrylic, silicone) surrounded with a compliant electrode, which is able to work as an actuator or as a sensor. Features such as large deformation, high compliance, flexibility, energy efficiency, lightweight, self-sensing, and low cost make DE technology particularly attractive for the realization of mechatronic systems that are capable of performance not achievable with alternative technologies. If several DEs are arranged in an array-like configuration, new concepts of cooperative actuator/sensor systems can be enabled, in which novel applications and features are made possible by the synergistic operations among nearby elements. The goal of this paper is to review recent advances in the area of cooperative DE systems technology. After summarizing the basic operating principle of DE transducers, several applications of cooperative DE actuators and sensors from the recent literature are discussed, ranging from haptic interfaces and bio-inspired robots to micro-scale devices and tactile sensors. Finally, challenges and perspectives for the future development of cooperative DE systems are discussed

Intelligent Haptic Perception for Physical Robot Interaction

Doctorado en Ingeniería mecatrónica. Fecha de entrega de la Tesis doctoral: 8 de enero de 2020. Fecha de lectura de Tesis doctoral: 30 de marzo 2020.The dream of having robots living among us is coming true thanks to the recent advances in Artificial Intelligence (AI). The gap that still exists between that dream and reality will be filled by scientific research, but manifold challenges are yet to be addressed. Handling the complexity and uncertainty of real-world scenarios is still the major challenge in robotics nowadays. In this respect, novel AI methods are giving the robots the capability to learn from experience and therefore to cope with real-life situations. Moreover, we live in a physical world in which physical interactions are both vital and natural. Thus, those robots that are being developed to live among humans must perform tasks that require physical interactions. Haptic perception, conceived as the idea of feeling and processing tactile and kinesthetic sensations, is essential for making this physical interaction possible. This research is inspired by the dream of having robots among us, and therefore, addresses the challenge of developing robots with haptic perception capabilities that can operate in real-world scenarios. This PhD thesis tackles the problems related to physical robot interaction by employing machine learning techniques. Three AI solutions are proposed for different physical robot interaction challenges: i) Grasping and manipulation of humans’ limbs; ii) Tactile object recognition; iii) Control of Variable-Stiffness-Link (VSL) manipulators. The ideas behind this research work have potential robotic applications such as search and rescue, healthcare or rehabilitation. This dissertation consists of a compendium of publications comprising as the main body a compilation of previously published scientific articles. The baseline of this research is composed of a total of five papers published in prestigious peer-reviewed scientific journals and international robotics conferences

Soft pneumatic devices for blood circulation improvement

The research activity I am presenting in this thesis lies within the framework of a cooperation between the University of Cagliari (Applied Mechanics and Robotics lab, headed by professor Andrea Manuello Bertetto, and the research group of physicians referencing to professor Alberto Concu at the Laboratory of Sports Physiology, Department of Medical Sciences), and the Polytechnic of Turin (professor Carlo Ferraresi and his equipe at the Group of Automation and Robotics, Department of Mechanical and Aerospace Engineering) This research was also funded by the Italian Ministry of Research (MIUR – PRIN 2009). My activity has been mainly carried on at the Department of Mechanics, Robotics lab under the supervision of prof. Manuello; I have also spent one year at the Control Lab of the School of Electrical Engineering at Aalto University (Helsinki, Finland). The tests on the patients were taken at the Laboratory of Sports Physiology, Cagliari. I will be describing the design, development and testing of some soft pneumatic flexible devices meant to apply an intermittent massage and to restore blood circulation in lower limbs in order to improve cardiac output and wellness in general. The choice of the actuators, as well as the pneumatic circuits and air distribution system and PLC control patterns will be outlined. The trial run of the devices have been field--‐tested as soon a prototype was ready, so as to tune its features step--‐by--‐ step. I am also giving a characterization of a commercial thin force sensor after briefly reviewing some other type of thin pressure transducer. It has been used to gauge the contact pressure between the actuator and the subject’s skin in order to correlate the level of discomfort to the supply pressure, and to feed this value back to regulate the supply air flow. In order for the massage to be still effective without causing pain or distress or any cutoff to the blood flow, some control objective have been set, consisting in the regulation of the contact force so that it comes to the constant set point smoothly and its value holds constant until unloading occurs. The targets of such mechatronic devices range from paraplegic patients lacking of muscle tone because of their spinal cord damage, to elite endurance athletes needing a circulation booster when resting from practicing after serious injuries leading to bed rest. Encouraging results have been attained for both these two categories, based on the monitored hemodynamic variables

Development and applicability of a soft and flexible robotic arm in digestive surgery

Introduction The oncologic adequacy of laparoscopy in digestive surgery is still controversial, especially in some technically demanding operations like Total Mesorectal Excision (TME). Even if standard robotic platforms, i.e. the da Vinci Surgical System, can improve dexterity and manouvrability of surgical instruments, there is no evidence supporting its use in digestive and rectal cancer surgery. The only multi-centre prospective RCT (ROLARR trial) suggests that robotic TME has no advantages compared to laparoscopic TME in terms of clinical and oncologic outcomes. A possible explanation of this lack of real advantages is that the articulation is possible only on the tip of the instrument. The opportunity to have a robotic platform with modular flexibility on the whole length of the arm could overcome technical limitations, improving results and allowing standardization and diffusion of the procedures. Methods The 7FP STIFF FLOP project was financed by the European Commission in order to develop a STIFFness controllable Flexible and Learn-able manipulator for surgical operations. Engineers were inspired by the tentacles of an octopus. A prototype was realized, consisting of multiple soft, pneumatically actuated threechamber segments. Additional chambers are integrated within the segments to allow their stiffening, employing an approach based on the concept of granular jamming. The STIFF-FLOP segments are actuated using pressure regulators and the stiffening chambers are interfaced via valves, applying a vacuum to the granules in the chambers. Sensors are embedded in the STIFF-FLOP modules to measure interaction forces (between the robot and its environment) and the robot’s configuration. A newly developed user interface, based on a Delta robot design, is used to move and position the tip of the STIFF-FLOP arm inside the abdomen. Signals obtained from sensors are fed back to the user interface console providing the operator with force feedback. The entire soft robot is equipped with a 4 mm in diameter centre-free lumen, which allows the passage of the electrical wires needed for the laparoscopic miniaturized optic system positioned at the tip of the robot. Phantom test The prototype was tested in order to assess learnability and satisfaction of the operators. The test was designed as a spatial motion task, consisting of movements between predefined target points clockwise and counter clockwise in a 3D phantom of the abdominal cavity. The participants were asked to conclude the task for the first time with the STIFF-FLOP prototype (SF1), then to repeat the task using conventional laparoscopic instrumentation (LAP) and finally to perform the task once more with the STIFF-FLOP arm (SF2). Surface EMG signals from the forearm muscles were recorded during the test. Results SF1 took a longer time than the other tasks, i.e. 36.4% more than LAP (p=0.0071). However, from SF1 to SF2 there was a 32.1% time reduction (p=0.0232). EMG amplitude analysis showed a higher overall average muscle activity during LAP. Moving from LAP to SF2 there was a 25.9% reduction in average muscle activity (p=0.0128). Cadaver test. The main objective of the test was to validate the compatibility of the system with human anatomy for laparoscopic TME and to determine whether the soft robot could represent a potential improvement compared to standard rigid laparoscopic instrumentation. The study was performed on two cadavers prepared according to the method described by Thiel. Results The use of the STIFF-FLOP camera allowed the surgeon to clearly visualize the inferior mesenteric vessels and the autonomic nerves that were subsequently spared from injury. The ability to smoothly follow the sacral curve due to the flexibility of the manipulator allowed the surgeons to perform a very precise dissection of the posterior part of the mesorectum. The same procedure was performed on both human cadavers, demonstrating the ease of use of the system. Completion times of the procedure were 165 and 145 min, respectively. No intraoperative complications were recorded. No technical failures were registered. Conclusion The STIFF FLOP flexible robotic arm is an intuitive technology that can be easily learned. The prolonged use of the STIFF FLOP manipulator is more comfortable than standard laparoscopic instrumentation and can be used for a long time without exhaustion. The system is compatible with human anatomy and allows to perform a standard surgical abdominal operation. The STIFF FLOP arm seems to improve visualization of the operatory field especially in narrow spaces like the pelvis