Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics

A novel sensory substitution technique is presented. Kinesthetic and cutaneous force feedback are substituted by cutaneous feedback (CF) only, provided by two wearable devices able to apply forces to the index finger and the thumb, while holding a handle during a teleoperation task. The force pattern, fed back to the user while using the cutaneous devices, is similar, in terms of intensity and area of application, to the cutaneous force pattern applied to the finger pad while interacting with a haptic device providing both cutaneous and kinesthetic force feedback. The pattern generated using the cutaneous devices can be thought as a subtraction between the complete haptic feedback (HF) and the kinesthetic part of it. For this reason, we refer to this approach as sensory subtraction instead of sensory substitution. A needle insertion scenario is considered to validate the approach. The haptic device is connected to a virtual environment simulating a needle insertion task. Experiments show that the perception of inserting a needle using the cutaneous-only force feedback is nearly indistinguishable from the one felt by the user while using both cutaneous and kinesthetic feedback. As most of the sensory substitution approaches, the proposed sensory subtraction technique also has the advantage of not suffering from stability issues of teleoperation systems due, for instance, to communication delays. Moreover, experiments show that the sensory subtraction technique outperforms sensory substitution with more conventional visual feedback (VF)

Development and human performance evaluation of a ground vehicle robotic force-feedback tele-operation system

ABSTRACT DEVELOPMENT AND HUMAN PERFORMANCE EVALUATION OF A GROUND VEHICLE ROBOTIC FORCE-FEEDBACK TELE-OPERATION SYSTEM by ANKUR SARAF May 2011 Advisor: Dr. Abhilash K. Pandya. Major: Electrical Engineering. Degree: Master of Science. Modern tele-operation systems are trying to take into account haptic and audio information in addition to visual data to provide as feedback to the tele-operator.This research emphasizes on the development of hardware and software architecture to enhance the tele-operation capabilities of omni-directional inspection robot (ODIS). The system enhances the tele-operation capabilities by introducing force-feedback to the tele-operators. The conventional joystick is replaced with Novint Falcon haptic joystick which gets the feedback from the wireless accelerometer sensor module mounted on the top of ODIS. The wireless accelerometer sensor module uses XBee modules for sending the acceleration data to the server. The server in-turn is connected to the joystick which is used to direct the ODIS. The advantage of the wireless accelerometer system is it can be used not only with ODIS but with any other unmanned vehicle as well. Though this research uses ODIS robot as the platform, the ideas and concepts put forward are applicable to tele-operation of robots in general

Human-Machine Interfaces using Distributed Sensing and Stimulation Systems

As the technology moves towards more natural human-machine interfaces (e.g. bionic limbs, teleoperation, virtual reality), it is necessary to develop a sensory feedback system in order to foster embodiment and achieve better immersion in the control system. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing a wide bandwidth of information. To provide this type of feedback, it is necessary to develop a distributed sensing system that could extract a wide range of information during the interaction between the robot and the environment. In addition, a distributed feedback interface is needed to deliver such information to the user. This thesis proposes the development of a distributed sensing system (e-skin) to acquire tactile sensation, a first integration of distributed sensing system on a robotic hand, the development of a sensory feedback system that compromises the distributed sensing system and a distributed stimulation system, and finally the implementation of deep learning methods for the classification of tactile data. It\u2019s core focus addresses the development and testing of a sensory feedback system, based on the latest distributed sensing and stimulation techniques. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives, and the used methodology and contributions; as well as six studies that tackled the development of human-machine interfaces

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Novel haptic interface For viewing 3D images

In recent years there has been an explosion of devices and systems capable of displaying stereoscopic 3D images. While these systems provide an improved experience over traditional bidimensional displays they often fall short on user immersion. Usually these systems only improve depth perception by relying on the stereopsis phenomenon. We propose a system that improves the user experience and immersion by having a position dependent rendering of the scene and the ability to touch the scene. This system uses depth maps to represent the geometry of the scene. Depth maps can be easily obtained on the rendering process or can be derived from the binocular-stereo images by calculating their horizontal disparity. This geometry is then used as an input to be rendered in a 3D display, do the haptic rendering calculations and have a position depending render of the scene. The author presents two main contributions. First, since the haptic devices have a finite work space and limited resolution, we used what we call detail mapping algorithms. These algorithms compress geometry information contained in a depth map, by reducing the contrast among pixels, in such a way that it can be rendered into a limited resolution display medium without losing any detail. Second, the unique combination of a depth camera as a motion capturing system, a 3D display and haptic device to enhance user experience. While developing this system we put special attention on the cost and availability of the hardware. We decided to use only off-the-shelf, mass consumer oriented hardware so our experiments can be easily implemented and replicated. As an additional benefit the total cost of the hardware did not exceed the one thousand dollars mark making it affordable for many individuals and institutions

The Impact of Stimulation Intensity on Spatial Discrimination with Multi-Pad Finger Electrode

Multi-pad electrotactile stimulation can be used to provide tactile feedback in different applications. The electrotactile interface needs to be calibrated before each use, which entails adjusting the intensity to obtain clear sensations while allowing the subjects to differentiate between active pads. The present study investigated how the stimulation intensity affects the localization of sensations using a multi-pad electrode placed on a fingertip and proximal phalange. First, the sensation, localization, smearing and discomfort thresholds were determined in 11 subjects. Then, the same subjects performed a spatial discrimination test across a range of stimulation intensities. The results have shown that all thresholds were significantly different, while there was no difference in the threshold values between the pads and phalanges. Despite the subjective feeling of spreading of sensations, the success rates in spatial discrimination were not significantly different across the tested stimulation intensities. However, the performance was better for distal compared to proximal phalange. Presented results indicate that spatial discrimination is robust to changes in the stimulation intensity. Considering the lack of significant difference in the thresholds between the pads, these results imply that more coarse adjustment of stimulation amplitude (faster calibration) might be enough for practical applications of a multi-pad electrotactile interface.This research was funded by the TACTILITY project, which has received funding by European Union’s Horizon 2020 framework programme for research and innovation H2020-ICT 2018-2020/H2020-ICT-2018-3 under grant agreement no. 856718