Haptics in Robot-Assisted Surgery: Challenges and Benefits

Robotic surgery is transforming the current surgical practice, not only by improving the conventional surgical methods but also by introducing innovative robot-enhanced approaches that broaden the capabilities of clinicians. Being mainly of man-machine collaborative type, surgical robots are seen as media that transfer pre- and intra-operative information to the operator and reproduce his/her motion, with appropriate filtering, scaling, or limitation, to physically interact with the patient. The field, however, is far from maturity and, more critically, is still a subject of controversy in medical communities. Limited or absent haptic feedback is reputed to be among reasons that impede further spread of surgical robots. In this paper objectives and challenges of deploying haptic technologies in surgical robotics is discussed and a systematic review is performed on works that have studied the effects of providing haptic information to the users in major branches of robotic surgery. It has been tried to encompass both classical works and the state of the art approaches, aiming at delivering a comprehensive and balanced survey both for researchers starting their work in this field and for the experts

SMART IMAGE-GUIDED NEEDLE INSERTION FOR TISSUE BIOPSY

M.S

The Effect of Visual and Haptic Feedback on Manual and Teleoperated Needle Insertion

In this paper, we present a study that evaluates the effect of visual and haptic feedbacks and their relevance to human performance in a needle insertion task. A virtual needle insertion simulator with a four-layer tissue model (skin, fat, muscle, and bone) and haptic feedback is used for perceptual experiments. The task was to detect the puncture of a particular layer using haptic and visual cues provided by the simulation. The results show that the addition of force feedback reduces error in detection of transitions between tissue layers by at least 55%. The addition of real-time visual feedback (image overlay) improved user performance by at least 87% in scenarios without force feedback. Presentation of both force and visual feedback improved performance by at least 43% in comparison to scenarios without feedback. These results suggest that real-time image overlay provides greater improvement in performance than force feedback

Multimodal feedback cues on manual lifting in virtual environments

Improper manipulation of real-world objects increases the risk of developing work- related back injuries. In an effort to reduce such a risk and encourage appropriate lifting and moving methods, a Virtual Environment (VE) was employed. Virtual simulations can be used for ergonomic analysis. In this work, the VEs made use of multiple feedback techniques to allow a person to estimate the forces acting on their lower back. A person's head and hand movements were tracked in real-time whilst manipulating an object. A NIOSH lifting equation was used to calculate and determine the Lifting Index whereby the results were conveyed in real time. Visual display feedback techniques were designed and the effect of cues to enhance user performance was experimentally evaluated. The feedback cues provide the user with information about the forces acting on their lower back as they perform manual lifting tasks in VEs. Four different methods were compared and contrasted: No Feedback, Text, Colour and Combined Colour and Text. This work also investigated various types of auditory feedback technique to support object manipulation in VEs. Auditory feedback has been demonstrated to convey information in computer applications effectively, but little work has been reported on the efficacy of such techniques, particularly for ergonomic design. Four different methods were compared and contrasted: No Feedback, White-noise, Pitch and Tempo. A combined Audio-Visual (AV) technique was also examined by mixing both senses. The effect of Tactile Augmentation was also examined. Three different weights (real) were used and the results obtained by experiment were compared with the experiment using virtual weights in order to evaluate whether or not the presence of a real weighted object enhanced people's sense of realism. The goals of this study were to explore various senses of feedback technique (visual, auditory and tactile), compare the performance characteristics of each technique and understand their relative advantages and drawbacks.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Determining the Contribution of Visual and Haptic Cues during Compliance Discrimination in the Context of Minimally Invasive Surgery

While minimally invasive surgery is replacing open surgery in an increasing number of surgical procedures, it still poses risks such as unintended tissue damage due to reduced visual and haptic feedback. Surgeons assess tissue health by analysing mechanical properties such as compliance. The literature shows that while both types of feedback contribute to the final percept, visual information is dominant during compliance discrimination tasks. The magnitude of that contribution, however, was never quantitatively determined. To determine the effect of the type of visual feedback on compliance discrimination, a psychophysical experiment was set up using different combinations of direct and indirect visual and haptic cues. Results reiterated the significance of visual information and suggested a visio-haptic cross-modal integration. Consequently, to determine which cues contributed most to visual feedback, the impact of force and position on the ability to discriminate compliance using visual information only was assessed. Results showed that isolating force and position cues during indentation enhanced performance. Furthermore, under force and position constraints, visual information was shown to be sufficient to determine the compliance of deformable objects. A pseudo-haptic feedback system was developed to quantitatively determine the contribution of visual feedback during compliance discrimination. A psychophysical experiment showed that the system realistically simulated viscoelastic behaviour of compliant objects. Through a magnitude estimation experiment, the pseudo-haptic system was shown to be successful at generating haptic sensations of compliance during stimuli indentation only by modifying the visual feedback presented to participants. This can be implemented in research and educational facilities where advanced force feedback devices are inaccessible. Moreover, it can be incorporated into virtual reality simulators to enhance force ranges. Future work will assess the value of visual cue augmentation in more complicated surgical tasks

Multimodal feedback cues on manual lifting in virtual environments

Improper manipulation of real-world objects increases the risk of developing work- related back injuries. In an effort to reduce such a risk and encourage appropriate lifting and moving methods, a Virtual Environment (VE) was employed. Virtual simulations can be used for ergonomic analysis. In this work, the VEs made use of multiple feedback techniques to allow a person to estimate the forces acting on their lower back. A person's head and hand movements were tracked in real-time whilst manipulating an object. A NIOSH lifting equation was used to calculate and determine the Lifting Index whereby the results were conveyed in real time. Visual display feedback techniques were designed and the effect of cues to enhance user performance was experimentally evaluated. The feedback cues provide the user with information about the forces acting on their lower back as they perform manual lifting tasks in VEs. Four different methods were compared and contrasted: No Feedback, Text, Colour and Combined Colour and Text. This work also investigated various types of auditory feedback technique to support object manipulation in VEs. Auditory feedback has been demonstrated to convey information in computer applications effectively, but little work has been reported on the efficacy of such techniques, particularly for ergonomic design. Four different methods were compared and contrasted: No Feedback, White-noise, Pitch and Tempo. A combined Audio-Visual (AV) technique was also examined by mixing both senses. The effect of Tactile Augmentation was also examined. Three different weights (real) were used and the results obtained by experiment were compared with the experiment using virtual weights in order to evaluate whether or not the presence of a real weighted object enhanced people's sense of realism. The goals of this study were to explore various senses of feedback technique (visual, auditory and tactile), compare the performance characteristics of each technique and understand their relative advantages and drawbacks

Synthèse et commande de dispositifs haptiques pour la communication à distance : application à une interface robotique anthropomorphe pour la poignée de main

Les systèmes de communication à distance entre les individus ont évolué significativement au cours des dernières années, de concert avec les innovations technologiques caractérisant notre société. Afin de réaliser une communication réaliste et intuitive, le système doit être capable de stimuler les sens qui sont habituellement impliqués dans l’interaction entre deux personnes, tels que l’ouïe, la vision et le toucher. Le téléphone a représenté une innovation importante dans les communications en permettant enfin de pouvoir parler avec son interlocuteur directement, sans devoir employer un signal codé comme le code Morse. Cette communication a été améliorée en introduisant les appels vidéo, lesquels permettent non seulement d’entendre l’interlocuteur mais aussi de le voir. Plusieurs recherches ont cependant démontré que le sens du toucher joue également un rôle très important dans les interactions entre individus. Une technologie relativement récente, connue comme technologie haptique, aborde le problème de la transmission du sens du toucher à distance, dans le but de mettre en oeuvre une communication complète et encore plus réaliste. Cette technologie a également d’autres applications tout aussi importantes. À titre d’exemple, l’haptique est utilisée dans le domaine de la réadaptation et de l’apprentissage guidé de personnes ayant des déficiences motrices. Cette thèse porte sur le développement de la technologie haptique pour la communication à distance entre deux individus. L’objectif final est la réalisation d’un système permettant aux deux utilisateurs de se serrer la main à distance. Afin d’atteindre cet objectif, nous devons aborder deux problèmes différents, tels que la conception d’une interface capable de reproduire le mouvement désiré et l’implémentation d’une loi de commande garantissant le comportement correct de cette interface. Toujours dans le cadre de l’interaction à distance par le biais d’un dispositif haptique, une interface pour l’apprentissage de l’écriture manuelle est également présentée. Cette application permet de démontrer, entre autres, l’importance d’un signal haptique dans l’interaction humain-humain et son influence sur les utilisateurs.Remote communication systems have significantly improved in the course of the recent years, in concert with technological innovations of our society. In order to realize a realistic and intuitive communication, the system must activate the part of the sensory system that is normally stimulated in an interaction between two people, i.e., the auditory system, the visual system and the haptic perception system, which concerns touch. The telephone represented an innovating communication system. It allowed to directly talk to the interlocutor without any need for a coded signal such as the Morse code. Remote communications have been further improved with the introduction of the video calls, which allow people not only to talk to but also to see each other. Several researches proved that the sense of touch plays a crucial role in social interactions. Haptic technology, which is relatively recent, approaches the problems related to the perception and the transmission of touch. One objective is to render remote communications even more complete and realistic. Haptic technology is also used in other important applications such as, for instance, rehabilitation and guided learning process of people with movement impairments. This thesis concerns the development of the haptic technology devoted to the implementation of remote communication systems. The final objective is to realize a teleoperation system which allows two users to remotely shake hands. In order to achieve this objective, two main issues must be faced : the design of a haptic interface capable of reproducing the required movement and the implementation of a control law which guarantees the proper response of such an interface. In the framework of a remote interaction via a haptic device, an interface for the training and assessment of handwriting capabilities is also presented. The latter application demonstrates the importance of haptic signals in a remote human-human interaction and its influence on the users