Pseudo-Haptics for Rigid Tool/Soft Object Interaction Feedback in Virtual Environments

This paper proposes a novel pseudo-haptics soft object stiffness simulation technique which is a marked improvement to currently used simulation methods and an effective low-cost alternative to expensive 3-DOF haptic devices. Soft object stiffness simulation is achieved by maneuvering an indenter avatar over the surface of a virtual soft object by means of an input device, such as a mouse, a joystick, or a touch-sensitive tablet. The alterations to the indenter avatar behavior produced by the proposed technique create for the user the illusion of interaction with a hard inclusion embedded in the soft object. The proposed pseudo-haptics technique is validated with a series of experiments conducted by employing three types of 2-DOF force-sensitive haptic surfaces, including a touchpad, a tablet with an S-pen input, and a tablet with a bare finger input. It is found that both the sensitivity and the positive predictive value of hard inclusion detection can be significantly improved by 33.3% and 13.9% respectively by employing tablet computers. Using tablet computers could produce results comparable to direct hand touch in detecting hard inclusions in a soft object. The experimental results presented here confirm the potential of the proposed technique for conveying haptic information in rigid tool / soft object interaction in virtual environments

Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

Using Visual Cues to Enhance Haptic Feedback for Palpation on Virtual Model of Soft Tissue

This paper explores methods that make use of visual cues aimed at generating actual haptic sensation to the user, namely pseudo-haptics. We propose a new pseudo-haptic feedback based method capable of conveying 3D haptic information and combining visual haptics with force feedback to enhance the user’s haptic experience. We focused on an application related to tumor identification during palpation and evaluated the proposed method in an experimental study where users interacted with a haptic device and graphical interface while exploring a virtual model of soft tissue, which represented stiffness distribution of a silicone phantom tissue with embedded hard inclusions. The performance of hard inclusion detection using force feedback only, pseudo-haptic feedback only, and the combination of the two feedbacks were compared with the direct hand touch. The combination method and direct hand touch had no significant difference in the detection results. Compared with the force feedback alone, our method increased the sensitivity by 5%, the positive predictive value by 4%, and decreased detection time by 48.7%. The proposed methodology has great potential for robot-assisted minimally invasive surgery and in all applications where remote haptic feedback is needed

Validation of the Haptic Cow: A simulator for training veterinary students

A virtual reality simulator, the Haptic Cow, has been developed using touch feedback technology for training veterinary students to perform bovine rectal palpation of the reproductive tract. The simulator was designed to supplement existing training and address some of the difficulties associated with teaching palpation-based skills. Students need to achieve a certain level of proficiency by graduation but this has become increasingly difficult because of problems with current training methods and a reduction in the number of opportunities to practice. A simulator- based teaching tool was developed as a potential solution. The first step involved designing a simulator on the basis of requirements established through consultation with both veterinary surgeons, as teachers, and students, as learners. Research was then undertaken to validate the simulator by following a set of established criteria described for the evaluation of new technologies used in medical education. The virtual models were assessed by experts as realistic enough representations of the same structures in the cow. An experiment to assess the effect of simulator training compared the performance of one group of students, whose training was supplemented with a simulator session, with another group of traditionally trained students. The subsequent performance for finding and identifying the uterus when examining cows for the first time, was significantly better for the simulator trained group, indicating that skills learned in the simulator environment transferred to the real task. A project was also undertaken to integrate the simulator into a curriculum, with training included as part of the farm animal course at the University of Glasgow Veterinary School. The training was well received by students, useful feedback was gathered and the simulator continues to be used as part of the course. Further developments were undertaken with the aim of creating a more versatile teaching tool and addressing some of the questions and issues raised. An automated version of the Haptic Cow was designed for students to use on their own, with computer guidance replacing the instructor's role. An evaluation found that the new version of the teaching tool was both usable and an effective way of equipping students with the skills required to find and identify the uterus. The potential to use haptic technology to investigate various aspects of performance was also explored in relation to the question of hand choice for certain palpation-based skills: differentiating between objects on the basis of softness and size. Ongoing research and development options are discussed, with the aim of building on the current work by expanding the role of haptic technology in veterinary education in the future

Development and Validation of a Hybrid Virtual/Physical Nuss Procedure Surgical Trainer

With continuous advancements and adoption of minimally invasive surgery, proficiency with nontrivial surgical skills involved is becoming a greater concern. Consequently, the use of surgical simulation has been increasingly embraced by many for training and skill transfer purposes. Some systems utilize haptic feedback within a high-fidelity anatomically-correct virtual environment whereas others use manikins, synthetic components, or box trainers to mimic primary components of a corresponding procedure. Surgical simulation development for some minimally invasive procedures is still, however, suboptimal or otherwise embryonic. This is true for the Nuss procedure, which is a minimally invasive surgery for correcting pectus excavatum (PE) – a congenital chest wall deformity. This work aims to address this gap by exploring the challenges of developing both a purely virtual and a purely physical simulation platform of the Nuss procedure and their implications in a training context. This work then describes the development of a hybrid mixed-reality system that integrates virtual and physical constituents as well as an augmentation of the haptic interface, to carry out a reproduction of the primary steps of the Nuss procedure and satisfy clinically relevant prerequisites for its training platform. Furthermore, this work carries out a user study to investigate the system’s face, content, and construct validity to establish its faithfulness as a training platform

The benefits of haptic feedback in robot assisted surgery and their moderators: a metaanalysis

Robot assisted surgery (RAS) provides medical practitioners with valuable tools, decreasing strain during surgery and leading to better patient outcomes. While the loss of haptic sensation is a commonly cited disadvantage of RAS, new systems aim to address this problem by providing artificial haptic feedback. N = 56 papers that compared robotic surgery systems with and without haptic feedback were analyzed to quantify the performance benefits of restoring the haptic modality. Additionally, this study identifies factors moderating the effect of restoring haptic sensation. Overall results showed haptic feedback was effective in reducing average forces (Hedges' g = 0.83) and peak forces (Hedges' g = 0.69) applied during surgery, as well as reducing the completion time (Hedges' g = 0.83). Haptic feedback has also been found to lead to higher accuracy (Hedges' g = 1.50) and success rates (Hedges' g = 0.80) during surgical tasks. Effect sizes on several measures varied between tasks, the type of provided feedback, and the subjects' levels of surgical expertise, with higher levels of expertise generally associated with smaller effect sizes. No significant differences were found between virtual fixtures and rendering contact forces. Implications for future research are discussed

Interventional radiology virtual simulator for liver biopsy

Purpose Training in Interventional Radiology currently uses the apprenticeship model, where clinical and technical skills of invasive procedures are learnt during practice in patients. This apprenticeship training method is increasingly limited by regulatory restrictions on working hours, concerns over patient risk through trainees’ inexperience and the variable exposure to case mix and emergencies during training. To address this, we have developed a computer-based simulation of visceral needle puncture procedures. Methods A real-time framework has been built that includes: segmentation, physically based modelling, haptics rendering, pseudo-ultrasound generation and the concept of a physical mannequin. It is the result of a close collaboration between different universities, involving computer scientists, clinicians, clinical engineers and occupational psychologists. Results The technical implementation of the framework is a robust and real-time simulation environment combining a physical platform and an immersive computerized virtual environment. The face, content and construct validation have been previously assessed, showing the reliability and effectiveness of this framework, as well as its potential for teaching visceral needle puncture. Conclusion A simulator for ultrasound-guided liver biopsy has been developed. It includes functionalities and metrics extracted from cognitive task analysis. This framework can be useful during training, particularly given the known difficulties in gaining significant practice of core skills in patients

Development of a Cardiac Auscultation Virtual Reality App

La auscultación cardiaca es un procedimiento que permite diagnosticar la condición del corazón de un paciente haciendo uso de un estetoscopio. Una examinaciOn apropiada con el estetoscopio puede resultar en un adecuado diagnóstico y tratamiento. Habilidades en auscultación cardiaca están en decaimiento entre practicantes y profesionales de áreas relacionadas al cuidado de la salud debido al extenso tiempo requerido de entrenamiento. Herramientas actuales de entrenamiento incluyen maniquíes de simulación y equipamiento de diagnóstico que está tomando un papel con mayor prominencia en la práctica de auscultación cardiaca que el entrenamiento con el estetoscopio. De acuerdo con la Organización Mundial de la Salud, en 2015 los dos causantes más grandes de muerte fueron ataques al corazón y enfermedades cerebrovasculares. La disponibilidad y bajo costos del estetoscopio lo hace una herramienta rentable en el entrenamiento en auscultación cardiaca. Actualmente, dispositivos de realidad virtual disponibles para el consumidor común están permitiendo el desarrollo de experiencias virtuales asequibles que tienen el potencial para complementar otras formas de entrenamiento. En este trabajo se presenta una aplicación móvil que combina elementos multimedia tales como sonidos de ritmos cardiacos e imágenes indicando áreas donde la exploración cardiaca debería ser llevada a cabo. Modelados 3D con animaciones realistas de pacientes con diferentes características físicas para simulación y texto explicativo para el diagnóstico fueron usados para proveer inmersión y realismo. El sistema está basado en el procedimiento de entrenamiento en auscultación cardiaca, haciendo uso de elementos de juego para enganchar y motivar los usuarios en el proceso de aprendizaje.Cardiac auscultation is a procedure that allows diagnosing a patient’s heart condition making use of the stethoscope. Proper examination skills with the stethoscope may result in proper diagnostics and treatment. Auscultation skills are in decay among health care trainees and even professionals, due to the required extensive training. Current training tools include simulation manikins and diagnostics equipment that is taking a more prominent role in auscultation practice than the training with the stethoscope. According to the World Health Organization, in 2015 the top two causes of death were ischaemic heart disease and stroke. The availability and low cost of the stethoscope makes it a cost-effective tool in cardiac auscultation skills. Currently, virtual reality consumer-grade devices are allowing the development of affordable virtual experiences that have the potential to complement other forms of training. In this work a mobile app combines multimedia elements such as heart rhythm sounds and images indicating areas where cardiac exploration should be performed is presented. 3D models with realistic animations of patients with different physical characteristics for the simulation and explanatory text for the diagnosis were used to provide immersion and realism. The system is based on the cardiac auscultation training procedure, making use of game elements in order to engage and motivate users in the learning process.Pregrad

Investigating the Feasibility of Using Focussed Airborne Ultrasound as Tactile Feedback in Medical Simulators

Novice medical practitioners commonly practice on live patients in real medical procedures. However, due to the inexperience of the practitioner, mistakes are likely which exposes the patient to undue risk. To improve the training of novices, medical simulators create a virtual patient providing a safe environment for the user to practice within. An important clinical skill is palpation, a physical examination technique. The practitioners use their hands to feel the body of the patient to make diagnosis. A virtual patient has a visual representation but as it is virtual, the patient is not physically present. Haptics technology provide additional benefits to the training session by stimulating the physical sense of touch. A novel technique has recently emerged for stimulating tactile sensation called acoustic radiation pressure from focussed airborne ultrasound. Acoustic radiation creates a focal point of concentrated acoustic pressure in a three-dimensional field producing a force in mid-air. Airborne ultrasound has several advantages over conventional technologies. It was also initially theorised that using airborne ultrasound to simulate palpation compared to a previous system called PalpSim which consists of a rubber tube filled with water permanently embedded in a block of silicone, will offer better controllability over the displayed sensation to simulate various tactile sensations. The thesis has investigated the feasibility of using focussed airborne ultrasound as tactile feedback in medical simulators. A tactile device called UltraSendo was completely custom built to simulate an arterial pulse and a thrill sensation. UltraSendo was integrated with an augmented reality simulator displaying a virtual patient for user interaction. The simulator was brought to Ysbyty Glan Clwyd hospital for user feedback. A wide range of user responses were gathered. The majority of responses felt the arterial pulse was not sufficiently realistic whilst there were higher ratings for the thrill sensation which is acceptably realistic. Positive feedback suggests that airborne ultrasound can indeed provide tactile feedback in a medical context and is better at simulating a thrill sensation compared to a pulse sensation