Haptic feedback in teleoperation in Micro-and Nano-Worlds.

International audienceRobotic systems have been developed to handle very small objects, but their use remains complex and necessitates long-duration training. Simulators, such as molecular simulators, can provide access to large amounts of raw data, but only highly trained users can interpret the results of such systems. Haptic feedback in teleoperation, which provides force-feedback to an operator, appears to be a promising solution for interaction with such systems, as it allows intuitiveness and flexibility. However several issues arise while implementing teleoperation schemes at the micro-nanoscale, owing to complex force-fields that must be transmitted to users, and scaling differences between the haptic device and the manipulated objects. Major advances in such technology have been made in recent years. This chapter reviews the main systems in this area and highlights how some fundamental issues in teleoperation for micro- and nano-scale applications have been addressed. The chapter considers three types of teleoperation, including: (1) direct (manipulation of real objects); (2) virtual (use of simulators); and (3) augmented (combining real robotic systems and simulators). Remaining issues that must be addressed for further advances in teleoperation for micro-nanoworlds are also discussed, including: (1) comprehension of phenomena that dictate very small object (< 500 micrometers) behavior; and (2) design of intuitive 3-D manipulation systems. Design guidelines to realize an intuitive haptic feedback teleoperation system at the micro-nanoscale level are proposed

Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

Robotic manipulators for single access surgery

This thesis explores the development of cooperative robotic manipulators for enhancing surgical precision and patient outcomes in single-access surgery and, specifically, Transanal Endoscopic Microsurgery (TEM). During these procedures, surgeons manipulate a heavy set of instruments via a mechanical clamp inserted in the patient’s body through a surgical port, resulting in imprecise movements, increased patient risks, and increased operating time. Therefore, an articulated robotic manipulator with passive joints is initially introduced, featuring built-in position and force sensors in each joint and electronic joint brakes for instant lock/release capability. The articulated manipulator concept is further improved with motorised joints, evolving into an active tool holder. The joints allow the incorporation of advanced robotic capabilities such as ultra-lightweight gravity compensation and hands-on kinematic reconfiguration, which can optimise the placement of the tool holder in the operating theatre. Due to the enhanced sensing capabilities, the application of the active robotic manipulator was further explored in conjunction with advanced image guidance approaches such as endomicroscopy. Recent advances in probe-based optical imaging such as confocal endomicroscopy is making inroads in clinical uses. However, the challenging manipulation of imaging probes hinders their practical adoption. Therefore, a combination of the fully cooperative robotic manipulator with a high-speed scanning endomicroscopy instrument is presented, simplifying the incorporation of optical biopsy techniques in routine surgical workflows. Finally, another embodiment of a cooperative robotic manipulator is presented as an input interface to control a highly-articulated robotic instrument for TEM. This master-slave interface alleviates the drawbacks of traditional master-slave devices, e.g., using clutching mechanics to compensate for the mismatch between slave and master workspaces, and the lack of intuitive manipulation feedback, e.g. joint limits, to the user. To address those drawbacks a joint-space robotic manipulator is proposed emulating the kinematic structure of the flexible robotic instrument under control.Open Acces

Monitoring companion for industrial robotic processes

For system integrators, optimizing complex industrial robotic applications (e.g. robotised welding) is a difficult and time-consuming task. This procedure is rendered tedious and often very hard to achieve when the operator cannot access the robotic system once in operation, perhaps because the installation is far away or because of the operational environment. In these circumstances, as an alternative to physically visiting the installation site, the system integrator may rely on additional nearby sensors to remotely acquire the necessary process information. While it is hard to completely replace this trial and error approach, it is possible to provide a way to gather process information more effectively that can be used in several robotic installations.This thesis investigates the use of a "monitoring robot" in addition to the task robot(s) that belong to the industrial process to be optimized. The monitoring robot can be equipped with several different sensors and can be moved into close proximity of any installed task robot so that it can be used to collect information from that process during and/or after the operation without interfering. The thesis reviews related work in the industry and in the field of teleoperation to identify the most important challenges in remote monitoring and teleoperation. From the background investigation it is clear that two very important issues are: i) the nature of the teleoperator’s interface and; ii) the efficiency of the shared control between the human operator and the monitoring system. In order to investigate these two issues efficiently it was necessary to create experimental scenarios that operate independently from any application scenario, so an abstract problem domain is created. This way the monitoring system's control and interface can be evaluated in a context that presents challenges that are typical of a remote monitoring task but are not application domain specific. Therefore the validity of the proposed approach can be assessed from a generic and, therefore, more powerful and widely applicable perspective. The monitoring framework developed in this thesis is described, both in the shared control design choices based on virtual fixtures (VF) and the implementation in a 3D visualization environment. The monitoring system developed is evaluated with a usability study with user participants. The usability study aims at assessing the system's performance along with its acceptance and ease of use in a static monitoring task, accompanied by user\hyp{}filled TLX questionnaires. Since future work will apply this system in real robotic welding scenarios, this thesis finally reports some preliminary work in such an application

Electroencephalography (EEG), electromyography (EMG) and eye-tracking for astronaut training and space exploration

The ongoing push to send humans back to the Moon and to Mars is giving rise to a wide range of novel technical solutions in support of prospective astronaut expeditions. Against this backdrop, the European Space Agency (ESA) has recently launched an investigation into unobtrusive interface technologies as a potential answer to such challenges. Three particular technologies have shown promise in this regard: EEG-based brain-computer interfaces (BCI) provide a non-invasive method of utilizing recorded electrical activity of a user's brain, electromyography (EMG) enables monitoring of electrical signals generated by the user's muscle contractions, and finally, eye tracking enables, for instance, the tracking of user's gaze direction via camera recordings to convey commands. Beyond simply improving the usability of prospective technical solutions, our findings indicate that EMG, EEG, and eye-tracking could also serve to monitor and assess a variety of cognitive states, including attention, cognitive load, and mental fatigue of the user, while EMG could furthermore also be utilized to monitor the physical state of the astronaut. In this paper, we elaborate on the key strengths and challenges of these three enabling technologies, and in light of ESA's latest findings, we reflect on their applicability in the context of human space flight. Furthermore, a timeline of technological readiness is provided. In so doing, this paper feeds into the growing discourse on emerging technology and its role in paving the way for a human return to the Moon and expeditions beyond the Earth's orbit

Haptic communication for remote mobile and manipulator robot operations in hazardous environments

Nuclear decommissioning involves the use of remotely deployed mobile vehicles and manipulators controlled via teleoperation systems. Manipulators are used for tooling and sorting tasks, and mobile vehicles are used to locate a manipulator near to the area that it is to be operated upon and also to carry a camera into a remote area for monitoring and assessment purposes. Teleoperations in hazardous environments are often hampered by a lack of visual information. Direct line of sight is often only available through small, thick windows, which often become discoloured and less transparent over time. Ideal camera locations are generally not possible, which can lead to areas of the cell not being visible, or at least difficult to see. Damage to the mobile, manipulator, tool or environment can be very expensive and dangerous. Despite the advances in the recent years of autonomous systems, the nuclear industry prefers generally to ensure that there is a human in the loop. This is due to the safety critical nature of the industry. Haptic interfaces provide a means of allowing an operator to control aspects of a task that would be difficult or impossible to control with impoverished visual feedback alone. Manipulator endeffector force control and mobile vehicle collision avoidance are examples of such tasks. Haptic communication has been integrated with both a Schilling Titan II manipulator teleoperation system and Cybermotion K2A mobile vehicle teleoperation system. The manipulator research was carried out using a real manipulator whereas the mobile research was carried out in simulation. Novel haptic communication generation algorithms have been developed. Experiments have been conducted using both the mobile and the manipulator to assess the performance gains offered by haptic communication. The results of the mobile vehicle experiments show that haptic feedback offered performance improvements in systems where the operator is solely responsible for control of the vehicle. However in systems where the operator is assisted by semi autonomous behaviour that can perform obstacle avoidance, the advantages of haptic feedback were more subtle. The results from the manipulator experiments served to support the results from the mobile vehicle experiments since they also show that haptic feedback does not always improve operator performance. Instead, performance gains rely heavily on the nature of the task, other system feedback channels and operator assistance features. The tasks performed with the manipulator were peg insertion, grinding and drilling.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Design of a Haptic Interface for Medical Applications using Magneto-Rheological Fluid based Actuators

This thesis reports on the design, construction, and evaluation of a prototype two degrees-of-freedom (DOF) haptic interface, which takes advantage of Magneto-Rheological Fluid (MRF) based clutches for actuation. Haptic information provides important cues in teleoperated systems and enables the user to feel the interaction with a remote or virtual environment during teleoperation. The two main objectives in designing a haptic interface are stability and transparency. Indeed, deficiencies in these factors in haptics-enabled telerobotic systems has the introduction of haptics in medical environments where safety and reliability are prime considerations. An actuator with poor dynamics, high inertia, large size, and heavy weight can significantly undermine the stability and transparency of a teleoperated system. In this work, the potential benefits of MRF-based actuators to the field of haptics in medical applications are studied. Devices developed with such fluids are known to possess superior mechanical characteristics over conventional servo systems. These characteristics significantly contribute to improved stability and transparency of haptic devices. This idea is evaluated and verified through both theoretical and experimental points of view. The design of a small-scale MRF-based clutch, suitable for a multi-DOF haptic interface, is discussed and its performance is compared with conventional servo systems. This design is developed into four prototype clutches. In addition, a closed-loop torque control strategy is presented. The feedback signal used in this control scheme comes from the magnetic field acquired from embedded Hall sensors in the clutch. The controller uses this feedback signal to compensate for the nonlinear behavior using an estimated model, based on Artificial Neural Networks. Such a control strategy eliminates the need for torque sensors for providing feedback signals. The performance of the developed design and the effectiveness of the proposed modeling and control techniques are experimentally validated. Next, a 2-DOF haptic interface based on a distributed antagonistic configuration of MRF-based clutches is constructed for a class of medical applications. This device is incorporated in a master-slave teleoperation setup that is used for applications involving needle insertion and soft-tissue palpation. Phantom and in vitro animal tissue were used to assess the performance of the haptic interface. The results show a great potential of MRF-based actuators for integration in haptic devices for medical interventions that require reliable, safe, accurate, highly transparent, and stable force reflection