Increasing Transparency and Presence of Teleoperation Systems Through Human-Centered Design

Teleoperation allows a human to control a robot to perform dexterous tasks in remote, dangerous, or unreachable environments. A perfect teleoperation system would enable the operator to complete such tasks at least as easily as if he or she was to complete them by hand. This ideal teleoperator must be perceptually transparent, meaning that the interface appears to be nearly nonexistent to the operator, allowing him or her to focus solely on the task environment, rather than on the teleoperation system itself. Furthermore, the ideal teleoperation system must give the operator a high sense of presence, meaning that the operator feels as though he or she is physically immersed in the remote task environment. This dissertation seeks to improve the transparency and presence of robot-arm-based teleoperation systems through a human-centered design approach, specifically by leveraging scientific knowledge about the human motor and sensory systems. First, this dissertation aims to improve the forward (efferent) teleoperation control channel, which carries information from the human operator to the robot. The traditional method of calculating the desired position of the robot\u27s hand simply scales the measured position of the human\u27s hand. This commonly used motion mapping erroneously assumes that the human\u27s produced motion identically matches his or her intended movement. Given that humans make systematic directional errors when moving the hand under conditions similar to those imposed by teleoperation, I propose a new paradigm of data-driven human-robot motion mappings for teleoperation. The mappings are determined by having the human operator mimic the target robot as it autonomously moves its arm through a variety of trajectories in the horizontal plane. Three data-driven motion mapping models are described and evaluated for their ability to correct for the systematic motion errors made in the mimicking task. Individually-fit and population-fit versions of the most promising motion mapping model are then tested in a teleoperation system that allows the operator to control a virtual robot. Results of a user study involving nine subjects indicate that the newly developed motion mapping model significantly increases the transparency of the teleoperation system. Second, this dissertation seeks to improve the feedback (afferent) teleoperation control channel, which carries information from the robot to the human operator. We aim to improve a teleoperation system a teleoperation system by providing the operator with multiple novel modalities of haptic (touch-based) feedback. We describe the design and control of a wearable haptic device that provides kinesthetic grip-force feedback through a geared DC motor and tactile fingertip-contact-and-pressure and high-frequency acceleration feedback through a pair of voice-coil actuators mounted at the tips of the thumb and index finger. Each included haptic feedback modality is known to be fundamental to direct task completion and can be implemented without great cost or complexity. A user study involving thirty subjects investigated how these three modalities of haptic feedback affect an operator\u27s ability to control a real remote robot in a teleoperated pick-and-place task. This study\u27s results strongly support the utility of grip-force and high-frequency acceleration feedback in teleoperation systems and show more mixed effects of fingertip-contact-and-pressure feedback

Wearable haptic systems for the fingertip and the hand: taxonomy, review and perspectives

In the last decade, we have witnessed a drastic change in the form factor of audio and vision technologies, from heavy and grounded machines to lightweight devices that naturally fit our bodies. However, only recently, haptic systems have started to be designed with wearability in mind. The wearability of haptic systems enables novel forms of communication, cooperation, and integration between humans and machines. Wearable haptic interfaces are capable of communicating with the human wearers during their interaction with the environment they share, in a natural and yet private way. This paper presents a taxonomy and review of wearable haptic systems for the fingertip and the hand, focusing on those systems directly addressing wearability challenges. The paper also discusses the main technological and design challenges for the development of wearable haptic interfaces, and it reports on the future perspectives of the field. Finally, the paper includes two tables summarizing the characteristics and features of the most representative wearable haptic systems for the fingertip and the hand

Kinesthetic Feedback for Robot-Assisted Minimally Invasive Surgery (Da Vinci) with Two Fingers Exoskeleton

Minimally Invasive Surgery and, in particular, Robotic Minimally Invasive Surgery may benefit from the integration of Haptic device: here we propose a preliminary study on a two-finger exoskeleton for kinesthetic feed-back of surgeon thumb and index finger while controlling a Da Vinci Robotic Device through its Master Tool Manipulator (MTM). Simulation of contact be-tween rigid and soft objects with the Patient Side Manipulator (PSM) are inte-grated with Force Feedback on the MTM coupled with the exoskeleton

Force Control for One Degree of Freedom Haptic Device using PID Controller

Haptics has been used as an additional feedback to increase human experience to the environment over years and its application has been widening into education, manufacturing and medical. The most developed haptic devices are for rehabilitation purpose. The rehabilitation process usually depends on the physiotherapist. But, it requires repetitive movements for long-term rehabilitation, thus haptic devices are needed. Most of the rehabilitation devices are included with haptic feedback to enhance therapy exercise during the rehabilitation process. However, the devices come with multiple degrees of freedom (DOF), complex design and costly. Rehabilitation for hand movement such as grasping, squeezing, holding and pinching usually does not need an expensive and complex device. Therefore, the goal of this study is to make an enhancement to One DOF Haptic Device for grasping rehabilitation exercise. It is improved to perform a force control mechanism with few types of conventional controller which are Proportional (P) controller, Proportional-Integral (PI) controller, Proportional-Derivative (PD) controller and Proportional-Integral-Derivative (PID) controller. The performance of the haptic device is tested with different conventional controller to obtain the best proposed controller based on the lowest value of Mean Square Error (MSE). The results show that PID Controller (MSE = 0.0028) is the most suitable for the haptic device with Proportional gain (Kp), Integral gain (Ki) and Derivative gain (Kd) are 1.3, 0.01 and 0.2 respectively. The force control mechanism can imitate the training motion of grasping movement for the patient

An exploration of grip force regulation with a low-impedance myoelectric prosthesis featuring referred haptic feedback

Abstract Background Haptic display technologies are well suited to relay proprioceptive, force, and contact cues from a prosthetic terminal device back to the residual limb and thereby reduce reliance on visual feedback. The ease with which an amputee interprets these haptic cues, however, likely depends on whether their dynamic signal behavior corresponds to expected behaviors—behaviors consonant with a natural limb coupled to the environment. A highly geared motor in a terminal device along with the associated high back-drive impedance influences dynamic interactions with the environment, creating effects not encountered with a natural limb. Here we explore grasp and lift performance with a backdrivable (low backdrive impedance) terminal device placed under proportional myoelectric position control that features referred haptic feedback. Methods We fabricated a back-drivable terminal device that could be used by amputees and non-amputees alike and drove aperture (or grip force, when a stiff object was in its grasp) in proportion to a myoelectric signal drawn from a single muscle site in the forearm. In randomly ordered trials, we assessed the performance of N=10 participants (7 non-amputee, 3 amputee) attempting to grasp and lift an object using the terminal device under three feedback conditions (no feedback, vibrotactile feedback, and joint torque feedback), and two object weights that were indiscernible by vision. Results Both non-amputee and amputee participants scaled their grip force according to the object weight. Our results showed only minor differences in grip force, grip/load force coordination, and slip as a function of sensory feedback condition, though the grip force at the point of lift-off for the heavier object was significantly greater for amputee participants in the presence of joint torque feedback. An examination of grip/load force phase plots revealed that our amputee participants used larger safety margins and demonstrated less coordination than our non-amputee participants. Conclusions Our results suggest that a backdrivable terminal device may hold advantages over non-backdrivable devices by allowing grip/load force coordination consistent with behaviors observed in the natural limb. Likewise, the inconclusive effect of referred haptic feedback on grasp and lift performance suggests the need for additional testing that includes adequate training for participants.http://deepblue.lib.umich.edu/bitstream/2027.42/116041/1/12984_2015_Article_98.pd

Mechanisms for enabling closed-loop upper limb sensorimotor prosthetic control

Myoelectric upper limb prostheses are limited in their ability to provide sensory feedback to a user. The lack of sensory feedback forces prosthesis users to rely on visual feedback alone in manipulating objects, and often leads to abandonment of the prosthesis in favor of the user's unimpaired arm. Consequently, there is a critical need to develop mechanisms that enable people with upper limb amputations to be able to receive sensory feedback from the environment. The goal of this dissertation is to describe the development and evaluation of various mechanisms that enable simultaneous myoelectric control of hand prostheses with proprioceptive and touch/pressure feedback. Sensory feedback is enabled through the use of a passive skin stretch mechanism for proprioception (Chapter 2), an epidermal electronic device that can provide electrotactile stimulation (Chapter 3), and a custom-built prosthetic hand that relays contact and pressure information from the fingertips (Chapter 4). In each of these chapters, motor control is simultaneously enabled through the use of electromyographic sensors. The remainder of the dissertation focuses on a method of enabling long-term wear of electrotactile stimulation electrodes by modeling (Chapter 5) and controlling (Chapter 6) sensation intensity in response to changes in the impedance of the electrode-skin interface. The techniques described in this dissertation have the potential to improve prosthesis embodiment for a person with an upper limb amputation, with the ultimate goal of reducing prosthesis abandonment and improving quality of life

Sensors for Robotic Hands: A Survey of State of the Art

Recent decades have seen significant progress in the field of artificial hands. Most of the surveys, which try to capture the latest developments in this field, focused on actuation and control systems of these devices. In this paper, our goal is to provide a comprehensive survey of the sensors for artificial hands. In order to present the evolution of the field, we cover five year periods starting at the turn of the millennium. At each period, we present the robot hands with a focus on their sensor systems dividing them into categories, such as prosthetics, research devices, and industrial end-effectors.We also cover the sensors developed for robot hand usage in each era. Finally, the period between 2010 and 2015 introduces the reader to the state of the art and also hints to the future directions in the sensor development for artificial hands

Robotic simulators for tissue examination training with multimodal sensory feedback

Tissue examination by hand remains an essential technique in clinical practice. The effective application depends on skills in sensorimotor coordination, mainly involving haptic, visual, and auditory feedback. The skills clinicians have to learn can be as subtle as regulating finger pressure with breathing, choosing palpation action, monitoring involuntary facial and vocal expressions in response to palpation, and using pain expressions both as a source of information and as a constraint on physical examination. Patient simulators can provide a safe learning platform to novice physicians before trying real patients. This paper reviews state-of-the-art medical simulators for the training for the first time with a consideration of providing multimodal feedback to learn as many manual examination techniques as possible. The study summarizes current advances in tissue examination training devices simulating different medical conditions and providing different types of feedback modalities. Opportunities with the development of pain expression, tissue modeling, actuation, and sensing are also analyzed to support the future design of effective tissue examination simulators

Pseudo-haptics survey: Human-computer interaction in extended reality & teleoperation

Pseudo-haptic techniques are becoming increasingly popular in human-computer interaction. They replicate haptic sensations by leveraging primarily visual feedback rather than mechanical actuators. These techniques bridge the gap between the real and virtual worlds by exploring the brain’s ability to integrate visual and haptic information. One of the many advantages of pseudo-haptic techniques is that they are cost-effective, portable, and flexible. They eliminate the need for direct attachment of haptic devices to the body, which can be heavy and large and require a lot of power and maintenance. Recent research has focused on applying these techniques to extended reality and mid-air interactions. To better understand the potential of pseudo-haptic techniques, the authors developed a novel taxonomy encompassing tactile feedback, kinesthetic feedback, and combined categories in multimodal approaches, ground not covered by previous surveys. This survey highlights multimodal strategies and potential avenues for future studies, particularly regarding integrating these techniques into extended reality and collaborative virtual environments.info:eu-repo/semantics/publishedVersio