Sensory Manipulation as a Countermeasure to Robot Teleoperation Delays: System and Evidence

In the field of robotics, robot teleoperation for remote or hazardous environments has become increasingly vital. A major challenge is the lag between command and action, negatively affecting operator awareness, performance, and mental strain. Even with advanced technology, mitigating these delays, especially in long-distance operations, remains challenging. Current solutions largely focus on machine-based adjustments. Yet, there's a gap in using human perceptions to improve the teleoperation experience. This paper presents a unique method of sensory manipulation to help humans adapt to such delays. Drawing from motor learning principles, it suggests that modifying sensory stimuli can lessen the perception of these delays. Instead of introducing new skills, the approach uses existing motor coordination knowledge. The aim is to minimize the need for extensive training or complex automation. A study with 41 participants explored the effects of altered haptic cues in delayed teleoperations. These cues were sourced from advanced physics engines and robot sensors. Results highlighted benefits like reduced task time and improved perceptions of visual delays. Real-time haptic feedback significantly contributed to reduced mental strain and increased confidence. This research emphasizes human adaptation as a key element in robot teleoperation, advocating for improved teleoperation efficiency via swift human adaptation, rather than solely optimizing robots for delay adjustment.Comment: Submitted to Scientific Report

Neural Dynamics of Delayed Feedback in Robot Teleoperation: Insights from fNIRS Analysis

As robot teleoperation increasingly becomes integral in executing tasks in distant, hazardous, or inaccessible environments, the challenge of operational delays remains a significant obstacle. These delays are inherent in signal transmission and processing and can adversely affect the operators performance, particularly in tasks requiring precision and timeliness. While current research has made strides in mitigating these delays through advanced control strategies and training methods, a crucial gap persists in understanding the neurofunctional impacts of these delays and the efficacy of countermeasures from a cognitive perspective. Our study narrows this gap by leveraging functional Near-Infrared Spectroscopy (fNIRS) to examine the neurofunctional implications of simulated haptic feedback on cognitive activity and motor coordination under delayed conditions. In a human-subject experiment (N=41), we manipulated sensory feedback to observe its influences on various brain regions of interest (ROIs) response during teleoperation tasks. The fNIRS data provided a detailed assessment of cerebral activity, particularly in ROIs implicated in time perception and the execution of precise movements. Our results reveal that certain conditions, which provided immediate simulated haptic feedback, significantly optimized neural functions related to time perception and motor coordination, and improved motor performance. These findings provide empirical evidence about the neurofunctional basis of the enhanced motor performance with simulated synthetic force feedback in the presence of teleoperation delays.Comment: Submitted to Frontiers in Human Neuroscienc

Sensory substitution for space gloves and for space robots

Sensory substitution systems for space applications are described. Physical sensors replace missing human receptors and feed information to the interpretive centers of a different sense. The brain is plastic enough so that, with training, the subject localizes the input as if it were received through the missing receptors. Astronauts have difficulty feeling objects through space suit gloves because of their thickness and because of the 4.3 psi pressure difference. Miniature force sensors on the glove palm drive an electrotactile belt around the waist, thus augmenting the missing tactile sensation. A proposed teleoperator system with telepresence for a space robot would incorporate teleproprioception and a force sensor/electrotactile belt sensory substitution system for teletouch

Investigating the Usability of a Vibrotactile Torso Display for Improving Simulated Teleoperation Obstacle Avoidance

While unmanned ground vehicle (UGV) teleoperation is advantageous in terms of adaptability and safety, it introduces challenges resulting from the operator\u27s poor perception of the remote environment. Previous literature on the ability of haptic feedback to augment visual displays indicates that UGV obstacle avoidance information may be more meaningfully communicated via vibrotactile torso systems. Presenting this information so that operators can accurately detect the proximity from walls and obstructions could result in a significant reduction in errors, ultimately improving task performance and increasing the usability of teleoperation. The goal of the current study was to determine the degree to which a vibrotactile torso belt could improve UGV teleoperation performance over video feed alone in a simulated environment. Sixty operators controlled a UGV using a simulated video feed, while half also utilized a vibrotactile belt. Results indicated that the vibrotactile display did not improve navigational performance or decrease subjective workload over video feed alone. Possible reasons for this and limitations are discussed

Method to Estimate Human Inattention in Teleoperation of Mobile Robots

In teleoperation of mobile robots the operator is remotely located. As a result, generally the human perception of the remote environment is distorted affecting the mission negatively. Visual information can be degraded because of video images bandwith, time lags, frame rates, point of view and motion effects among other reasons. Although many researchers have proposed a variety of methods for measuring perception, just a few can be used in control closed loop systems. This paper aims to provide a novel metric to the human visual inattention upon risk for a remotely navigated mobile robot. We present both qualitative and quantitative guidelines for designing the metric in a teleoperation of a mobile robot. The method allows to incorporate the metric in a control closed loop system, and task consists in guiding the robot from an initial point to a final one as quick as possible, considering the constraint of avoiding collisions. Furthermore, a haptic cue based on the metric is proposed in order to help the human to avoid collisions. A system stability analysis considering time varying delays is proposed. Additionally, we present a human in the loop experiment of a teleoperation of a 3D mobile robot simulator in order to remark the advantages of using human factors in the controller.Fil: Penizzotto Bacha, Franco Victor. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico San Juan. Instituto de Automática; ArgentinaFil: Mut, Vicente Antonio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico San Juan. Instituto de Automática; ArgentinaFil: Slawiñski, Emanuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico San Juan. Instituto de Automática; Argentin

An intelligent, free-flying robot

The ground based demonstration of the extensive extravehicular activity (EVA) Retriever, a voice-supervised, intelligent, free flying robot, is designed to evaluate the capability to retrieve objects (astronauts, equipment, and tools) which have accidentally separated from the Space Station. The major objective of the EVA Retriever Project is to design, develop, and evaluate an integrated robotic hardware and on-board software system which autonomously: (1) performs system activation and check-out; (2) searches for and acquires the target; (3) plans and executes a rendezvous while continuously tracking the target; (4) avoids stationary and moving obstacles; (5) reaches for and grapples the target; (6) returns to transfer the object; and (7) returns to base

Brain Functional Connectivity under Teleoperation Latency: a fNIRS Study

Objective: This study aims to understand the cognitive impact of latency in teleoperation and the related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity. Background: Latency between command, execution, and feedback in teleoperation can impair performance and affect operators mental state. The neural underpinnings of these effects are not well understood. Method: A human subject experiment (n = 41) of a simulated remote robot manipulation task was performed. Three conditions were tested: no latency, with visual and haptic latency, with visual latency and no haptic latency. fNIRS and performance data were recorded and analyzed. Results: The presence of latency in teleoperation significantly increased functional connectivity within and between prefrontal and motor cortexes. Maintaining visual latency while providing real-time haptic feedback reduced the average functional connectivity in all cortical networks and showed a significantly different connectivity ratio within prefrontal and motor cortical networks. The performance results showed the worst performance in the all-delayed condition and best performance in no latency condition, which echoes the neural activity patterns. Conclusion: The study provides neurological evidence that latency in teleoperation increases cognitive load, anxiety, and challenges in motion planning and control. Real-time haptic feedback, however, positively influences neural pathways related to cognition, decision-making, and sensorimotor processes. Application: This research can inform the design of ergonomic teleoperation systems that mitigate the effects of latency.Comment: Submitted to Human Factor