Objective: This study aims to understand the cognitive impact of latency in
teleoperation and the related mitigation methods, using functional
Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity.
Background: Latency between command, execution, and feedback in teleoperation
can impair performance and affect operators mental state. The neural
underpinnings of these effects are not well understood. Method: A human subject
experiment (n = 41) of a simulated remote robot manipulation task was
performed. Three conditions were tested: no latency, with visual and haptic
latency, with visual latency and no haptic latency. fNIRS and performance data
were recorded and analyzed. Results: The presence of latency in teleoperation
significantly increased functional connectivity within and between prefrontal
and motor cortexes. Maintaining visual latency while providing real-time haptic
feedback reduced the average functional connectivity in all cortical networks
and showed a significantly different connectivity ratio within prefrontal and
motor cortical networks. The performance results showed the worst performance
in the all-delayed condition and best performance in no latency condition,
which echoes the neural activity patterns. Conclusion: The study provides
neurological evidence that latency in teleoperation increases cognitive load,
anxiety, and challenges in motion planning and control. Real-time haptic
feedback, however, positively influences neural pathways related to cognition,
decision-making, and sensorimotor processes. Application: This research can
inform the design of ergonomic teleoperation systems that mitigate the effects
of latency.Comment: Submitted to Human Factor