Synthesis Study on Employing Snowplow Driving Simulators in Training

Departments of Transportation (DOTs) need to mobilize workers under harsh weather conditions for winter operations. Traditional snowplow driver training at INDOT is usually conducted annually before the snow season; therefore, it does not replicate the conditions which drivers will be exposed to during winter operations. To this point, some state DOTs have incorporated simulators in their snowplow driver training. Despite this raised interest, few studies have (1) surveyed other state DOTs about the use of this equipment in winter operations driver training, or (2) provided a systematic consideration of all factors involved in the decision to use driving simulators in snowplow driver training. To fill these gaps, the present study synthesizes information from previous literature, revises current information from INDOT, and surveys other state DOTs to identify the benefits and challenges of driving simulators for snowplow driver training. A mixed methods approach was utilized including a review of current INDOT practices, interviews with stakeholders, a survey of other state DOTs, and results from a pilot training. Based on the findings, the researchers recommend that INDOT continues to explore the use of driving simulators for training purposes in addition to the yearly snowplow driver training, due the ability to reinforce learning in a safe environment. Moreover, the research team suggests the following areas for further research: evaluating optimal simulator “seat time,” peer learning in simulator training, and the impact of experience level and work assignment in the perception of driving simulator training effectiveness

DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis

We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth. The synthesized realistic depth can then be used to train task-specific networks facilitating label transfer from the synthetic domain. Unlike existing image synthesis pipelines, where geometries are mostly ignored, we treat geometries carried by the depth scans based on their own existence. We propose differential contrastive learning that explicitly enforces the underlying geometric properties to be invariant regarding the real variations been learned. The resulting depth synthesis method is task-agnostic, and we demonstrate the effectiveness of the proposed synthesis method by extensive evaluations on real-world geometric reasoning tasks. The networks trained with the depth synthesized by our method consistently achieve better performance across a wide range of tasks than state of the art, and can even surpass the networks supervised with full real-world annotations when slightly fine-tuned, showing good transferability.Comment: Accepted by International Conference on Robotics and Automation (ICRA) 2022 and RA-L 202

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations

Robust reinforcement learning (RL) seeks to train policies that can perform well under environment perturbations or adversarial attacks. Existing approaches typically assume that the space of possible perturbations remains the same across timesteps. However, in many settings, the space of possible perturbations at a given timestep depends on past perturbations. We formally introduce temporally-coupled perturbations, presenting a novel challenge for existing robust RL methods. To tackle this challenge, we propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially-observable two-player zero-sum game. By finding an approximate equilibrium in this game, GRAD ensures the agent's robustness against temporally-coupled perturbations. Empirical experiments on a variety of continuous control tasks demonstrate that our proposed approach exhibits significant robustness advantages compared to baselines against both standard and temporally-coupled attacks, in both state and action spaces

TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning

Despite recent progress in reinforcement learning (RL) from raw pixel data, sample inefficiency continues to present a substantial obstacle. Prior works have attempted to address this challenge by creating self-supervised auxiliary tasks, aiming to enrich the agent's learned representations with control-relevant information for future state prediction. However, these objectives are often insufficient to learn representations that can represent the optimal policy or value function, and they often consider tasks with small, abstract discrete action spaces and thus overlook the importance of action representation learning in continuous control. In this paper, we introduce TACO: Temporal Action-driven Contrastive Learning, a simple yet powerful temporal contrastive learning approach that facilitates the concurrent acquisition of latent state and action representations for agents. TACO simultaneously learns a state and an action representation by optimizing the mutual information between representations of current states paired with action sequences and representations of the corresponding future states. Theoretically, TACO can be shown to learn state and action representations that encompass sufficient information for control, thereby improving sample efficiency. For online RL, TACO achieves 40% performance boost after one million environment interaction steps on average across nine challenging visual continuous control tasks from Deepmind Control Suite. In addition, we show that TACO can also serve as a plug-and-play module adding to existing offline visual RL methods to establish the new state-of-the-art performance for offline visual RL across offline datasets with varying quality

GIMO: Gaze-Informed Human Motion Prediction in Context

Predicting human motion is critical for assistive robots and AR/VR applications, where the interaction with humans needs to be safe and comfortable. Meanwhile, an accurate prediction depends on understanding both the scene context and human intentions. Even though many works study scene-aware human motion prediction, the latter is largely underexplored due to the lack of ego-centric views that disclose human intent and the limited diversity in motion and scenes. To reduce the gap, we propose a large-scale human motion dataset that delivers high-quality body pose sequences, scene scans, as well as ego-centric views with eye gaze that serves as a surrogate for inferring human intent. By employing inertial sensors for motion capture, our data collection is not tied to specific scenes, which further boosts the motion dynamics observed from our subjects. We perform an extensive study of the benefits of leveraging eye gaze for ego-centric human motion prediction with various state-of-the-art architectures. Moreover, to realize the full potential of gaze, we propose a novel network architecture that enables bidirectional communication between the gaze and motion branches. Our network achieves the top performance in human motion prediction on the proposed dataset, thanks to the intent information from the gaze and the denoised gaze feature modulated by the motion. The proposed dataset and our network implementation will be publicly available

Effect of Anodal Direct-Current Stimulation on Cortical Hemodynamic Responses With Laser-Speckle Contrast Imaging

Transcranial direct-current stimulation (DCS) offers a method for noninvasive neuromodulation usable in basic and clinical human neuroscience. Laser-speckle contrast imaging (LSCI), a powerful, low-cost method for obtaining images of dynamic systems, can detect regional blood-flow distributions with high spatial and temporal resolutions. Here, we used LSCI for measuring DCS-induced cerebral blood flow in real-time. Results showed that the change-rate of cerebral blood flow could reach approximately 10.1 ± 5.1% by DCS, indicating that DCS can increase cerebral blood flow and alter cortical hemodynamic responses. Thus, DCS shows potential for the clinical treatment and rehabilitation of ischemic strokes