Perspective Switching in Virtual Environments

When exploring new environments, people regularly alternate among many sources of spatial information including direct visual input, navigation aids such as maps and mobile devices, and verbal route descriptions. These spatial representations typically depict the environment from one of two perspectives: first-person, embedded route perspective or top-down, bird's eye survey perspective. Visual spatial cognition research has explored the nature of learning within each of these perspectives independently, but little work has been done to explore how on-line visual processing of combined perspectives affects cognition, meaning there is little understanding of the cognitive costs of using different navigation tools to learn large-scale environments. This dissertation addresses such questions through two experiments that guide participants through simple paths in large-scale environments, each consisting of a simple path through a small virtual town presented on a desktop computer display. By timing participants' movement through each environment and how they respond to either externally-controlled or participant-controlled perspective switches, the experiments measure the cognitive load of visually processing dynamic perspectives during navigation. These on-line processing measures are complemented by tests of visual recognition and recall memory, which reveal how switching perspectives affects the accuracy of the resulting spatial mental model. The results indicate that the cognitive load associated with changing perspectives is primarily dependent on the quantity of visual information the change introduces -- the transformation itself is not particularly disorienting after the first exposure to the environment. Furthermore, although forced perspective switches do not appear to significantly affect spatial memory accuracy relative to viewing the environment from a consistent perspective, navigator-controlled switching results in significantly more accurate spatial memory, indicating that navigation aids which allow for perspective control might better support spatial learning than fixed-perspective interfaces. The findings also support previous research showing that route perspective navigation generally yields more accurate spatial memory than survey perspective learning, particularly after extensive experience in the environment. Overall, the findings demonstrate many new aspects of how perspective affects spatial cognition, with implications for spatial learning and the design of navigation aids

Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Sounds

Audio-visual navigation combines sight and hearing to navigate to a sound-emitting source in an unmapped environment. While recent approaches have demonstrated the benefits of audio input to detect and find the goal, they focus on clean and static sound sources and struggle to generalize to unheard sounds. In this work, we propose the novel dynamic audio-visual navigation benchmark which requires catching a moving sound source in an environment with noisy and distracting sounds, posing a range of new challenges. We introduce a reinforcement learning approach that learns a robust navigation policy for these complex settings. To achieve this, we propose an architecture that fuses audio-visual information in the spatial feature space to learn correlations of geometric information inherent in both local maps and audio signals. We demonstrate that our approach consistently outperforms the current state-of-the-art by a large margin across all tasks of moving sounds, unheard sounds, and noisy environments, on two challenging 3D scanned real-world environments, namely Matterport3D and Replica. The benchmark is available at http://dav-nav.cs.uni-freiburg.de

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Social Attention: Modeling Attention in Human Crowds

Robots that navigate through human crowds need to be able to plan safe, efficient, and human predictable trajectories. This is a particularly challenging problem as it requires the robot to predict future human trajectories within a crowd where everyone implicitly cooperates with each other to avoid collisions. Previous approaches to human trajectory prediction have modeled the interactions between humans as a function of proximity. However, that is not necessarily true as some people in our immediate vicinity moving in the same direction might not be as important as other people that are further away, but that might collide with us in the future. In this work, we propose Social Attention, a novel trajectory prediction model that captures the relative importance of each person when navigating in the crowd, irrespective of their proximity. We demonstrate the performance of our method against a state-of-the-art approach on two publicly available crowd datasets and analyze the trained attention model to gain a better understanding of which surrounding agents humans attend to, when navigating in a crowd