Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

LCrowdV: Generating Labeled Videos for Simulation-based Crowd Behavior Learning

We present a novel procedural framework to generate an arbitrary number of labeled crowd videos (LCrowdV). The resulting crowd video datasets are used to design accurate algorithms or training models for crowded scene understanding. Our overall approach is composed of two components: a procedural simulation framework for generating crowd movements and behaviors, and a procedural rendering framework to generate different videos or images. Each video or image is automatically labeled based on the environment, number of pedestrians, density, behavior, flow, lighting conditions, viewpoint, noise, etc. Furthermore, we can increase the realism by combining synthetically-generated behaviors with real-world background videos. We demonstrate the benefits of LCrowdV over prior lableled crowd datasets by improving the accuracy of pedestrian detection and crowd behavior classification algorithms. LCrowdV would be released on the WWW

Towards agent-based crowd simulation in airports using games technology

We adapt popular video games technology for an agent-based crowd simulation in an airport terminal. To achieve this, we investigate the unique traits of airports and implement a virtual crowd by exploiting a scalable layered intelligence technique in combination with physics middleware and a socialforces approach. Our experiments show that the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating navigation behaviour

Online Learning and Planning for Crowd-aware Service Robot Navigation

Mobile service robots are increasingly used in indoor environments (e.g., shopping malls or museums) among large crowds of people. To efficiently navigate in these environments, such a robot should be able to exhibit a variety of behaviors. It should avoid crowded areas, and not oppose the flow of the crowd. It should be able to identify and avoid specific crowds that result in additional delays (e.g., children in a particular area might slow down the robot). and to seek out a crowd if its task requires it to interact with as many people as possible. These behaviors require the ability to learn and model crowd behavior in an environment. Earlier work used a dataset of paths navigated by people to solve this problem. That approach is expensive, risks privacy violations, and can become outdated as the environment evolves. To overcome these drawbacks, this thesis proposes a new approach where the robot learns models of crowd behavior online and relies only on local onboard sensors. This work develops and tests multiple planners that leverage these models in simulated environments and demonstrate statistically significant improvements in performance. The work reported here is applicable not only to navigation to target locations, but also to a variety of other services

The design and simulation of traffic networks in virtual environments

For over half a century, researchers from a diverse set of disciplines have been studying the behaviour of traffic flow to better understand the causes of traffic congestion, accidents, and related phenomena. As the global population continues to rise, there is an increasing demand for more efficient and effective transportation infrastructures that are able to accommodate a greater number of civilians without compromising travel times, journey quality, cost, or accessibility. With recent advances in computing technology, transportation infrastructures are now typically developed using design and simulation packages that enable engineers to accurately model large-scale road networks and evaluate their designs through visual simulation. However, as these projects increase in scale and complexity, methodologies to intuitively design more complex and realistic simulations are highly desirable. The need of such technology translates across to the entertainment industry, where traffic simulations are integrated into computer games, television, film, and virtual tourism applications to enhance the realism and believability of the simulated scenario. In this thesis two significant challenges related to the design and simulation of traffic networks for use in virtual environments are presented. The first challenge is the development of intuitive techniques to assist the design and construction of high-fidelity three-dimensional road networks for use in both urban and rural virtual environments. The second challenge considers the implementation of computational models to accurately simulate the behaviour of drivers and pedestrians in transportation networks, in real time. An overview of the literature in the field is presented in this work with novel contributions relating to the challenges defined above

Interactive Tracking, Prediction, and Behavior Learning of Pedestrians in Dense Crowds

The ability to automatically recognize human motions and behaviors is a key skill for autonomous machines to exhibit to interact intelligently with a human-inhabited environment. The capabilities autonomous machines should have include computing the motion trajectory of each pedestrian in a crowd, predicting his or her position in the near future, and analyzing the personality characteristics of the pedestrian. Such techniques are frequently used for collision-free robot navigation, data-driven crowd simulation, and crowd surveillance applications. However, prior methods for these problems have been restricted to low-density or sparse crowds where the pedestrian movement is modeled using simple motion models. In this thesis, we present several interactive algorithms to extract pedestrian trajectories from videos in dense crowds. Our approach combines different pedestrian motion models with particle tracking and mixture models and can obtain an average of $20\%$ improvement in accuracy in medium-density crowds over prior work. We compute the pedestrian dynamics from these trajectories using Bayesian learning techniques and combine them with global methods for long-term pedestrian prediction in densely crowded settings. Finally, we combine these techniques with Personality Trait Theory to automatically classify the dynamic behavior or the personality of a pedestrian based on his or her movements in a crowded scene. The resulting algorithms are robust and can handle sparse and noisy motion trajectories. We demonstrate the benefits of our long-term prediction and behavior classification methods in dense crowds and highlight the benefits over prior techniques. We highlight the performance of our novel algorithms on three different applications. The first application is interactive data-driven crowd simulation, which includes crowd replication as well as the combination of pedestrian behaviors from different videos. Secondly, we combine the prediction scheme with proxemic characteristics from psychology and use them to perform socially-aware navigation. Finally, we present novel techniques for anomaly detection in low-to medium-density crowd videos using trajectory-level behavior learning.Doctor of Philosoph

Social robot navigation tasks: combining machine learning techniques and social force model

© 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/)Social robot navigation in public spaces, buildings or private houses is a difficult problem that is not well solved due to environmental constraints (buildings, static objects etc.), pedestrians and other mobile vehicles. Moreover, robots have to move in a human-aware manner—that is, robots have to navigate in such a way that people feel safe and comfortable. In this work, we present two navigation tasks, social robot navigation and robot accompaniment, which combine machine learning techniques with the Social Force Model (SFM) allowing human-aware social navigation. The robots in both approaches use data from different sensors to capture the environment knowledge as well as information from pedestrian motion. The two navigation tasks make use of the SFM, which is a general framework in which human motion behaviors can be expressed through a set of functions depending on the pedestrians’ relative and absolute positions and velocities. Additionally, in both social navigation tasks, the robot’s motion behavior is learned using machine learning techniques: in the first case using supervised deep learning techniques and, in the second case, using Reinforcement Learning (RL). The machine learning techniques are combined with the SFM to create navigation models that behave in a social manner when the robot is navigating in an environment with pedestrians or accompanying a person. The validation of the systems was performed with a large set of simulations and real-life experiments with a new humanoid robot denominated IVO and with an aerial robot. The experiments show that the combination of SFM and machine learning can solve human-aware robot navigation in complex dynamic environments.This research was supported by the grant MDM-2016-0656 funded by MCIN/AEI / 10.13039/501100011033, the grant ROCOTRANSP PID2019-106702RB-C21 funded by MCIN/AEI/ 10.13039/501100011033 and the grant CANOPIES H2020-ICT-2020-2-101016906 funded by the European Union.Peer ReviewedPostprint (published version