Coupling camera-tracked humans with a simulated virtual crowd

Our objective with this paper is to show how we can couple a group of real people and a simulated crowd of virtual humans. We attach group behaviors to the simulated humans to get a plausible reaction to real people. We use a two stage system: in the first stage, a group of people are segmented from a live video, then a human detector algorithm extracts the positions of the people in the video, which are finally used to feed the second stage, the simulation system. The positions obtained by this process allow the second module to render the real humans as avatars in the scene, while the behavior of additional virtual humans is determined by using a simulation based on a social forces model. Developing the method required three specific contributions: a GPU implementation of the codebook algorithm that includes an auxiliary codebook to improve the background subtraction against illumination changes; the use of semantic local binary patterns as a human descriptor; the parallelization of a social forces model, in which we solve a case of agents merging with each other. The experimental results show how a large virtual crowd reacts to over a dozen humans in a real environment.Peer ReviewedPostprint (author’s final draft

Interactive Tracking, Prediction, and Behavior Learning of Pedestrians in Dense Crowds

The ability to automatically recognize human motions and behaviors is a key skill for autonomous machines to exhibit to interact intelligently with a human-inhabited environment. The capabilities autonomous machines should have include computing the motion trajectory of each pedestrian in a crowd, predicting his or her position in the near future, and analyzing the personality characteristics of the pedestrian. Such techniques are frequently used for collision-free robot navigation, data-driven crowd simulation, and crowd surveillance applications. However, prior methods for these problems have been restricted to low-density or sparse crowds where the pedestrian movement is modeled using simple motion models. In this thesis, we present several interactive algorithms to extract pedestrian trajectories from videos in dense crowds. Our approach combines different pedestrian motion models with particle tracking and mixture models and can obtain an average of $20\%$ improvement in accuracy in medium-density crowds over prior work. We compute the pedestrian dynamics from these trajectories using Bayesian learning techniques and combine them with global methods for long-term pedestrian prediction in densely crowded settings. Finally, we combine these techniques with Personality Trait Theory to automatically classify the dynamic behavior or the personality of a pedestrian based on his or her movements in a crowded scene. The resulting algorithms are robust and can handle sparse and noisy motion trajectories. We demonstrate the benefits of our long-term prediction and behavior classification methods in dense crowds and highlight the benefits over prior techniques. We highlight the performance of our novel algorithms on three different applications. The first application is interactive data-driven crowd simulation, which includes crowd replication as well as the combination of pedestrian behaviors from different videos. Secondly, we combine the prediction scheme with proxemic characteristics from psychology and use them to perform socially-aware navigation. Finally, we present novel techniques for anomaly detection in low-to medium-density crowd videos using trajectory-level behavior learning.Doctor of Philosoph

Efficient resource allocation for automotive active vision systems

Individual mobility on roads has a noticeable impact upon peoples' lives, including traffic accidents resulting in severe, or even lethal injuries. Therefore the main goal when operating a vehicle is to safely participate in road-traffic while minimising the adverse effects on our environment. This goal is pursued by road safety measures ranging from safety-oriented road design to driver assistance systems. The latter require exteroceptive sensors to acquire information about the vehicle's current environment. In this thesis an efficient resource allocation for automotive vision systems is proposed. The notion of allocating resources implies the presence of processes that observe the whole environment and that are able to effeciently direct attentive processes. Directing attention constitutes a decision making process dependent upon the environment it operates in, the goal it pursues, and the sensor resources and computational resources it allocates. The sensor resources considered in this thesis are a subset of the multi-modal sensor system on a test vehicle provided by Audi AG, which is also used to evaluate our proposed resource allocation system. This thesis presents an original contribution in three respects. First, a system architecture designed to efficiently allocate both high-resolution sensor resources and computational expensive processes based upon low-resolution sensor data is proposed. Second, a novel method to estimate 3-D range motion, e cient scan-patterns for spin image based classifiers, and an evaluation of track-to-track fusion algorithms present contributions in the field of data processing methods. Third, a Pareto efficient multi-objective resource allocation method is formalised, implemented, and evaluated using road traffic test sequences

A framework for crowd simulation based on the JMonkey game engine

La simulación de multitudes juega un papel crucial cuando se trata del desarrollo de entornos inteligentes. La mayoría de los investigadores desarrollan simulaciones usando motores de juegos comerciales a través de los editores que éstos proporcionan. Esto di culta el poder realizar una experimentación profunda sobre simulaciones de multitudes, y fuerza que la línea de investigación deba atenerse al paradigma de desarrollo propuesto por la herramienta. El objetivo principal del trabajo desarrollado es la contribución de un simulador de multitudes basado en 3D, con una arquitectura modular y extensible, adecuada para la experimentación con simulaciones de multitudes. Este framework se centrará de forma especial en la navegación y la coordinación de multitudes sobre un modelo realista del entorno, capaz de reproducir situaciones del mundo real. El simulador incluye implementaciones de algoritmos conocidos para el movimiento de multitudes, integrando también implementaciones de terceros. El trabajo tiene en cuenta la necesidad de representaciones visualmente convincentes de la simulación más allá de las representaciones 2D, utilizadas regularmente en la literatura. Para ello, se contribuye con extensiones a herramientas de terceros que permiten importar texturas, animaciones y mallas que mejoran la calidad de la simulación. El desempeño de la simulación se demuestra en un caso de estudio donde el desafío es encontrar una población cuyo comportamiento, dentro del simulador, reproduce un determinado tráfico entrante / saliente medido en áreas específicas de un edificio. Este trabajo ha sido financiado por el proyecto MOSI-AGIL (S2013 / ICE-3019) a través de la Gobierno de la Comunidad de Madrid y Fondos Estructurales Europeos (FEDER)