Generating a Multipliciy of Policies for Agent Steering in Crowd Simulation

Pedestrian steering algorithms range from completely procedural to entirely data-driven, but the former grossly generalize across possible human behaviors and suffer computationally, whereas the latter are limited by the burden of ever-increasing data samples. Our approach seeks the balanced middle ground by deriving a collection of machine-learned policies based on the behavior of a procedural steering algorithm through the decomposition of the space of possible steering scenarios into steering contexts. The resulting algorithm scales well in the number of contexts, the use of new data sets to create new policies, and in the number of controlled agents as the policies become a simple evaluation of the rules asserted by the machine-learning process. We also explore the use of synthetic data from an “oracle algorithm” that serves as an as-needed source of samples, which can be stochastically polled for effective coverage. We observe that our approach produces pedestrian steering similar to that of the oracle steering algorithm, but with a significant performance boost. Runtime was reduced from hours under the oracle algorithm with 10 agents to on the order of 10 frames per second (FPS) with 3000 agents. We also analyze the nature of collisions in such a framework with no explicit collision avoidance

Steering Contexts for Autonomous Agents Using Synthetic Data

Data-driven techniques have become synonymous with replication of real-world phenomena. Efforts have been underway to use these techniques in crowd simulation through a mapping of pedestrian trajectories onto virtual agents using a similarity of circumstance. These works have exposed two fundamental issues with data-driven crowds. First, robust real-world data is logistically difficult to accurately collect and filled with unknown variables, such as a person\u27s mental state, which change behavior without providing a means to replicate their effects. Second, current data-driven approaches store and search the entire set of training data to decide the next course of action for each agent. A straightforward single-model system would alleviate the burden of storing and searching the data. The problem with a monolithic model, though, is that a single steering policy cannot handle all possible scenarios. To counter this we propose the splitting of possible scenarios into separable contexts, with each context in turn learning a model. The model used by an agent can then be dynamically swapped at runtime based on the evolving conditions around the agent. This results in a more scalable approach to data-driven simulation. In lieu of tracked data from real pedestrians, we propose the use of an oracle steering algorithm. This algorithm stands in for real data and can be queried for a steering decision for any combination of factors. This allows us to more thoroughly explore the problem space as needed. Furthermore, we can control all variables and capture behavior from scenarios that are otherwise infeasible to adequately sample in reality. This synthetic source of training data allows for a scalable and structured approach to training machine-learned models which virtual agents can use to navigate at runtime

Emergent behaviors and scalability for multi-agent reinforcement learning-based pedestrian models

This paper analyzes the emergent behaviors of pedestrian groups that learn through the multiagent reinforcement learning model developed in our group. Five scenarios studied in the pedestrian model literature, and with different levels of complexity, were simulated in order to analyze the robustness and the scalability of the model. Firstly, a reduced group of agents must learn by interaction with the environment in each scenario. In this phase, each agent learns its own kinematic controller, that will drive it at a simulation time. Secondly, the number of simulated agents is increased, in each scenario where agents have previously learnt, to test the appearance of emergent macroscopic behaviors without additional learning. This strategy allows us to evaluate the robustness and the consistency and quality of the learned behaviors. For this purpose several tools from pedestrian dynamics, such as fundamental diagrams and density maps, are used. The results reveal that the developed model is capable of simulating human-like micro and macro pedestrian behaviors for the simulation scenarios studied, including those where the number of pedestrians has been scaled by one order of magnitude with respect to the situation learned.This work has been supported by grant TIN2015-65686-C5-1-R of Ministerio de Economía y Competitividad

Authoring virtual crowds: a survey

Recent advancements in crowd simulation unravel a wide range of functionalities for virtual agents, delivering highly-realistic,natural virtual crowds. Such systems are of particular importance to a variety of applications in fields such as: entertainment(e.g., movies, computer games); architectural and urban planning; and simulations for sports and training. However, providingtheir capabilities to untrained users necessitates the development of authoring frameworks. Authoring virtual crowds is acomplex and multi-level task, varying from assuming control and assisting users to realise their creative intents, to deliveringintuitive and easy to use interfaces, facilitating such control. In this paper, we present a categorisation of the authorable crowdsimulation components, ranging from high-level behaviours and path-planning to local movements, as well as animation andvisualisation. We provide a review of the most relevant methods in each area, emphasising the amount and nature of influencethat the users have over the final result. Moreover, we discuss the currently available authoring tools (e.g., graphical userinterfaces, drag-and-drop), identifying the trends of early and recent work. Finally, we suggest promising directions for futureresearch that mainly stem from the rise of learning-based methods, and the need for a unified authoring framework.This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital PolicyPeer ReviewedPostprint (author's final draft

Learning Multi-Agent Navigation from Human Crowd Data

The task of safely steering agents amidst static and dynamic obstacles has many applications in robotics, graphics, and traffic engineering. While decentralized solutions are essential for scalability and robustness, achieving globally efficient motions for the entire system of agents is equally important. In a traditional decentralized setting, each agent relies on an underlying local planning algorithm that takes as input a preferred velocity and the current state of the agent\u27s neighborhood and then computes a new velocity for the next time-step that is collision-free and as close as possible to the preferred one. Typically, each agent promotes a goal-oriented preferred velocity, which can result in myopic behaviors as actions that are locally optimal for one agent is not necessarily optimal for the global system of agents. In this thesis, we explore a human-inspired approach for efficient multi-agent navigation that allows each agent to intelligently adapt its preferred velocity based on feedback from the environment. Using supervised learning, we investigate different egocentric representations of the local conditions that the agents face and train various deep neural network architectures on extensive collections of human trajectory datasets to learn corresponding life-like velocities. During simulation, we use the learned velocities as high-level, preferred velocities signals passed as input to the underlying local planning algorithm of the agents. We evaluate our proposed framework using two state-of-the-art local methods, the ORCA method, and the PowerLaw method. Qualitative and quantitative results on a range of scenarios show that adapting the preferred velocity results in more time- and energy-efficient navigation policies, allowing agents to reach their destinations faster as compared to agents simulated with vanilla ORCA and PowerLaw

Automated Discovery of Candidate Simulation Models for Steering Behavior Simulation

Steering behavior of autonomous agents plays important roles in many simulation applications, such as simulation of pedestrian crowds, simulation of evacuation scenarios, simulation of ecosystems, simulation of autonomous robots, and simulation of artificial life in virtual environments used in computer games. It is desirable to have an approach that can automatically discover multiple candidate models for steering behavior simulation besides manual approach (trial-and-error fashion) and data-driven approach. Towards this goal, this work presents an approach that searches for candidate models of steering behavior in an automated way. The proposed framework includes two components. A model space specification provides a formal specification for a general structure from which various models can be constructed, and a search method to search for a set of candidate models based on requirements. To support more complex scenarios, we further add three major extensions including: (1) Activation component assign dynamic priorities for behaviors depending on surround environments. (2) Multiple search stages are provided to assist the evolutionary search algorithm to distribute computational resources better. (3) A special type of entity called space entity to assist agents receive information not only from other entities (agents, obstacles), but also from surrounding empty space. The approach is able to discover multiple candidate models for three basic steering behaviors including the leader- following ( Bleader_following), personal space maintenance ( Bpersonal_space), and mobile obstacle avoidance ( Bobstacle_avoidance). The results show that different possibilities of steering behavior support modelers to have a better understanding of the problem under study, hence assist modelers to develop more advanced models by testing different combinations of the basic steering behaviors. We evaluate all combinations between three basic steering behaviors including: (1) Bleader_following + Bobstacle_avoidance, (2) Bobstacle_avoidance + Bpersonal_space, (3) Bleader_following + Bpersonal_space, and (4) Bleader_following + Bobstacle_avoidance + Bpersonal_space. We further test the approach with two variations of scenario 4: (5) The leader surrounding + Bpersonal_space, (6) Hall-way evacuation with an obstacle in the middle. The results show that the framework is also able to discover multiple models for each of these composite steering behaviors, and several of them have good scalability and robustness

Foundations of Human-Aware Planning -- A Tale of Three Models

abstract: A critical challenge in the design of AI systems that operate with humans in the loop is to be able to model the intentions and capabilities of the humans, as well as their beliefs and expectations of the AI system itself. This allows the AI system to be "human- aware" -- i.e. the human task model enables it to envisage desired roles of the human in joint action, while the human mental model allows it to anticipate how its own actions are perceived from the point of view of the human. In my research, I explore how these concepts of human-awareness manifest themselves in the scope of planning or sequential decision making with humans in the loop. To this end, I will show (1) how the AI agent can leverage the human task model to generate symbiotic behavior; and (2) how the introduction of the human mental model in the deliberative process of the AI agent allows it to generate explanations for a plan or resort to explicable plans when explanations are not desired. The latter is in addition to traditional notions of human-aware planning which typically use the human task model alone and thus enables a new suite of capabilities of a human-aware AI agent. Finally, I will explore how the AI agent can leverage emerging mixed-reality interfaces to realize effective channels of communication with the human in the loop.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Bearing-Only Control Laws For Balanced Circular Formations of Ground Robots

Abstract — For a group of constant-speed ground robots, a simple control law is designed to stabilize the motion of the group into a balanced circular formation using a consensus approach. It is shown that the measurements of the bearing angles between the robots are sufficient for reaching a balanced circular formation. We consider two different scenarios that the connectivity graph of the system is either a complete graph or a ring. Collision avoidance capabilities are added to the team members and the effectiveness of the control laws are demonstrated on a group of mobile robots. I