Generating, animating, and rendering varied individuals for real-time crowds

To simulate realistic crowds of virtual humans in real time, three main requirements need satisfaction. First of all, quantity, i.e., the ability to simulate thousands of characters. Secondly, quality, because each virtual human composing a crowd needs to look unique in its appearance and animation. Finally, efficiency is paramount, for an operation usually efficient on a single virtual human, becomes extremely costly when applied on large crowds. Developing an architecture able to manage all three aspects is a challenging problem that we have addressed in our research. Our first contribution is an efficient and versatile architecture called YaQ, able to simulate thousands of characters in real time. This platform, developed at EPFL-VRLab, results from several years of research and integrates state-of-the-art techniques at all levels: YaQ aims at providing efficient algorithms and real-time solutions for populating globally and massively large-scale empty environments. YaQ thus fits various application domains, such as video games and virtual reality. Our architecture is especially efficient in managing the large quantity of data that is used to simulate crowds. In order to simulate large crowds, many instances of a small set of human templates have to be generated. From this starting point, if no care is taken to vary each character individually, many clones appear in the crowd. We present several algorithms to make each individual unique in the crowd. Firstly, we introduce a new method to distinguish body parts of a human and apply detailed color variety and patterns to each one of them. Secondly, we present two techniques to modify the shape and profile of a virtual human: a simple and efficient method for attaching accessories to individuals, and efficient tools to scale the skeleton and mesh of an instance. Finally, we also contribute to varying individuals' animation by introducing variations to the upper body movements, thus allowing characters to make a phone call, have a hand in their pocket, or carry heavy accessories, etc. To achieve quantity in a crowd, levels of detail need to be used. We explore the most adequate solutions to simulate large crowds with levels of detail, while avoiding disturbing switches between two different representations of a virtual human. To do so, we develop solutions to make most variety techniques scalable to all levels of detail

Visual modeling and simulation of multiscale phenomena

Many large-scale systems seen in real life, such as human crowds, fluids, and granular materials, exhibit complicated motion at many different scales, from a characteristic global behavior to important small-scale detail. Such multiscale systems are computationally expensive for traditional simulation techniques to capture over the full range of scales. In this dissertation, I present novel techniques for scalable and efficient simulation of these large, complex phenomena for visual computing applications. These techniques are based on a new approach of representing a complex system by coupling together separate models for its large-scale and fine-scale dynamics. In fluid simulation, it remains a challenge to efficiently simulate fine local detail such as foam, ripples, and turbulence without compromising the accuracy of the large-scale flow. I present two techniques for this problem that combine physically-based numerical simulation for the global flow with efficient local models for detail. For surface features, I propose the use of texture synthesis, guided by the physical characteristics of the macroscopic flow. For turbulence in the fluid motion itself, I present a technique that tracks the transfer of energy from the mean flow to the turbulent fluctuations and synthesizes these fluctuations procedurally, allowing extremely efficient visual simulation of turbulent fluids. Another large class of problems which are not easily handled by traditional approaches is the simulation of very large aggregates of discrete entities, such as dense pedestrian crowds and granular materials. I present a technique for crowd simulation that couples a discrete per-agent model of individual navigation with a novel continuum formulation for the collective motion of pedestrians. This approach allows simulation of dense crowds of a hundred thousand agents at near-real-time rates on desktop computers. I also present a technique for simulating granular materials, which generalizes this model and introduces a novel computational scheme for friction. This method efficiently reproduces a wide range of granular behavior and allows two-way interaction with simulated solid bodies. In all of these cases, the proposed techniques are typically an order of magnitude faster than comparable existing methods. Through these applications to a diverse set of challenging simulation problems, I demonstrate the benefits of the proposed approach, showing that it is a powerful and versatile technique for the simulation of a broad range of large and complex systems

Authoring virtual crowds: a survey

Recent advancements in crowd simulation unravel a wide range of functionalities for virtual agents, delivering highly-realistic,natural virtual crowds. Such systems are of particular importance to a variety of applications in fields such as: entertainment(e.g., movies, computer games); architectural and urban planning; and simulations for sports and training. However, providingtheir capabilities to untrained users necessitates the development of authoring frameworks. Authoring virtual crowds is acomplex and multi-level task, varying from assuming control and assisting users to realise their creative intents, to deliveringintuitive and easy to use interfaces, facilitating such control. In this paper, we present a categorisation of the authorable crowdsimulation components, ranging from high-level behaviours and path-planning to local movements, as well as animation andvisualisation. We provide a review of the most relevant methods in each area, emphasising the amount and nature of influencethat the users have over the final result. Moreover, we discuss the currently available authoring tools (e.g., graphical userinterfaces, drag-and-drop), identifying the trends of early and recent work. Finally, we suggest promising directions for futureresearch that mainly stem from the rise of learning-based methods, and the need for a unified authoring framework.This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital PolicyPeer ReviewedPostprint (author's final draft

Agora : unified framework for crowd simulation research

Crowd simulation focuses on modeling the movements and behaviors of large groups of people. This area of study has become increasingly important because of its several applications in various fields such as urban planning, safety, and entertainment. In each of these domains, the presence of virtual agents exhibiting realistic behavior greatly enhances the quality of the simulations. However, the inherently multifaceted and intricate nature of human behavior presents a unique challenge, necessitating the effective combination of multiple behavior models. This thesis introduces a novel theoretical framework for modeling human behavior in crowd simulations, addressing the unresolved issue of combining a plethora of behavior models, often developed in isolation. The proposed framework decomposes human behavior into fundamental driving stimuli, which are then represented graphically through the heatmap paradigm. Subsequently, the agent behavior is influenced by the heatmaps, which guide them toward attractive areas and steer them away from repulsive locations based on the encoded stimuli. A key advantage of this approach lies in the ability to combine heatmaps using well-defined color operations, effectively integrating different aspects of human behavior. Furthermore, the heatmap paradigm facilitates objective comparison of simulation output with real-world data, employing image similarity metrics to evaluate model accuracy. To realize this framework, the thesis presents a modular software architecture designed to support various tasks involved in crowd simulation, emphasizing the separation of concerns for each task. This architecture comprises a collection of abstract modules, which are subsequently implemented using appropriate software components to realize the underlying features, resulting in the Agora framework. To assess the ability of Agora to support the various tasks involved in crowd simulation, two case studies are implemented and analyzed. The first case study simulates tourists visiting Þingvellir national park in Iceland, examining how their behavior is influenced by the visibility of the surrounding environment. The second case study employs Agora to model the thermal and density comfort levels of virtual pedestrians in an urban setting. The results demonstrate that Agora successfully supports the development, combination, and evaluation of crowd simulation models against real-world data. The authoring process, assisted by Agora, is significantly more streamlined compared to its native counterpart. The integration of multiple models is achieved by combining the heatmaps, resulting in plausible behavior, and the model assessment is made convenient through the evaluator within the framework. The thesis concludes by discussing the implications of these findings for the field of crowd simulation, highlighting the contributions and potential future directions of the Agora framework.Mannfjöldahermun fæst við gerð líkana af hreyfingu og hegðun stórra hópa af fólki. Mikilvægi þessa rannsóknasviðs hefur vaxið stöðugt vegna hagnýtingar á margvíslegum vetvangi, eins og til dæmis á vetvangi borgarskipulags, öryggis og afþreyingar. Þegar sýndarmenni hegða sér á sannfærandi hátt, leiðir það til betri hermunar fyrir þessi notkunarsvið. En mannleg hegðun er í eðli sínu margbrotin og flókin og því er það sérstök áskorun við smíði sýndarmenna að sameina, með áhrifaríkum hætti, mörg mismunandi hegðunarlíkön. Þessi ritgerð kynnir nýja fræðilega umgjörð líkanasmíði mannlegrar hegðunar fyrir mannfjöldahermun, sem tekur á þeim óleysta vanda að sameina fjölda hegðunarlíkana, sem oft eru þróuð með aðskildum hætti. Umgjörðin brýtur mannlega hegðun niður í grundvallar drifáreiti, sem eru sett fram grafískt útfrá hugmyndafræði hitakorta. Sýndarmennin hegða sér síðan undir áhrifum frá hitakortunum, sem vísa þeim í áttina að aðlaðandi svæðum og stýra þeim burt frá fráhrindandi svæðum, útfrá hinu umritaða áreiti. Lykilkostur þessarar nálgunar er sá eiginleiki að geta blandað saman hitakortum með vel skilgreindum litaaðgerðum, sem eru þá í raun samþætting mismunandi hliða mannlegrar hegðunar. Hitakortshugmyndafræðin auðveldar ennfremur hlutlægan samanburð hermunarúttaks og raungagna með notkun myndsamanburðarmælinga, til að meta nákvæmni líkana. Varðandi útfærslu, þá kynnir þessi ritgerð einingadrifna hugbúnaðarhögun sem er hönnuð til að styðja við ýmsa ferla mannfjöldahermunar, með áherslu á aðskilnað helstu viðfangsefna hvers ferlis. Þessi högun inniheldur safn huglægra eininga, sem síðan eru útfærðar með viðeigandi hugbúnaðarhlutum, sem raungera undirliggjandi eiginleika. Útkoman er sjálf Agora umbjörðin. Tvö sýnidæmi eru útfærð og greind til að meta getu Agoru til að styðja við ýmis mannfjöldahermunarverkefni. Fyrra dæmið hermir eftir ferðamönnum sem heimsækja Þingvallaþjóðgarð, og skoðar hvernig hegðun þeirra verður fyrir áhrifum sýnileika umhverfisins sem umleikur þá. Seinna dæmið nýtir Agoru til að smíða líkan af hitauppstreymis- og þéttleikaþægindum hjá sýndarvegfarendum í borgarumhverfi. Niðurstöðurnar sýna góðan árangur Agoru við að styðja þróun, samþættingu og mat mannfjöldahermunarlíkana gagnvart raungögnum. Þróunarferlið er verulega þjálla með Agoru en með hefðbundnum aðferðum. Samþætting margra líkana tókst með blöndun hitakorta, möguleg hegðun var framkölluð og mat á líkönunum varð þægilegra með umgjörðinni. Ritgerðinni lýkur með því að fjalla um áhrif þessara niðurstaðna á svið mannfjöldahegðunar, með áherslu á nýstálegt framlag þessarar rannsóknar og mögulega framtíðarþróun Agora umgjarðarinnar

Improving image-based rendering fo r crowds and perceptual evaluation

We propose some improvements for two imaged-based impostor techniques. We have also developed a module of software integrated in a prototyping framework for crowd simulation. Finally, in order to analyze the visual quality of these techniques we have carried out a perceptual experiment

Crowd simulation and visualization

Large-scale simulation and visualization are essential topics in areas as different as sociology, physics, urbanism, training, entertainment among others. This kind of systems requires a vast computational power and memory resources commonly available in High Performance Computing HPC platforms. Currently, the most potent clusters have heterogeneous architectures with hundreds of thousands and even millions of cores. The industry trends inferred that exascale clusters would have thousands of millions. The technical challenges for simulation and visualization process in the exascale era are intertwined with difficulties in other areas of research, including storage, communication, programming models and hardware. For this reason, it is necessary prototyping, testing, and deployment a variety of approaches to address the technical challenges identified and evaluate the advantages and disadvantages of each proposed solution. The focus of this research is interactive large-scale crowd simulation and visualization. To exploit to the maximum the capacity of the current HPC infrastructure and be prepared to take advantage of the next generation. The project develops a new approach to scale crowd simulation and visualization on heterogeneous computing cluster using a task-based technique. Its main characteristic is hardware agnostic. It abstracts the difficulties that imply the use of heterogeneous architectures like memory management, scheduling, communications, and synchronization — facilitating development, maintenance, and scalability. With the goal of flexibility and take advantage of computing resources as best as possible, the project explores different configurations to connect the simulation with the visualization engine. This kind of system has an essential use in emergencies. Therefore, urban scenes were implemented as realistic as possible; in this way, users will be ready to face real events. Path planning for large-scale crowds is a challenge to solve, due to the inherent dynamism in the scenes and vast search space. A new path-finding algorithm was developed. It has a hierarchical approach which offers different advantages: it divides the search space reducing the problem complexity, it can obtain a partial path instead of wait for the complete one, which allows a character to start moving and compute the rest asynchronously. It can reprocess only a part if necessary with different levels of abstraction. A case study is presented for a crowd simulation in urban scenarios. Geolocated data are used, they were produced by mobile devices to predict individual and crowd behavior and detect abnormal situations in the presence of specific events. It was also address the challenge of combining all these individual’s location with a 3D rendering of the urban environment. The data processing and simulation approach are computationally expensive and time-critical, it relies thus on a hybrid Cloud-HPC architecture to produce an efficient solution. Within the project, new models of behavior based on data analytics were developed. It was developed the infrastructure to be able to consult various data sources such as social networks, government agencies or transport companies such as Uber. Every time there is more geolocation data available and better computation resources which allow performing analysis of greater depth, this lays the foundations to improve the simulation models of current crowds. The use of simulations and their visualization allows to observe and organize the crowds in real time. The analysis before, during and after daily mass events can reduce the risks and associated logistics costs.La simulación y visualización a gran escala son temas esenciales en áreas tan diferentes como la sociología, la física, el urbanismo, la capacitación, el entretenimiento, entre otros. Este tipo de sistemas requiere una gran capacidad de cómputo y recursos de memoria comúnmente disponibles en las plataformas de computo de alto rendimiento. Actualmente, los equipos más potentes tienen arquitecturas heterogéneas con cientos de miles e incluso millones de núcleos. Las tendencias de la industria infieren que los equipos en la era exascale tendran miles de millones. Los desafíos técnicos en el proceso de simulación y visualización en la era exascale se entrelazan con dificultades en otras áreas de investigación, incluidos almacenamiento, comunicación, modelos de programación y hardware. Por esta razón, es necesario crear prototipos, probar y desplegar una variedad de enfoques para abordar los desafíos técnicos identificados y evaluar las ventajas y desventajas de cada solución propuesta. El foco de esta investigación es la visualización y simulación interactiva de multitudes a gran escala. Aprovechar al máximo la capacidad de la infraestructura actual y estar preparado para aprovechar la próxima generación. El proyecto desarrolla un nuevo enfoque para escalar la simulación y visualización de multitudes en un clúster de computo heterogéneo utilizando una técnica basada en tareas. Su principal característica es que es hardware agnóstico. Abstrae las dificultades que implican el uso de arquitecturas heterogéneas como la administración de memoria, las comunicaciones y la sincronización, lo que facilita el desarrollo, el mantenimiento y la escalabilidad. Con el objetivo de flexibilizar y aprovechar los recursos informáticos lo mejor posible, el proyecto explora diferentes configuraciones para conectar la simulación con el motor de visualización. Este tipo de sistemas tienen un uso esencial en emergencias. Por lo tanto, se implementaron escenas urbanas lo más realistas posible, de esta manera los usuarios estarán listos para enfrentar eventos reales. La planificación de caminos para multitudes a gran escala es un desafío a resolver, debido al dinamismo inherente en las escenas y el vasto espacio de búsqueda. Se desarrolló un nuevo algoritmo de búsqueda de caminos. Tiene un enfoque jerárquico que ofrece diferentes ventajas: divide el espacio de búsqueda reduciendo la complejidad del problema, puede obtener una ruta parcial en lugar de esperar a la completa, lo que permite que un personaje comience a moverse y calcule el resto de forma asíncrona, puede reprocesar solo una parte si es necesario con diferentes niveles de abstracción. Se presenta un caso de estudio para una simulación de multitud en escenarios urbanos. Se utilizan datos geolocalizados producidos por dispositivos móviles para predecir el comportamiento individual y público y detectar situaciones anormales en presencia de eventos específicos. También se aborda el desafío de combinar la ubicación de todos estos individuos con una representación 3D del entorno urbano. Dentro del proyecto, se desarrollaron nuevos modelos de comportamiento basados ¿¿en el análisis de datos. Se creo la infraestructura para poder consultar varias fuentes de datos como redes sociales, agencias gubernamentales o empresas de transporte como Uber. Cada vez hay más datos de geolocalización disponibles y mejores recursos de cómputo que permiten realizar un análisis de mayor profundidad, esto sienta las bases para mejorar los modelos de simulación de las multitudes actuales. El uso de simulaciones y su visualización permite observar y organizar las multitudes en tiempo real. El análisis antes, durante y después de eventos multitudinarios diarios puede reducir los riesgos y los costos logísticos asociadosPostprint (published version

Geometric Collision Avoidance for Heterogeneous Crowd Simulation

Simulation of human crowds can create plausible human trajectories, predict likely flows of pedestrians, and has application in areas such as games, movies, safety planning, and virtual environments. This dissertation presents new crowd simulation methods based on geometric techniques. I will show how geometric optimization techniques can be used to efficiently compute collision-avoidance constraints, and use these constraints to generate human-like trajectories in simulated environments. This process of reacting to the nearby environment is known as local navigation and it forms the basis for many crowd simulation techniques, including those described in this dissertation. Given the importance of local navigation computations, I devote much of this dissertation to the derivation, analysis, and implementation of new local navigation techniques. I discuss how to efficiently exploit parallelization features available on modern processors, and show how an efficient parallel implementation allows simulations of hundreds of thousands of agents in real time on many-core processors and tens of thousands of agents on multi-core CPUs. I analyze the macroscopic flows which arise from these geometric collision avoidance techniques and compare them to flows seen in real human crowds, both qualitatively (in terms of flow patterns) and quantitatively (in terms of flow rates). Building on the basis of these strong local navigation models, I further develop many important extensions to the simulation framework. Firstly, I develop a model for global navigation which allows for more complex scenarios by accounting for long-term planning around large obstacles or emergent congestion. Secondly, I demonstrate methods for using data-driven approaches to improve crowd simulations. These include using real-world data to automatically tune parameters, and using perceptual user study data to introduce behavioral variation. Finally, looking beyond geometric avoidance based crowd simulation methods, I discuss methods for objectively evaluating different crowd simulation strategies using statistical measures. Specifically, I focus on the problem of quantifying how closely a simulation approach matches real-world data. I propose a similarity metric that can be applied to a wide variety of simulation approaches and datasets. Taken together, the methods presented in this dissertation enable simulations of large, complex humans crowds with a level of realism and efficiency not previously possible.Doctor of Philosoph

The design and simulation of traffic networks in virtual environments

For over half a century, researchers from a diverse set of disciplines have been studying the behaviour of traffic flow to better understand the causes of traffic congestion, accidents, and related phenomena. As the global population continues to rise, there is an increasing demand for more efficient and effective transportation infrastructures that are able to accommodate a greater number of civilians without compromising travel times, journey quality, cost, or accessibility. With recent advances in computing technology, transportation infrastructures are now typically developed using design and simulation packages that enable engineers to accurately model large-scale road networks and evaluate their designs through visual simulation. However, as these projects increase in scale and complexity, methodologies to intuitively design more complex and realistic simulations are highly desirable. The need of such technology translates across to the entertainment industry, where traffic simulations are integrated into computer games, television, film, and virtual tourism applications to enhance the realism and believability of the simulated scenario. In this thesis two significant challenges related to the design and simulation of traffic networks for use in virtual environments are presented. The first challenge is the development of intuitive techniques to assist the design and construction of high-fidelity three-dimensional road networks for use in both urban and rural virtual environments. The second challenge considers the implementation of computational models to accurately simulate the behaviour of drivers and pedestrians in transportation networks, in real time. An overview of the literature in the field is presented in this work with novel contributions relating to the challenges defined above

Digital Alchemy: Matter and Metamorphosis in Contemporary Digital Animation and Interface Design

The recent proliferation of special effects in Hollywood film has ushered in an era of digital transformation. Among scholars, digital technology is hailed as a revolutionary moment in the history of communication and representation. Nevertheless, media scholars and cultural historians have difficulty finding a language adequate to theorizing digital artifacts because they are not just texts to be deciphered. Rather, digital media artifacts also invite critiques about the status of reality because they resurrect ancient problems of embodiment and transcendence.In contrast to scholarly approaches to digital technology, computer engineers, interface designers, and special effects producers have invented a robust set of terms and phrases to describe the practice of digital animation. In order to address this disconnect between producers of new media and scholars of new media, I argue that the process of digital animation borrows extensively from a set of preexisting terms describing materiality that were prominent for centuries prior to the scientific revolution. Specifically, digital animators and interface designers make use of the ancient science, art, and technological craft of alchemy. Both alchemy and digital animation share several fundamental elements: both boast the power of being able to transform one material, substance, or thing into a different material, substance, or thing. Both seek to transcend the body and materiality but in the process, find that this elusive goal (realism and gold) is forever receding onto the horizon.The introduction begins with a literature review of the field of digital media studies. It identifies a gap in the field concerning disparate arguments about new media technology. On the one hand, scholars argue that new technologies like cyberspace and digital technology enable radical new forms of engagement with media on individual, social, and economic levels. At the same time that media scholars assert that our current epoch is marked by a historical rupture, many other researchers claim that new media are increasingly characterized by ancient metaphysical problems like embodiment and transcendence. In subsequent chapters I investigate this disparity

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion