Video game Design and Development Degree Technical Report of the Final Degree Project

Treball final de Grau en Disseny i Desenvolupament de Videojocs. Codi: VJ1241. Curs acadèmic: 2017/2018This section is the technical proposal of my Final Degree Project in Video game Design and Development. The project consists of the development of a 3D Roguelike game using Unity3D. The main features are the design of a procedural dungeon generation system and the use of Machine Learning and Artificial Neural Networks for the NPCs (non-player characters) behavior. This artificial intelligence will be implemented with the new Machine Learning agents of Unity3D

Toward human-like pathfinding with hierarchical approaches and the GPS of the brain theory

Pathfinding for autonomous agents and robots has been traditionally driven by finding optimal paths. Where typically optimality means finding the shortest path between two points in a given environment. However, optimality may not always be strictly necessary. For example, in the case of video games, often computing the paths for non-player characters (NPC) must be done under strict time constraints to guarantee real time simulation. In those cases, performance is more important than finding the shortest path, specially because often a sub-optimal path can be just as convincing from the point of view of the player. When simulating virtual humanoids, pathfinding has also been used with the same goal: finding the shortest path. However, humans very rarely follow precise shortest paths, and thus there are other aspects of human decision making and path planning strategies that should be incorporated in current simulation models. In this thesis we first focus on improving performance optimallity to handle as many virtual agents as possible, and then introduce neuroscience research to propose pathfinding algorithms that attempt to mimic humans in a more realistic manner.In the case of simulating NPCs for video games, one of the main challenges is to compute paths as efficiently as possible for groups of agents. As both the size of the environments and the number of autonomous agents increase, it becomes harder to obtain results in real time under the constraints of memory and computing resources. For this purpose we explored hierarchical approaches for two reasons: (1) they have shown important performance improvements for regular grids and other abstract problems, and (2) humans tend to plan trajectories also following an topbottom abstraction, focusing first on high level location and then refining the path as they move between those high level locations. Therefore, we believe that hierarchical approaches combine the best of our two goals: improving performance for multi-agent pathfinding and achieving more human-like pathfinding. Hierarchical approaches, such as HNA* (Hierarchical A* for Navigation Meshes) can compute paths more efficiently, although only for certain configurations of the hierarchy. For other configurations, the method suffers from a bottleneck in the step that connects the Start and Goal positions with the hierarchy. This bottleneck can drop performance drastically.In this thesis we present different approaches to solve the HNA* bottleneck and thus obtain a performance boost for all hierarchical configurations. The first method relies on further memory storage, and the second one uses parallelism on the GPU. Our comparative evaluation shows that both approaches offer speed-ups as high as 9x faster than A*, and show no limitations based on hierarchical configuration. Then we further exploit the potential of CUDA parallelism, to extend our implementation to HNA* for multi-agent path finding. Our method can now compute paths for over 500K agents simultaneously in real-time, with speed-ups above 15x faster than a parallel multi-agent implementation using A*. We then focus on studying neurosience research to learn about the way that humans build mental maps, in order to propose novel algorithms that take those finding into account when simulating virtual humans. We propose a novel algorithm for path finding that is inspired by neuroscience research on how the brain learns and builds cognitive maps. Our method represents the space as a hexagonal grid, based on the GPS of the brain theory, and fires memory cells as counters. Our path finder then combines a method for exploring unknown environments while building such a cognitive map, with an A* search using a modified heuristic that takes into account the GPS of the brain cognitive map.El problema de Pathfinding para agentes autónomos o robots, ha consistido tradicionalmente en encontrar un camino óptimo, donde por óptimo se entiende el camino de distancia mínima entre dos posiciones de un entorno. Sin embargo, en ocasiones puede que no sea estrictamente necesario encontrar una solución óptima. Para ofrecer la simulación de multitudes de agentes autónomos moviéndose en tiempo real, es necesario calcular sus trayectorias bajo condiciones estrictas de tiempo de computación, pero no es fundamental que las soluciones sean las de distancia mínima ya que, con frecuencia, el observador no apreciará la diferencia entre un camino óptimo y un sub-óptimo. Por tanto, suele ser suficiente con que la solución encontrada sea visualmente creíble para el observado. Cuando se simulan humanoides virtuales en aplicaciones de realidad virtual que requieren avatares que simulen el comportamiento de los humanos, se tiende a emplear los mismos algoritmos que en video juegos, con el objetivo de encontrar caminos de distancia mínima. Pero si realmente queremos que los avatares imiten el comportamiento humano, tenemos que tener en cuenta que, en el mundo real, los humanos rara vez seguimos precisamente el camino más corto, y por tanto se deben considerar otros aspectos que influyen en la toma de decisiones de los humanos y la selección de rutas en el mundo real. En esta tesis nos centraremos primero en mejorar el rendimiento de la búsqueda de caminos para poder simular grandes números de humanoides virtuales autónomos, y a continuación introduciremos conceptos de investigación con mamíferos en neurociencia, para proponer soluciones al problema de pathfinding que intenten imitar con mayor realismo, el modo en el que los humanos navegan el entorno que les rodea. A medida que aumentan tanto el tamaño de los entornos virtuales como el número de agentes autónomos, resulta más difícil obtener soluciones en tiempo real, debido a las limitaciones de memoria y recursos informáticos. Para resolver este problema, en esta tesis exploramos enfoques jerárquicos porque consideramos que combinan dos objetivos fundamentales: mejorar el rendimiento en la búsqueda de caminos para multitudes de agentes y lograr una búsqueda de caminos similar a la de los humanos. El primer método presentado en esta tesis se basa en mejorar el rendimiento del algoritmo HNA* (Hierarchical A* for Navigation Meshes) incrementando almacenamiento de datos en memoria, y el segundo utiliza el paralelismo para mejorar drásticamente el rendimiento. La evaluación cuantitativa realizada en esta tesis, muestra que ambos enfoques ofrecen aceleraciones que pueden llegar a ser hasta 9 veces más rápidas que el algoritmo A* y no presentan limitaciones debidas a la configuración jerárquica. A continuación, aprovechamos aún más el potencial del paralelismo ofrecido por CUDA para extender nuestra implementación de HNA* a sistemas multi-agentes. Nuestro método permite calcular caminos simultáneamente y en tiempo real para más de 500.000 agentes, con una aceleración superior a 15 veces la obtenida por una implementación paralela del algoritmo A*. Por último, en esta tesis nos hemos centrado en estudiar los últimos avances realizados en el ámbito de la neurociencia, para comprender la manera en la que los humanos construyen mapas mentales y poder así proponer nuevos algoritmos que imiten de forma más real el modo en el que navegamos los humanos. Nuestro método representa el espacio como una red hexagonal, basada en la distribución de ¿place cells¿ existente en el cerebro, e imita las activaciones neuronales como contadores en dichas celdas. Nuestro buscador de rutas combina un método para explorar entornos desconocidos mientras construye un mapa cognitivo hexagonal, utilizando una búsqueda A* con una nueva heurística adaptada al mapa cognitivo del cerebro y sus contadores

Sketching for Real-time Control of Crowd Simulations

Controlling the behaviour of a crowd simulation typically involves tuning of a system's parameters through trial and error, a time-consuming process relying on knowledge of a potentially complex parameter set. Numerous graphical control approaches have been proposed to allow the user to interact with a simulation intuitively. This research investigates the use of a real-time sketch-based approach for crowd simulation control. This is done by modifying the environment of the simulation. Users can create entrances/exits, barriers and flow lines in real-time on top of an environment. This process requires a data structure to represent the environment and navigate the crowd through it. Two alternatives are presented: grid and navigation mesh. A detailed comparison shows that the navigation mesh is a more scalable approach since it uses less memory, has a similar pathfinding time, and is a better structure to represent the environment than the grid. The thesis also presents extensions to the sketch-based approach in the form of novel control tools, including storyboards to define the journey of the crowd, a timeline interface to simulate events through the day, and a sketch-based group storyboard to link behaviours and paths to be followed by a group. These tools are used to create two complex scenarios to exemplify possible applications of the sketch-based approach. The work on timelines also raises a new problem for an approach that dynamically modifies an environment in real-time which is 'when does the crowd know about the change?' Some initial solutions to how this should be handled are presented. The sketch-based system is evaluated by comparing it to a validated commercial system called MassMotion. The comparison takes into account the plausibility of the simulation and usability of the user interface. A user study is carried out to evaluate the graphical user interface of both systems. Formal evaluation methods are used to make the comparison: the benchmark suite 'steersuite', an adapted version of the Keystroke-Level Model (KLM) and the System Usability Scale (SUS). The results show that the sketch-based approach is faster and easier to use than MassMotion, but with fewer control options. An implementation of the sketching interface in a Virtual Reality environment is also considered. However, when compared to the desktop interface using a proposed adaptation to KLM for VR, the results show that sketching in a VR environment is slower and less accurate than the desktop version

PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav

We study ObjectGoal Navigation -- where a virtual robot situated in a new environment is asked to navigate to an object. Prior work has shown that imitation learning (IL) using behavior cloning (BC) on a dataset of human demonstrations achieves promising results. However, this has limitations -- 1) BC policies generalize poorly to new states, since the training mimics actions not their consequences, and 2) collecting demonstrations is expensive. On the other hand, reinforcement learning (RL) is trivially scalable, but requires careful reward engineering to achieve desirable behavior. We present PIRLNav, a two-stage learning scheme for BC pretraining on human demonstrations followed by RL-finetuning. This leads to a policy that achieves a success rate of $65.0\%$ on ObjectNav ($+5.0\%$ absolute over previous state-of-the-art). Using this BC$\rightarrow$RL training recipe, we present a rigorous empirical analysis of design choices. First, we investigate whether human demonstrations can be replaced with `free' (automatically generated) sources of demonstrations, e.g. shortest paths (SP) or task-agnostic frontier exploration (FE) trajectories. We find that BC$\rightarrow$RL on human demonstrations outperforms BC$\rightarrow$RL on SP and FE trajectories, even when controlled for same BC-pretraining success on train, and even on a subset of val episodes where BC-pretraining success favors the SP or FE policies. Next, we study how RL-finetuning performance scales with the size of the BC pretraining dataset. We find that as we increase the size of BC-pretraining dataset and get to high BC accuracies, improvements from RL-finetuning are smaller, and that $90\%$ of the performance of our best BC$\rightarrow$RL policy can be achieved with less than half the number of BC demonstrations. Finally, we analyze failure modes of our ObjectNav policies, and present guidelines for further improving them.Comment: 8 pages + supplemen

Development of a virtual reality puzzle video game

Treball final de Grau en Disseny i Desenvolupament de Videojocs. Codi: VJ1241. Curs acadèmic: 2019/2020This document presents the Degree’s Final Project in the Design and Development of Video Games Degree. The work to develop will consists of a virtual reality puzzle video game for the Oculus Quest lens. In this game the player plays as a figure toy in a child’s disorganized room. His objective is to throw a ball and make it reach the goal point. To do this, he builds blocks and other items to make a path for the ball. Care has been taken to avoid the most common causes of motion sickness with a resulting of a paused gameplay

Synthesizing play: exploring the use of artificial intelligence to evaluate game user experience

Digital games are a complex interactive medium providing a multitude of different experiences. The field of games user research (GUR) is dedicated to investigating and optimizing user experience in games. Such inquiries are of both commercial and academic importance, enhancing product quality and our understanding of human behaviour. A common GUR methodology is usertesting, where researchers gain insights from human users interacting with products. However, usertesting is expensive in terms of expert labour, time, and resource costs. To address these concerns, we developed PathOS, a free, open-source tool for game testing with AI agents. PathOS simulates player navigation in games using a basic model of human behaviour. We conducted an evaluation of PathOS with developers, finding that it provides valuable predictions of user behaviour in the iterative design process. Ultimately, we aim to give the game development community a useful and versatile augmentation to their testing processes

Make robots Be Bats: Specializing robotic swarms to the Bat algorithm

Bat algorithm is a powerful nature-inspired swarm intelligence method proposed by Prof. Xin-She Yang in 2010, with remarkable applications in industrial and scientific domains. However, to the best of authors' knowledge, this algorithm has never been applied so far in the context of swarm robotics. With the aim to fill this gap, this paper introduces the first practical implementation of the bat algorithm in swarm robotics. Our implementation is performed at two levels: a physical level, where we design and build a real robotic prototype; and a computational level, where we develop a robotic simulation framework. A very important feature of our implementation is its high specialization: all (physical and logical) components are fully optimized to replicate the most relevant features of the real microbats and the bat algorithm as faithfully as possible. Our implementation has been tested by its application to the problem of finding a target location within unknown static indoor 3D environments. Our experimental results show that the behavioral patterns observed in the real and the simulated robotic swarms are very similar. This makes our robotic swarm implementation an ideal tool to explore the potential and limitations of the bat algorithm for real-world practical applications and their computer simulations.This research has been kindly supported by the Computer Science National Program of the Spanish Research Agency (Agencia Estatal de Investigación) and European Funds, Project #TIN2017-89275-R (AEI/FEDER, UE), the project EVOLFORMAS Ref. #JU12, jointly supported by public body SODERCAN of the Regional Government of Cantabria and the European funds FEDER, the project PDE-GIR of the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Actions grant agreement #778035, Toho University (Funabashi, Japan), and the University of Cantabria (Santander, Spain). The authors are particularly grateful to the Department of Information Science of Toho University for all the facilities given to carry out this work. Special thanks are also due to the Editors and the three anonymous reviewers for their encouraging and constructive comments and very helpful feedback that allowed us to improve our paper signi cantly

Agora : unified framework for crowd simulation research

Crowd simulation focuses on modeling the movements and behaviors of large groups of people. This area of study has become increasingly important because of its several applications in various fields such as urban planning, safety, and entertainment. In each of these domains, the presence of virtual agents exhibiting realistic behavior greatly enhances the quality of the simulations. However, the inherently multifaceted and intricate nature of human behavior presents a unique challenge, necessitating the effective combination of multiple behavior models. This thesis introduces a novel theoretical framework for modeling human behavior in crowd simulations, addressing the unresolved issue of combining a plethora of behavior models, often developed in isolation. The proposed framework decomposes human behavior into fundamental driving stimuli, which are then represented graphically through the heatmap paradigm. Subsequently, the agent behavior is influenced by the heatmaps, which guide them toward attractive areas and steer them away from repulsive locations based on the encoded stimuli. A key advantage of this approach lies in the ability to combine heatmaps using well-defined color operations, effectively integrating different aspects of human behavior. Furthermore, the heatmap paradigm facilitates objective comparison of simulation output with real-world data, employing image similarity metrics to evaluate model accuracy. To realize this framework, the thesis presents a modular software architecture designed to support various tasks involved in crowd simulation, emphasizing the separation of concerns for each task. This architecture comprises a collection of abstract modules, which are subsequently implemented using appropriate software components to realize the underlying features, resulting in the Agora framework. To assess the ability of Agora to support the various tasks involved in crowd simulation, two case studies are implemented and analyzed. The first case study simulates tourists visiting Þingvellir national park in Iceland, examining how their behavior is influenced by the visibility of the surrounding environment. The second case study employs Agora to model the thermal and density comfort levels of virtual pedestrians in an urban setting. The results demonstrate that Agora successfully supports the development, combination, and evaluation of crowd simulation models against real-world data. The authoring process, assisted by Agora, is significantly more streamlined compared to its native counterpart. The integration of multiple models is achieved by combining the heatmaps, resulting in plausible behavior, and the model assessment is made convenient through the evaluator within the framework. The thesis concludes by discussing the implications of these findings for the field of crowd simulation, highlighting the contributions and potential future directions of the Agora framework.Mannfjöldahermun fæst við gerð líkana af hreyfingu og hegðun stórra hópa af fólki. Mikilvægi þessa rannsóknasviðs hefur vaxið stöðugt vegna hagnýtingar á margvíslegum vetvangi, eins og til dæmis á vetvangi borgarskipulags, öryggis og afþreyingar. Þegar sýndarmenni hegða sér á sannfærandi hátt, leiðir það til betri hermunar fyrir þessi notkunarsvið. En mannleg hegðun er í eðli sínu margbrotin og flókin og því er það sérstök áskorun við smíði sýndarmenna að sameina, með áhrifaríkum hætti, mörg mismunandi hegðunarlíkön. Þessi ritgerð kynnir nýja fræðilega umgjörð líkanasmíði mannlegrar hegðunar fyrir mannfjöldahermun, sem tekur á þeim óleysta vanda að sameina fjölda hegðunarlíkana, sem oft eru þróuð með aðskildum hætti. Umgjörðin brýtur mannlega hegðun niður í grundvallar drifáreiti, sem eru sett fram grafískt útfrá hugmyndafræði hitakorta. Sýndarmennin hegða sér síðan undir áhrifum frá hitakortunum, sem vísa þeim í áttina að aðlaðandi svæðum og stýra þeim burt frá fráhrindandi svæðum, útfrá hinu umritaða áreiti. Lykilkostur þessarar nálgunar er sá eiginleiki að geta blandað saman hitakortum með vel skilgreindum litaaðgerðum, sem eru þá í raun samþætting mismunandi hliða mannlegrar hegðunar. Hitakortshugmyndafræðin auðveldar ennfremur hlutlægan samanburð hermunarúttaks og raungagna með notkun myndsamanburðarmælinga, til að meta nákvæmni líkana. Varðandi útfærslu, þá kynnir þessi ritgerð einingadrifna hugbúnaðarhögun sem er hönnuð til að styðja við ýmsa ferla mannfjöldahermunar, með áherslu á aðskilnað helstu viðfangsefna hvers ferlis. Þessi högun inniheldur safn huglægra eininga, sem síðan eru útfærðar með viðeigandi hugbúnaðarhlutum, sem raungera undirliggjandi eiginleika. Útkoman er sjálf Agora umbjörðin. Tvö sýnidæmi eru útfærð og greind til að meta getu Agoru til að styðja við ýmis mannfjöldahermunarverkefni. Fyrra dæmið hermir eftir ferðamönnum sem heimsækja Þingvallaþjóðgarð, og skoðar hvernig hegðun þeirra verður fyrir áhrifum sýnileika umhverfisins sem umleikur þá. Seinna dæmið nýtir Agoru til að smíða líkan af hitauppstreymis- og þéttleikaþægindum hjá sýndarvegfarendum í borgarumhverfi. Niðurstöðurnar sýna góðan árangur Agoru við að styðja þróun, samþættingu og mat mannfjöldahermunarlíkana gagnvart raungögnum. Þróunarferlið er verulega þjálla með Agoru en með hefðbundnum aðferðum. Samþætting margra líkana tókst með blöndun hitakorta, möguleg hegðun var framkölluð og mat á líkönunum varð þægilegra með umgjörðinni. Ritgerðinni lýkur með því að fjalla um áhrif þessara niðurstaðna á svið mannfjöldahegðunar, með áherslu á nýstálegt framlag þessarar rannsóknar og mögulega framtíðarþróun Agora umgjarðarinnar