Artificial and Computational Intelligence in Games (Dagstuhl Seminar 12191)

This report documents the program and the outcomes of Dagstuhl Seminar 12191 "Artificial and Computational Intelligence in Games". The aim for the seminar was to bring together creative experts in an intensive meeting with the common goals of gaining a deeper understanding of various aspects of artificial and computational intelligence in games, to help identify the main challenges in game AI research and the most promising venues to deal with them. This was accomplished mainly by means of workgroups on 14 different topics (ranging from search, learning, and modeling to architectures, narratives, and evaluation), and plenary discussions on the results of the workgroups. This report presents the conclusions that each of the workgroups reached. We also added short descriptions of the few talks that were unrelated to any of the workgroups

Toward human-like pathfinding with hierarchical approaches and the GPS of the brain theory

Pathfinding for autonomous agents and robots has been traditionally driven by finding optimal paths. Where typically optimality means finding the shortest path between two points in a given environment. However, optimality may not always be strictly necessary. For example, in the case of video games, often computing the paths for non-player characters (NPC) must be done under strict time constraints to guarantee real time simulation. In those cases, performance is more important than finding the shortest path, specially because often a sub-optimal path can be just as convincing from the point of view of the player. When simulating virtual humanoids, pathfinding has also been used with the same goal: finding the shortest path. However, humans very rarely follow precise shortest paths, and thus there are other aspects of human decision making and path planning strategies that should be incorporated in current simulation models. In this thesis we first focus on improving performance optimallity to handle as many virtual agents as possible, and then introduce neuroscience research to propose pathfinding algorithms that attempt to mimic humans in a more realistic manner.In the case of simulating NPCs for video games, one of the main challenges is to compute paths as efficiently as possible for groups of agents. As both the size of the environments and the number of autonomous agents increase, it becomes harder to obtain results in real time under the constraints of memory and computing resources. For this purpose we explored hierarchical approaches for two reasons: (1) they have shown important performance improvements for regular grids and other abstract problems, and (2) humans tend to plan trajectories also following an topbottom abstraction, focusing first on high level location and then refining the path as they move between those high level locations. Therefore, we believe that hierarchical approaches combine the best of our two goals: improving performance for multi-agent pathfinding and achieving more human-like pathfinding. Hierarchical approaches, such as HNA* (Hierarchical A* for Navigation Meshes) can compute paths more efficiently, although only for certain configurations of the hierarchy. For other configurations, the method suffers from a bottleneck in the step that connects the Start and Goal positions with the hierarchy. This bottleneck can drop performance drastically.In this thesis we present different approaches to solve the HNA* bottleneck and thus obtain a performance boost for all hierarchical configurations. The first method relies on further memory storage, and the second one uses parallelism on the GPU. Our comparative evaluation shows that both approaches offer speed-ups as high as 9x faster than A*, and show no limitations based on hierarchical configuration. Then we further exploit the potential of CUDA parallelism, to extend our implementation to HNA* for multi-agent path finding. Our method can now compute paths for over 500K agents simultaneously in real-time, with speed-ups above 15x faster than a parallel multi-agent implementation using A*. We then focus on studying neurosience research to learn about the way that humans build mental maps, in order to propose novel algorithms that take those finding into account when simulating virtual humans. We propose a novel algorithm for path finding that is inspired by neuroscience research on how the brain learns and builds cognitive maps. Our method represents the space as a hexagonal grid, based on the GPS of the brain theory, and fires memory cells as counters. Our path finder then combines a method for exploring unknown environments while building such a cognitive map, with an A* search using a modified heuristic that takes into account the GPS of the brain cognitive map.El problema de Pathfinding para agentes autónomos o robots, ha consistido tradicionalmente en encontrar un camino óptimo, donde por óptimo se entiende el camino de distancia mínima entre dos posiciones de un entorno. Sin embargo, en ocasiones puede que no sea estrictamente necesario encontrar una solución óptima. Para ofrecer la simulación de multitudes de agentes autónomos moviéndose en tiempo real, es necesario calcular sus trayectorias bajo condiciones estrictas de tiempo de computación, pero no es fundamental que las soluciones sean las de distancia mínima ya que, con frecuencia, el observador no apreciará la diferencia entre un camino óptimo y un sub-óptimo. Por tanto, suele ser suficiente con que la solución encontrada sea visualmente creíble para el observado. Cuando se simulan humanoides virtuales en aplicaciones de realidad virtual que requieren avatares que simulen el comportamiento de los humanos, se tiende a emplear los mismos algoritmos que en video juegos, con el objetivo de encontrar caminos de distancia mínima. Pero si realmente queremos que los avatares imiten el comportamiento humano, tenemos que tener en cuenta que, en el mundo real, los humanos rara vez seguimos precisamente el camino más corto, y por tanto se deben considerar otros aspectos que influyen en la toma de decisiones de los humanos y la selección de rutas en el mundo real. En esta tesis nos centraremos primero en mejorar el rendimiento de la búsqueda de caminos para poder simular grandes números de humanoides virtuales autónomos, y a continuación introduciremos conceptos de investigación con mamíferos en neurociencia, para proponer soluciones al problema de pathfinding que intenten imitar con mayor realismo, el modo en el que los humanos navegan el entorno que les rodea. A medida que aumentan tanto el tamaño de los entornos virtuales como el número de agentes autónomos, resulta más difícil obtener soluciones en tiempo real, debido a las limitaciones de memoria y recursos informáticos. Para resolver este problema, en esta tesis exploramos enfoques jerárquicos porque consideramos que combinan dos objetivos fundamentales: mejorar el rendimiento en la búsqueda de caminos para multitudes de agentes y lograr una búsqueda de caminos similar a la de los humanos. El primer método presentado en esta tesis se basa en mejorar el rendimiento del algoritmo HNA* (Hierarchical A* for Navigation Meshes) incrementando almacenamiento de datos en memoria, y el segundo utiliza el paralelismo para mejorar drásticamente el rendimiento. La evaluación cuantitativa realizada en esta tesis, muestra que ambos enfoques ofrecen aceleraciones que pueden llegar a ser hasta 9 veces más rápidas que el algoritmo A* y no presentan limitaciones debidas a la configuración jerárquica. A continuación, aprovechamos aún más el potencial del paralelismo ofrecido por CUDA para extender nuestra implementación de HNA* a sistemas multi-agentes. Nuestro método permite calcular caminos simultáneamente y en tiempo real para más de 500.000 agentes, con una aceleración superior a 15 veces la obtenida por una implementación paralela del algoritmo A*. Por último, en esta tesis nos hemos centrado en estudiar los últimos avances realizados en el ámbito de la neurociencia, para comprender la manera en la que los humanos construyen mapas mentales y poder así proponer nuevos algoritmos que imiten de forma más real el modo en el que navegamos los humanos. Nuestro método representa el espacio como una red hexagonal, basada en la distribución de ¿place cells¿ existente en el cerebro, e imita las activaciones neuronales como contadores en dichas celdas. Nuestro buscador de rutas combina un método para explorar entornos desconocidos mientras construye un mapa cognitivo hexagonal, utilizando una búsqueda A* con una nueva heurística adaptada al mapa cognitivo del cerebro y sus contadores

GPU-based dynamic search on adaptive resolution grids

Abstract — This paper presents a GPU-based wave-front propagation technique for multi-agent path planning in ex-tremely large, complex, dynamic environments. Our work proposes an adaptive subdivision of the environment with efficient indexing, update, and neighbor-finding operations on the GPU to address several known limitations in prior work. In particular, an adaptive environment representation reduces the device memory requirements by an order of magnitude which enables for the first time, GPU-based goal path planning in truly large-scale environments (> 2048 m2) for hundreds of agents with different targets. We compare our approach to prior work that uses an uniform grid on several challenging navigation benchmarks and report significant memory savings, and up to a 1000X computational speedup. I

Best-first heuristic search for multicore machines

To harness modern multicore processors, it is imperative to develop parallel versions of fundamental algorithms. In this paper, we compare different approaches to parallel best-first search in a shared-memory setting. We present a new method, PBNF, that uses abstraction to partition the state space and to detect duplicate states without requiring frequent locking. PBNF allows speculative expansions when necessary to keep threads busy. We identify and fix potential livelock conditions in our approach, proving its correctness using temporal logic. Our approach is general, allowing it to extend easily to suboptimal and anytime heuristic search. In an empirical comparison on STRIPS planning, grid pathfinding, and sliding tile puzzle problems using 8-core machines, we show that A*, weighted A* and Anytime weighted A* implemented using PBNF yield faster search than improved versions of previous parallel search proposals

Planning under time pressure

Heuristic search is a technique used pervasively in artificial intelligence and automated planning. Often an agent is given a task that it would like to solve as quickly as possible. It must allocate its time between planning the actions to achieve the task and actually executing them. We call this problem planning under time pressure. Most popular heuristic search algorithms are ill-suited for this setting, as they either search a lot to find short plans or search a little and find long plans. The thesis of this dissertation is: when under time pressure, an automated agent should explicitly attempt to minimize the sum of planning and execution times, not just one or just the other. This dissertation makes four contributions. First we present new algorithms that use modern multi-core CPUs to decrease planning time without increasing execution. Second, we introduce a new model for predicting the performance of iterative-deepening search. The model is as accurate as previous offline techniques when using less training data, but can also be used online to reduce the overhead of iterative-deepening search, resulting in faster planning. Third we show offline planning algorithms that directly attempt to minimize the sum of planning and execution times. And, fourth we consider algorithms that plan online in parallel with execution. Both offline and online algorithms account for a user-specified preference between search and execution, and can greatly outperform the standard utility-oblivious techniques. By addressing the problem of planning under time pressure, these contributions demonstrate that heuristic search is no longer restricted to optimizing solution cost, obviating the need to choose between slow search times and expensive solutions

Influence-based motion planning algorithms for games

In games, motion planning has to do with the motion of non-player characters (NPCs) from one place to another in the game world. In today’s video games there are two major approaches for motion planning, namely, path-finding and influence fields. Path-finding algorithms deal with the problem of finding a path in a weighted search graph, whose nodes represent locations of a game world, and in which the connections among nodes (edges) have an associated cost/weight. In video games, the most employed pathfinders are A* and its variants, namely, Dijkstra’s algorithm and best-first search. As further will be addressed in detail, the former pathfinders cannot simulate or mimic the natural movement of humans, which is usually without discontinuities, i.e., smooth, even when there are sudden changes in direction. Additionally, there is another problem with the former pathfinders, namely, their lack of adaptivity when changes to the environment occur. Therefore, such pathfinders are not adaptive, i.e., they cannot handle with search graph modifications during path search as a consequence of an event that happened in the game (e.g., when a bridge connecting two graph nodes is destroyed by a missile). On the other hand, influence fields are a motion planning technique that does not suffer from the two problems above, i.e., they can provide smooth human-like movement and are adaptive. As seen further ahead, we will resort to a differentiable real function to represent the influence field associated with a game map as a summation of functions equally differentiable, each associated to a repeller or an attractor. The differentiability ensures that there are no abrupt changes in the influence field, consequently, the movement of any NPC will be smooth, regardless if the NPC walks in the game world in the growing sense of the function or not. Thus, it is enough to have a spline curve that interpolates the path nodes to mimic the smooth human-like movement. Moreover, given the nature of the differentiable real functions that represent an influence field, the removal or addition of a repeller/attractor (as the result of the destruction or the construction of a bridge) does not alter the differentiability of the global function associated with the map of a game. That is to say that, an influence field is adaptive, in that it adapts to changes in the virtual world during the gameplay. In spite of being able to solve the two problems of pathfinders, an influence field may still have local extrema, which, if reached, will prevent an NPC from fleeing from that location. The local extremum problem never occurs in pathfinders because the goal node is the sole global minimum of the cost function. Therefore, by conjugating the cost function with the influence function, the NPC will never be detained at any local extremum of the influence function, because the minimization of the cost function ensures that it will always walk in the direction of the goal node. That is, the conjugation between pathfinders and influence fields results in movement planning algorithms which, simultaneously, solve the problems of pathfinders and influence fields. As will be demonstrated throughout this thesis, it is possible to combine influence fields and A*, Dijkstra’s, and best-first search algorithms, so that we get hybrid algorithms that are adaptive. Besides, these algorithms can generate smooth paths that resemble the ones traveled by human beings, though path smoothness is not the main focus of this thesis. Nevertheless, it is not always possible to perform this conjugation between influence fields and pathfinders; an example of such a pathfinder is the fringe search algorithm, as well as the new pathfinder which is proposed in this thesis, designated as best neighbor first search.Em jogos de vídeo, o planeamento de movimento tem que ver com o movimento de NPCs (“Non-Player Characters”, do inglês) de um lugar para outro do mundo virtual de um jogo. Existem duas abordagens principais para o planeamento de movimento, nomeadamente descoberta de caminhos e campos de influência. Os algoritmos de descoberta de caminhos lidam com o problema de encontrar um caminho num grafo de pesquisa pesado, cujos nós representam localizações de um mapa de um jogo, e cujas ligações (arestas) entre nós têm um custo/peso associado. Os algoritmos de descoberta de caminhos mais utilizados em jogos são o A* e as suas variantes, nomeadamente, o algoritmo de Dijkstra e o algoritmo de pesquisa do melhor primeiro (“best-first search”, do inglês). Como se verá mais adiante, os algoritmos de descoberta de caminhos referidos não permitem simular ou imitar o movimento natural dos seres humanos, que geralmente não possui descontinuidades, i.e., o movimento é suave mesmo quando há mudanças repentinas de direcção. A juntar a este problema, existe um outro que afeta os algoritmos de descoberta de caminhos acima referidos, que tem que ver com a falta de adaptatividade destes algoritmos face a alterações ao mapa de um jogo. Ou seja, estes algoritmos não são adaptativos, pelo que não permitem lidar com alterações ao grafo durante a pesquisa de um caminho em resultado de algum evento ocorrido no jogo (e.g., uma ponte que ligava dois nós de um grafo foi destruída por um míssil). Por outro lado, os campos de influência são uma técnica de planeamento de movimento que não padece dos dois problemas acima referidos, i.e., os campos possibilitam um movimento suave semelhante ao realizado pelo ser humano e são adaptativos. Como se verá mais adiante, iremos recorrer a uma função real diferenciável para representar o campo de influência associado a um mapa de um jogo como um somatório de funções igualmente diferenciáveis, em que cada função está associada a um repulsor ou a um atractor. A diferenciabilidade garante que não existem alterações abruptas ao campo de influência; consequentemente, o movimento de qualquer NPC será suave, independentemente de o NPC caminhar no mapa de um jogo no sentido crescente ou no sentido decrescente da função. Assim, basta ter uma curva spline que interpola os nós do caminho de forma a simular o movimento suave de um ser humano. Além disso, dada a natureza das funções reais diferenciáveis que representam um campo de influência, a remoção ou adição de um repulsor/atractor (como resultado da destruição ou construção de uma ponte) não altera a diferenciabilidade da função global associada ao mapa de um jogo. Ou seja, um campo de influência é adaptativo, na medida em que se adapta a alterações que ocorram num mundo virtual durante uma sessão de jogo. Apesar de ser capaz de resolver os dois problemas dos algoritmos de descoberta de caminhos, um campo de influência ainda pode ter extremos locais, que, se alcançados, impedirão um NPC de fugir desse local. O problema do extremo local nunca ocorre nos algoritmos de descoberta de caminhos porque o nó de destino é o único mínimo global da função de custo. Portanto, ao conjugar a função de custo com a função de influência, o NPC nunca será retido num qualquer extremo local da função de influência, porque a minimização da função de custo garante que ele caminhe sempre na direção do nó de destino. Ou seja, a conjugação entre algoritmos de descoberta de caminhos e campos de influência tem como resultado algoritmos de planeamento de movimento que resolvem em simultâneo os problemas dos algoritmos de descoberta de caminhos e de campos de influência. Como será demonstrado ao longo desta tese, é possível combinar campos de influência e o algoritmo A*, o algoritmo de Dijkstra, e o algoritmo da pesquisa pelo melhor primeiro, de modo a obter algoritmos híbridos que são adaptativos. Além disso, esses algoritmos podem gerar caminhos suaves que se assemelham aos que são efetuados por seres humanos, embora a suavidade de caminhos não seja o foco principal desta tese. No entanto, nem sempre é possível realizar essa conjugação entre os campos de influência e os algoritmos de descoberta de caminhos; um exemplo é o algoritmo de pesquisa na franja (“fringe search”, do inglês), bem como o novo algoritmo de pesquisa proposto nesta tese, que se designa por algoritmo de pesquisa pelo melhor vizinho primeiro (“best neighbor first search”, do inglês)

The collaborative iterative search approach to multi agent path finding

PhD ThesisThis thesis presents a new approach to obtaining optimal and complete solutions to Multi Agent Path Finding (MAPF) problems called Collaborative Iterative Search (CIS). CIS employs a conflict based scheme inspired by the Conflict Based Search (CBS) algorithm and extends this to include a linear order lower level search. The structure of Planar Graphs is leveraged, permitting further optimization of the algorithm. This takes the form of reasoning-based culling of the search space, while maintaining optimality and completeness. Benchmarks provided demonstrate significant performance gains over the existing state of the art, particularly in the case of sparsely populated maps. The thesis draws to a conclusion with a summary of proposed future work

Pathfinding Algorithm Optimization Via Evolution

Pathfinding is a popular computer science problem in both academic research and industrial development. The objective of pathfinding is to search for a path, often the shortest path, from one location to another on a graph. Many real world applications can be considered as pathfinding problems, including motion planning, video games, logistics, and decision making. Computer scientists have proposed different algorithms to efficiently search for the shortest path. A* search algorithm is the de facto pathfinding algorithm that uses a heuristic function to determine the best action to take based on the given information. It is the most popular pathfinding algorithm due to its simplicity and efficiency. The performance of A* is heavily dependent on the quality of the heuristic function. The heuristic function determines the search speed, accuracy, and memory consumption. Hence, designing good heuristic functions for specific domains becomes the primary research focus on pathfinding algorithm optimization. In this dissertation, we address and solve several commonly known challenges in pathfinding problems and A* algorithm. First, designing new heuristic functions is a difficult and time-consuming task, especially when they are used to solve complex problems. The task requires the user to have expert knowledge of the problem. Moreover, a single heuristic function might not be enough to digest all the provided information and return the best guidance during the search. Previous works suggest that multiple heuristics for complex problems can dramatically speed up the search. However, choosing the appropriate combination of heuristic functions is tricky. Current optimization approaches rely on hand-tuning the parameters via trial and error by engineers over many iterations. There is a need to reduce the difficulty of designing heuristic functions for search performance maximization. Our first contribution is to propose an improved A* with a self-evolving heuristic function named Evolutionary Heuristic A* (EHA*) that reduces engineering effort to design the heuristic function for A* and maximize the search performance. Our experiment results show that EHA* (i) preserves path optimality; (ii) is not limited to a particular application; (iii) speeds up the path searching process; and (iv) most importantly, dramatically reduces the difficulty for software engineers to design heuristic functions for A* search. Moreover, our work can be applied to other existing works on the performance improvement of A* search. Search, A* search suffers from poor performance on large search spaces. Although EHA* improves the quality of heuristic functions, large search space still leads to many unnecessary searches. Our second contribution is Regions Discovery Algorithm (RDA), a map clustering technique to partition a grid based map into different categories to reduce search spaces and increase search speed. Our approach reduces the size of search spaces by partitioning a graph into many segments and identifying the segments by their characteristics. By identifying segments in different categories, we can easily eliminate search space, such as rooms, that are not possible (better use needed?) to be part of the optimal solution. Unlike the existing approaches that might result in non-optimal solutions, our experiment results show that RDA guarantees optimal solutions. Our third contribution, the Hierarchical Evolutionary Heuristic A* (HEHA*), further improves the search ability of handling complex pathfinding problems and boosting the search performance, by reducing search spaces and exploiting parallelism techniques. HEHA* combines the strength of EHA* and RDA to reduce search spaces and improve search speed. HEHA* shows that it provides better search performance with less memory consumption. In the pre-processing phase, first HEHA* partitions a graph into different segments and then applies different optimized heuristic functions for each segment to maximize the search performance. During the online process, HEHA* searches on the abstract level first to reduce search area, and exploits parallelism to speed up the search. Fourth, we improve and apply HEHA* to Multi-Agent Pathfinding (MAPF) problems. MAPF is the fundamental problem of many robotic and logistic applications, where the main constraint is that all agents can find the shortest paths while not colliding with each other. While the current trend favors the central controlled system, our approach is to develop a distributed version of HEHA* that can efficiently plan the optimal path for each agent. Such a system requires data sharing and exchanging among the agents, so that each agent can make its own decision without a supervising system. Our experiment results show that the Multi-Agent version of HEHA* maintains a high success rate when the number of agents increases. While EHA* and HEHA* provide a novel approach for heuristic function design, the pre-processing times are not trivial. To boost the performance of the preprocessing steps in EHA* and HEHA*, we propose a FPGA-based reconfigurable hardware accelerator that is not bound to any specific applications as our fifth contribution. Since GA requires many independent processes, it is suitable to implement it in a hardware accelerator to gain maximum performance. We apply the following techniques to enhance performance: deep pipelining, reconfigurable computing, massive parallel processing, and degree of parallelism maximization. Our results show that the FPGA accelerator for EHA* improves the scalability, throughput, and latency

Multi-agent algorithms with assignment strategy pursuing multiple moving targets in dynamic environments

Devising intelligent agents to successfully plan a path to a target is a common problem in artificial intelligence and in recent years, attention has increased to multi-agent pathfinding problems, especially due to the expansion in computer video games and robotics. Pathfinding for agents in real-world applications is a defined problem of multi-agent systems, where pursuing agents collaborate among themselves and autonomously plan their path to the targets. There are multi-agent algorithms that provide solutions with the shortest path without considering other pursuers and several of those use coordination. However, less attention has been paid to computing an assignment strategy for the pursuers and finding paths that collectively surround the targets. Comparatively fewer studies have been on target algorithms either. Besides, the multi-agent pathfinding problem becomes even more challenging if the goal destinations change over time. Existing solutions consider either a single target with moving capability or multiple targets that are stationary. The work presented in this thesis considers multiple moving targets in multi-agent systems. Therefore, the path planning problem for multiple pursuing agents requires more efficient pathfinding algorithms. In addition, when the target algorithms are improved for advanced behaviour with moving capabilities that smartly evade the pursuers makes the problem even harder. The research reported in this thesis aims to investigate multi-agent search algorithms to address the challenge associated with pursuing agents towards moving targets within a dynamically changing environment. In multi-agent scenarios, agents compute a path towards the target, while these target destinations in some cases are predefined in advance. Thus, this research proposes to investigate a solution to the path planning problem by utilising heuristic algorithms as well as assignment strategies for multiple pursuing agents. Furthermore, a state-of-the-art moving target algorithm, TrailMax, has been enhanced and implemented for multiple agent pathfinding problems, which aims to maximise the capture time if possible until timeout. The focus of this thesis is the investigation of the assignment strategy algorithms to coordinate multiple pursuing agents and explore pathfinding search algorithms to find a route towards moving targets. This will be achieved by dividing it into two stages. The first one is the coupled approach where the assignment strategy with a given criterion finds the optimal combination based on the current position of players. The second stage is the decoupled approach, where each agent independently finds its path towards the moving target. On the other hand, targets flee from pursuing agents using the specified escaping strategy. The novel contributions of the research presented in this thesis are summarised as follows: - A new algorithm is developed that uses existing assignment strategies, sum-of-costs and makespan, to assign targets, and then runs repetitive A* search until reaches the target. - An enhancement is provided for a state-of-the-art target algorithm that takes smart moves by avoiding capture from all pursuers. - To improve efficiency, six new approaches are investigated to find an optimal agent-to-target combination for target assignment. - A novel multi-agent algorithm is developed which uses cover heuristics to maximise its coverage to outmanoeuvre, trap and catch moving targets. The proposed pathfinding solutions and the results presented in this thesis demonstrate a significant contribution towards search algorithms in multi-agent systems