Game AI revisited

More than a decade after the early research efforts on the use of artificial intelligence (AI) in computer games and the establishment of a new AI domain the term “game AI” needs to be redefined. Traditionally, the tasks associated with game AI revolved around non player character (NPC) behavior at different levels of control, varying from navigation and pathfinding to decision making. Commercial-standard games developed over the last 15 years and current game productions, however, suggest that the traditional challenges of game AI have been well addressed via the use of sophisticated AI approaches, not necessarily following or inspired by advances in academic practices. The marginal penetration of traditional academic game AI methods in industrial productions has been mainly due to the lack of constructive communication between academia and industry in the early days of academic game AI, and the inability of academic game AI to propose methods that would significantly advance existing development processes or provide scalable solutions to real world problems. Recently, however, there has been a shift of research focus as the current plethora of AI uses in games is breaking the non-player character AI tradition. A number of those alternative AI uses have already shown a significant potential for the design of better games. This paper presents four key game AI research areas that are currently reshaping the research roadmap in the game AI field and evidently put the game AI term under a new perspective. These game AI flagship research areas include the computational modeling of player experience, the procedural generation of content, the mining of player data on massive-scale and the alternative AI research foci for enhancing NPC capabilities.peer-reviewe

Player Modeling

Player modeling is the study of computational models of players in games. This includes the detection, modeling, prediction and expression of human player characteristics which are manifested through cognitive, affective and behavioral patterns. This chapter introduces a holistic view of player modeling and provides a high level taxonomy and discussion of the key components of a player\u27s model. The discussion focuses on a taxonomy of approaches for constructing a player model, the available types of data for the model\u27s input and a proposed classification for the model\u27s output. The chapter provides also a brief overview of some promising applications and a discussion of the key challenges player modeling is currently facing which are linked to the input, the output and the computational model

Editorial

calls & calendarEDITORIA

Experience-driven procedural content generation (extended abstract)

Procedural content generation is an increasingly important area of technology within modern human-computer interaction with direct applications in digital games, the semantic web, and interface, media and software design. The personalization of experience via the modeling of the user, coupled with the appropriate adjustment of the content according to user needs and preferences are important steps towards effective and meaningful content generation. This paper introduces a framework for procedural content generation driven by computational models of user experience we name Experience-Driven Procedural Content Generation. While the framework is generic and applicable to various subareas of human computer interaction, we employ games as an indicative example of content-intensive software that enables rich forms of interaction.The research was supported, in part, by the FP7 ICT projects C2Learn (318480) and iLearnRW (318803).peer-reviewe

Bayesian opponent modeling in adversarial game environments.

This thesis investigates the use of Bayesian analysis upon an opponent¿s behaviour in order to determine the desired goals or strategy used by a given adversary. A terrain analysis approach utilising the A* algorithm is investigated, where a probability distribution between discrete behaviours of an opponent relative to a set of possible goals is generated. The Bayesian analysis of agent behaviour accurately determines the intended goal of an opponent agent, even when the opponent¿s actions are altered randomly. The environment of Poker is introduced and abstracted for ease of analysis. Bayes¿ theorem is used to generate an effective opponent model, categorizing behaviour according to its similarity with known styles of opponent. The accuracy of Bayes¿ rule yields a notable improvement in the performance of an agent once an opponent¿s style is understood. A hybrid of the Bayesian style predictor and a neuroevolutionary approach is shown to lead to effective dynamic play, in comparison to agents that do not use an opponent model. The use of recurrence in evolved networks is also shown to improve the performance and generalizability of an agent in a multiplayer environment. These strategies are then employed in the full-scale environment of Texas Hold¿em, where a betting round-based approach proves useful in determining and counteracting an opponent¿s play. It is shown that the use of opponent models, with the adaptive benefits of neuroevolution aid the performance of an agent, even when the behaviour of an opponent does not necessarily fit within the strict definitions of opponent ¿style¿.Engineering and Physical Sciences Research Council (EPSRC

Languages of games and play: A systematic mapping study

Digital games are a powerful means for creating enticing, beautiful, educational, and often highly addictive interactive experiences that impact the lives of billions of players worldwide. We explore what informs the design and construction of good games to learn how to speed-up game development. In particular, we study to what extent languages, notations, patterns, and tools, can offer experts theoretical foundations, systematic techniques, and practical solutions they need to raise their productivity and improve the quality of games and play. Despite the growing number of publications on this topic there is currently no overview describing the state-of-the-art that relates research areas, goals, and applications. As a result, efforts and successes are often one-off, lessons learned go overlooked, language reuse remains minimal, and opportunities for collaboration and synergy are lost. We present a systematic map that identifies relevant publications and gives an overview of research areas and publication venues. In addition, we categorize research perspectives along common objectives, techniques, and approaches, illustrated by summaries of selected languages. Finally, we distill challenges and opportunities for future research and development

Influence-based motion planning algorithms for games

In games, motion planning has to do with the motion of non-player characters (NPCs) from one place to another in the game world. In today’s video games there are two major approaches for motion planning, namely, path-finding and influence fields. Path-finding algorithms deal with the problem of finding a path in a weighted search graph, whose nodes represent locations of a game world, and in which the connections among nodes (edges) have an associated cost/weight. In video games, the most employed pathfinders are A* and its variants, namely, Dijkstra’s algorithm and best-first search. As further will be addressed in detail, the former pathfinders cannot simulate or mimic the natural movement of humans, which is usually without discontinuities, i.e., smooth, even when there are sudden changes in direction. Additionally, there is another problem with the former pathfinders, namely, their lack of adaptivity when changes to the environment occur. Therefore, such pathfinders are not adaptive, i.e., they cannot handle with search graph modifications during path search as a consequence of an event that happened in the game (e.g., when a bridge connecting two graph nodes is destroyed by a missile). On the other hand, influence fields are a motion planning technique that does not suffer from the two problems above, i.e., they can provide smooth human-like movement and are adaptive. As seen further ahead, we will resort to a differentiable real function to represent the influence field associated with a game map as a summation of functions equally differentiable, each associated to a repeller or an attractor. The differentiability ensures that there are no abrupt changes in the influence field, consequently, the movement of any NPC will be smooth, regardless if the NPC walks in the game world in the growing sense of the function or not. Thus, it is enough to have a spline curve that interpolates the path nodes to mimic the smooth human-like movement. Moreover, given the nature of the differentiable real functions that represent an influence field, the removal or addition of a repeller/attractor (as the result of the destruction or the construction of a bridge) does not alter the differentiability of the global function associated with the map of a game. That is to say that, an influence field is adaptive, in that it adapts to changes in the virtual world during the gameplay. In spite of being able to solve the two problems of pathfinders, an influence field may still have local extrema, which, if reached, will prevent an NPC from fleeing from that location. The local extremum problem never occurs in pathfinders because the goal node is the sole global minimum of the cost function. Therefore, by conjugating the cost function with the influence function, the NPC will never be detained at any local extremum of the influence function, because the minimization of the cost function ensures that it will always walk in the direction of the goal node. That is, the conjugation between pathfinders and influence fields results in movement planning algorithms which, simultaneously, solve the problems of pathfinders and influence fields. As will be demonstrated throughout this thesis, it is possible to combine influence fields and A*, Dijkstra’s, and best-first search algorithms, so that we get hybrid algorithms that are adaptive. Besides, these algorithms can generate smooth paths that resemble the ones traveled by human beings, though path smoothness is not the main focus of this thesis. Nevertheless, it is not always possible to perform this conjugation between influence fields and pathfinders; an example of such a pathfinder is the fringe search algorithm, as well as the new pathfinder which is proposed in this thesis, designated as best neighbor first search.Em jogos de vídeo, o planeamento de movimento tem que ver com o movimento de NPCs (“Non-Player Characters”, do inglês) de um lugar para outro do mundo virtual de um jogo. Existem duas abordagens principais para o planeamento de movimento, nomeadamente descoberta de caminhos e campos de influência. Os algoritmos de descoberta de caminhos lidam com o problema de encontrar um caminho num grafo de pesquisa pesado, cujos nós representam localizações de um mapa de um jogo, e cujas ligações (arestas) entre nós têm um custo/peso associado. Os algoritmos de descoberta de caminhos mais utilizados em jogos são o A* e as suas variantes, nomeadamente, o algoritmo de Dijkstra e o algoritmo de pesquisa do melhor primeiro (“best-first search”, do inglês). Como se verá mais adiante, os algoritmos de descoberta de caminhos referidos não permitem simular ou imitar o movimento natural dos seres humanos, que geralmente não possui descontinuidades, i.e., o movimento é suave mesmo quando há mudanças repentinas de direcção. A juntar a este problema, existe um outro que afeta os algoritmos de descoberta de caminhos acima referidos, que tem que ver com a falta de adaptatividade destes algoritmos face a alterações ao mapa de um jogo. Ou seja, estes algoritmos não são adaptativos, pelo que não permitem lidar com alterações ao grafo durante a pesquisa de um caminho em resultado de algum evento ocorrido no jogo (e.g., uma ponte que ligava dois nós de um grafo foi destruída por um míssil). Por outro lado, os campos de influência são uma técnica de planeamento de movimento que não padece dos dois problemas acima referidos, i.e., os campos possibilitam um movimento suave semelhante ao realizado pelo ser humano e são adaptativos. Como se verá mais adiante, iremos recorrer a uma função real diferenciável para representar o campo de influência associado a um mapa de um jogo como um somatório de funções igualmente diferenciáveis, em que cada função está associada a um repulsor ou a um atractor. A diferenciabilidade garante que não existem alterações abruptas ao campo de influência; consequentemente, o movimento de qualquer NPC será suave, independentemente de o NPC caminhar no mapa de um jogo no sentido crescente ou no sentido decrescente da função. Assim, basta ter uma curva spline que interpola os nós do caminho de forma a simular o movimento suave de um ser humano. Além disso, dada a natureza das funções reais diferenciáveis que representam um campo de influência, a remoção ou adição de um repulsor/atractor (como resultado da destruição ou construção de uma ponte) não altera a diferenciabilidade da função global associada ao mapa de um jogo. Ou seja, um campo de influência é adaptativo, na medida em que se adapta a alterações que ocorram num mundo virtual durante uma sessão de jogo. Apesar de ser capaz de resolver os dois problemas dos algoritmos de descoberta de caminhos, um campo de influência ainda pode ter extremos locais, que, se alcançados, impedirão um NPC de fugir desse local. O problema do extremo local nunca ocorre nos algoritmos de descoberta de caminhos porque o nó de destino é o único mínimo global da função de custo. Portanto, ao conjugar a função de custo com a função de influência, o NPC nunca será retido num qualquer extremo local da função de influência, porque a minimização da função de custo garante que ele caminhe sempre na direção do nó de destino. Ou seja, a conjugação entre algoritmos de descoberta de caminhos e campos de influência tem como resultado algoritmos de planeamento de movimento que resolvem em simultâneo os problemas dos algoritmos de descoberta de caminhos e de campos de influência. Como será demonstrado ao longo desta tese, é possível combinar campos de influência e o algoritmo A*, o algoritmo de Dijkstra, e o algoritmo da pesquisa pelo melhor primeiro, de modo a obter algoritmos híbridos que são adaptativos. Além disso, esses algoritmos podem gerar caminhos suaves que se assemelham aos que são efetuados por seres humanos, embora a suavidade de caminhos não seja o foco principal desta tese. No entanto, nem sempre é possível realizar essa conjugação entre os campos de influência e os algoritmos de descoberta de caminhos; um exemplo é o algoritmo de pesquisa na franja (“fringe search”, do inglês), bem como o novo algoritmo de pesquisa proposto nesta tese, que se designa por algoritmo de pesquisa pelo melhor vizinho primeiro (“best neighbor first search”, do inglês)

A Decentralized Partially Observable Markov Decision Model with Action Duration for Goal Recognition in Real Time Strategy Games

Multiagent goal recognition is a tough yet important problem in many real time strategy games or simulation systems. Traditional modeling methods either are in great demand of detailed agents’ domain knowledge and training dataset for policy estimation or lack clear definition of action duration. To solve the above problems, we propose a novel Dec-POMDM-T model, combining the classic Dec-POMDP, an observation model for recognizer, joint goal with its termination indicator, and time duration variables for actions with action termination variables. In this paper, a model-free algorithm named cooperative colearning based on Sarsa is used. Considering that Dec-POMDM-T usually encounters multiagent goal recognition problems with different sorts of noises, partially missing data, and unknown action durations, the paper exploits the SIS PF with resampling for inference under the dynamic Bayesian network structure of Dec-POMDM-T. In experiments, a modified predator-prey scenario is adopted to study multiagent joint goal recognition problem, which is the recognition of the joint target shared among cooperative predators. Experiment results show that (a) Dec-POMDM-T works effectively in multiagent goal recognition and adapts well to dynamic changing goals within agent group; (b) Dec-POMDM-T outperforms traditional Dec-MDP-based methods in terms of precision, recall, and F-measure