Deep learning for video game playing

In this article, we review recent Deep Learning advances in the context of how they have been applied to play different types of video games such as first-person shooters, arcade games, and real-time strategy games. We analyze the unique requirements that different game genres pose to a deep learning system and highlight important open challenges in the context of applying these machine learning methods to video games, such as general game playing, dealing with extremely large decision spaces and sparse rewards

COMBINED ARTIFICIAL INTELLIGENCE BEHAVIOUR SYSTEMS IN SERIOUS GAMING

This thesis proposes a novel methodology for creating Artificial Agents with semi-realistic behaviour, with such behaviour defined as overcoming common limitations of mainstream behaviour systems; rapidly switching between actions, ignoring “obvious” event priorities, etc. Behaviour in these Agents is not fully realistic as some limitations remain; Agents have a “perfect” knowledge about the surrounding environment, and an inability to transfer knowledge to other Agents (no communication). The novel methodology is achieved by hybridising existing Artificial Intelligence (AI) behaviour systems. In most artificial agents (Agents) behaviour is created using a single behaviour system, whereas this work combines several systems in a novel way to overcome the limitations of each. A further proposal is the separation of behavioural concerns into behaviour systems that are best suited to their needs, as well as describing a biologically inspired memory system that further aids in the production of semi-realistic behaviour. Current behaviour systems are often inherently limited, and in this work it is shown that by combining systems that are complementary to each other, these limitations can be overcome without the need for a workaround. This work examines in detail Belief Desire Intention systems, as well as Finite State Machines and explores how these methodologies can complement each other when combined appropriately. By combining these systems together a hybrid system is proposed that is both fast to react and simple to maintain by separating behaviours into fast-reaction (instinctual) and slow-reaction (behavioural) behaviours, and assigning these to the most appropriate system. Computational intelligence learning techniques such as Artificial Neural Networks have been intentionally avoided, as these techniques commonly present their data in a “black box” system, whereas this work aims to make knowledge explicitly available to the user. A biologically inspired memory system has further been proposed in order to generate additional behaviours in Artificial Agents, such as behaviour related to forgetfulness. This work explores how humans can quickly recall information while still being able to store millions of pieces of information, and how this can be achieved in an artificial system

Recognizing Teamwork Activity In Observations Of Embodied Agents

This thesis presents contributions to the theory and practice of team activity recognition. A particular focus of our work was to improve our ability to collect and label representative samples, thus making the team activity recognition more efficient. A second focus of our work is improving the robustness of the recognition process in the presence of noisy and distorted data. The main contributions of this thesis are as follows: We developed a software tool, the Teamwork Scenario Editor (TSE), for the acquisition, segmentation and labeling of teamwork data. Using the TSE we acquired a corpus of labeled team actions both from synthetic and real world sources. We developed an approach through which representations of idealized team actions can be acquired in form of Hidden Markov Models which are trained using a small set of representative examples segmented and labeled with the TSE. We developed set of team-oriented feature functions, which extract discrete features from the high-dimensional continuous data. The features were chosen such that they mimic the features used by humans when recognizing teamwork actions. We developed a technique to recognize the likely roles played by agents in teams even before the team action was recognized. Through experimental studies we show that the feature functions and role recognition module significantly increase the recognition accuracy, while allowing arbitrary shuffled inputs and noisy data

Proceedings of the SAB'06 Workshop on Adaptive Approaches for Optimizing Player Satisfaction in Computer and Physical Games

These proceedings contain the papers presented at the Workshop on Adaptive approaches for Optimizing Player Satisfaction in Computer and Physical Games held at the Ninth international conference on the Simulation of Adaptive Behavior (SAB’06): From Animals to Animats 9 in Rome, Italy on 1 October 2006. We were motivated by the current state-of-the-art in intelligent game design using adaptive approaches. Artificial Intelligence (AI) techniques are mainly focused on generating human-like and intelligent character behaviors. Meanwhile there is generally little further analysis of whether these behaviors contribute to the satisfaction of the player. The implicit hypothesis motivating this research is that intelligent opponent behaviors enable the player to gain more satisfaction from the game. This hypothesis may well be true; however, since no notion of entertainment or enjoyment is explicitly defined, there is therefore little evidence that a specific character behavior generates enjoyable games. Our objective for holding this workshop was to encourage the study, development, integration, and evaluation of adaptive methodologies based on richer forms of humanmachine interaction for augmenting gameplay experiences for the player. We wanted to encourage a dialogue among researchers in AI, human-computer interaction and psychology disciplines who investigate dissimilar methodologies for improving gameplay experiences. We expected that this workshop would yield an understanding of state-ofthe- art approaches for capturing and augmenting player satisfaction in interactive systems such as computer games. Our invited speaker was Hakon Steinø, Technical Producer of IO-Interactive, who discussed applied AI research at IO-Interactive, portrayed the future trends of AI in computer game industry and debated the use of academic-oriented methodologies for augmenting player satisfaction. The sessions of presentations and discussions where classified into three themes: Adaptive Learning, Examples of Adaptive Games and Player Modeling. The Workshop Committee did a great job in providing suggestions and informative reviews for the submissions; thank you! This workshop was in part supported by the Danish National Research Council (project no: 274-05-0511). Finally, thanks to all the participants; we hope you found this to be useful!peer-reviewe

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp

Complementary Layered Learning

Layered learning is a machine learning paradigm used to develop autonomous robotic-based agents by decomposing a complex task into simpler subtasks and learns each sequentially. Although the paradigm continues to have success in multiple domains, performance can be unexpectedly unsatisfactory. Using Boolean-logic problems and autonomous agent navigation, we show poor performance is due to the learner forgetting how to perform earlier learned subtasks too quickly (favoring plasticity) or having difficulty learning new things (favoring stability). We demonstrate that this imbalance can hinder learning so that task performance is no better than that of a suboptimal learning technique, monolithic learning, which does not use decomposition. Through the resulting analyses, we have identified factors that can lead to imbalance and their negative effects, providing a deeper understanding of stability and plasticity in decomposition-based approaches, such as layered learning. To combat the negative effects of the imbalance, a complementary learning system is applied to layered learning. The new technique augments the original learning approach with dual storage region policies to preserve useful information from being removed from an agent’s policy prematurely. Through multi-agent experiments, a 28% task performance increase is obtained with the proposed augmentations over the original technique

Self-tuning of game scenarios through self-adaptative multi-agent systems

Les jeux vidéo modernes deviennent de plus en plus complexes, tant par le nombre de règles qui les composent, que par le nombre d'entités artificielles qui y interagissent. D'un point de vue purement ludique, mais également en ayant des ambitions pédagogiques, les jeux doivent proposer aux joueurs des expériences qui correspondent à leurs niveaux de compétences et à leurs capacités. La diversité au sein de la population de joueurs rend difficile, voire impossible, de proposer une expérience qui aille à tout un chacun. Différents niveaux et différentes capacités de progression font que différents joueurs ont des besoins distincts. L'adaptation des jeux est proposée comme une solution pour palier ces difficultés. Cette thèse propose un ensemble de concepts afin que des concepteurs de jeux, ou des experts de différents domaines, puissent exprimer des objectifs pédagogiques ou ludiques, ainsi que des contraintes sur les expériences de jeu. La généricité de ces concepts les rend compatible avec une grande varieté d'application, potentiellement hors du domaine du jeu vidéo. Conjointement à ces concepts, nous proposons un système multi-agent conçu pour modifier dynamiquement les paramètres d'un moteur de jeu, afin que celui-ci satisfasse les objectifs définis par les experts ou les concepteurs. Le système est composé d'un ensemble d'agents autonomes, qui représentent les concepts du domaine. Ils n'ont qu'une vue locale de leur environnement et ne connaissent pas la fonction globale du système. Ils ne cherchent qu'a résoudre coopérativement les problème locaux qu'ils rencontrent. De l'organisation des agents émerge la fonctionnalité du système : l'adaptation de l'expérience de jeu menant à la satisfaction des objectifs ainsi qu'au respect des contraintes. Nous avons conduit plusieurs expériences pour démontrer que le système passe l'échelle, et qu'il est résistant au bruit. Le paradigme avec lequel les objectifs doivent être définis est utilisé dans des contextes variés pour démontrer sa généricité. D'autres applications démontrent que le système est capable d'adapter une expérience du joueur même quand les conditions de jeu évoluent significativement au cours du temps.Modern video games are getting more and more complex, by exhibiting more and more rules, as well as a growing number of co-existing artificial entities. Whether they only have entertainment objectives, or pedagogical ambitions, they need to provide a game experience that matches the skills and abilities of players. The diversity among the player population makes it difficult, if not impossible, to propose a single game that may suit everyone needs. Different skills, preferences, and progression abilities make players need different game experiences at different times. Adaptation of the game experience is advocated as a solution to keep it adequate. This thesis proposes a set a simple concepts in order for domain experts, games designers or others to express pedagogical or entertainment related objectives, as well as constraints on game experiences. By using only elementary concepts, such as measures and parameters, we remain compliant with a large diversity of domains, even outside of the field of video game. Along with the expression of game requirements, we propose a multi-agent system designed to dynamically modify the various parameters of a game engine, so that the game experience satisfies objectives expressed by experts or designers. The system is composed of a set of autonomous agents representing the domain concepts, that only have a local perception of their environment. They are not aware of the global function of the system, and they only seek to cooperatively solve their local problems. From the organization of these agents, the functionality of the system as a whole emerges: dynamic adaptation of a game experience to satisfy objectives and constraints. We conducted several experiments to demonstrate that the proposed system is scalable and noise resilient. The introduced paradigm with which the requirements must be expressed is used in various context to demonstrate its versatility. Other experiments demonstrate that this system is able to effectively adapt the game experience even when the conditions in which the game takes place significantly change over time

Life Long Learning In Sparse Learning Environments

Life long learning is a machine learning technique that deals with learning sequential tasks over time. It seeks to transfer knowledge from previous learning tasks to new learning tasks in order to increase generalization performance and learning speed. Real-time learning environments in which many agents are participating may provide learning opportunities but they are spread out in time and space outside of the geographical scope of a single learning agent. This research seeks to provide an algorithm and framework for life long learning among a network of agents in a sparse real-time learning environment. This work will utilize the robust knowledge representation of neural networks, and make use of both functional and representational knowledge transfer to accomplish this task. A new generative life long learning algorithm utilizing cascade correlation and reverberating pseudo-rehearsal and incorporating a method for merging divergent life long learning paths will be implemented