49 research outputs found
Deep learning for video game playing
In this article, we review recent Deep Learning advances in the context of
how they have been applied to play different types of video games such as
first-person shooters, arcade games, and real-time strategy games. We analyze
the unique requirements that different game genres pose to a deep learning
system and highlight important open challenges in the context of applying these
machine learning methods to video games, such as general game playing, dealing
with extremely large decision spaces and sparse rewards
Evolutionary Reinforcement Learning: A Survey
Reinforcement learning (RL) is a machine learning approach that trains agents
to maximize cumulative rewards through interactions with environments. The
integration of RL with deep learning has recently resulted in impressive
achievements in a wide range of challenging tasks, including board games,
arcade games, and robot control. Despite these successes, there remain several
crucial challenges, including brittle convergence properties caused by
sensitive hyperparameters, difficulties in temporal credit assignment with long
time horizons and sparse rewards, a lack of diverse exploration, especially in
continuous search space scenarios, difficulties in credit assignment in
multi-agent reinforcement learning, and conflicting objectives for rewards.
Evolutionary computation (EC), which maintains a population of learning agents,
has demonstrated promising performance in addressing these limitations. This
article presents a comprehensive survey of state-of-the-art methods for
integrating EC into RL, referred to as evolutionary reinforcement learning
(EvoRL). We categorize EvoRL methods according to key research fields in RL,
including hyperparameter optimization, policy search, exploration, reward
shaping, meta-RL, and multi-objective RL. We then discuss future research
directions in terms of efficient methods, benchmarks, and scalable platforms.
This survey serves as a resource for researchers and practitioners interested
in the field of EvoRL, highlighting the important challenges and opportunities
for future research. With the help of this survey, researchers and
practitioners can develop more efficient methods and tailored benchmarks for
EvoRL, further advancing this promising cross-disciplinary research field
Text-based Adventures of the Golovin AI Agent
The domain of text-based adventure games has been recently established as a
new challenge of creating the agent that is both able to understand natural
language, and acts intelligently in text-described environments.
In this paper, we present our approach to tackle the problem. Our agent,
named Golovin, takes advantage of the limited game domain. We use genre-related
corpora (including fantasy books and decompiled games) to create language
models suitable to this domain. Moreover, we embed mechanisms that allow us to
specify, and separately handle, important tasks as fighting opponents, managing
inventory, and navigating on the game map.
We validated usefulness of these mechanisms, measuring agent's performance on
the set of 50 interactive fiction games. Finally, we show that our agent plays
on a level comparable to the winner of the last year Text-Based Adventure AI
Competition
Non-determinism in the narrative structure of video games
PhD ThesisAt the present time, computer games represent a finite interactive system. Even in their more experimental forms, the number of possible interactions between player and NPCs (non-player characters) and among NPCs and the game world has a finite number and is led by a deterministic system in which events can therefore be predicted. This implies that the story itself, seen as the series of events that will unfold during gameplay, is a closed system that can be predicted a priori. This study looks beyond this limitation, and identifies the elements needed for the emergence of a non-finite, emergent narrative structure. Two major contributions are offered through this research. The first contribution comes in the form of a clear categorization of the narrative structures embracing all video game production since the inception of the medium. In order to look for ways to generate a non-deterministic narrative in games, it is necessary to first gain a clear understanding of the current narrative structures implemented and how their impact on users’ experiencing of the story. While many studies have observed the storytelling aspect, no attempt has been made to systematically distinguish among the different ways designers decide how stories are told in games. The second contribution is guided by the following research question: Is it possible to incorporate non-determinism into the narrative structure of computer games? The hypothesis offered is that non-determinism can be incorporated by means of nonlinear dynamical systems in general and Cellular Automata in particular
Accounting for the Neglected Dimensions of AI Progress
We analyze and reframe AI progress. In addition to the prevailing metrics of
performance, we highlight the usually neglected costs paid in the development
and deployment of a system, including: data, expert knowledge, human oversight,
software resources, computing cycles, hardware and network facilities,
development time, etc. These costs are paid throughout the life cycle of an AI
system, fall differentially on different individuals, and vary in magnitude
depending on the replicability and generality of the AI solution. The
multidimensional performance and cost space can be collapsed to a single
utility metric for a user with transitive and complete preferences. Even absent
a single utility function, AI advances can be generically assessed by whether
they expand the Pareto (optimal) surface. We explore a subset of these
neglected dimensions using the two case studies of Alpha* and ALE. This
broadened conception of progress in AI should lead to novel ways of measuring
success in AI, and can help set milestones for future progress
Evolutionary reinforcement learning for vision-based general video game playing.
Over the past decade, video games have become increasingly utilised for research in artificial intelligence. Perhaps the most extensive use of video games has been as benchmark problems in the field of reinforcement learning. Part of the reason for this is because video games are designed to challenge humans, and as a result, developing methods capable of mastering them is considered a stepping stone to achieving human-level per- formance in real-world tasks. Of particular interest are vision-based general video game playing (GVGP) methods. These are methods that learn from pixel inputs and can be applied, without modification, across sets of games. One of the challenges in evolutionary computing is scaling up neuroevolution methods, which have proven effective at solving simpler reinforcement learning problems in the past, to tasks with high- dimensional input spaces, such as video games. This thesis proposes a novel method for vision-based GVGP that combines the representational learning power of deep neural networks and the policy learning benefits of neuroevolution. This is achieved by separating state representation and policy learning and applying neuroevolution only to the latter. The method, AutoEncoder-augmented NeuroEvolution of Augmented Topologies (AE-NEAT), uses a deep autoencoder to learn compact state representations that are used as input for policy networks evolved using NEAT. Experiments on a selection of Atari games showed that this approach can successfully evolve high-performing agents and scale neuroevolution methods that evolve both weights and topology to do- mains with high-dimensional inputs. Overall, the experiments and results demonstrate a proof-of-concept of this separated state representation and policy learning approach and show that hybrid deep learning and neuroevolution-based GVGP methods are a promising avenue for future research
Microcosm
Microcosm explores the potential of responsive, evolving games through the lenses of play theory and cybernetics. It aims to provide an engaging play experience while supporting the exploration of dynamic networks. It is inspired by biological models of cell signalling and neural networks. Building on the framework of play theorist James Carse, microcosm is an attempt to create an infinite game that is played not to be won, but to keep all participants in play by continually shifting the relationships and boundaries that constitute the game. Microcosm is populated by virtual organisms that play with the boundaries between organic and artificial, component and whole, human and non-human