110 research outputs found
Learning macromanagement in starcraft from replays using deep learning
The real-time strategy game StarCraft has proven to be a challenging
environment for artificial intelligence techniques, and as a result, current
state-of-the-art solutions consist of numerous hand-crafted modules. In this
paper, we show how macromanagement decisions in StarCraft can be learned
directly from game replays using deep learning. Neural networks are trained on
789,571 state-action pairs extracted from 2,005 replays of highly skilled
players, achieving top-1 and top-3 error rates of 54.6% and 22.9% in predicting
the next build action. By integrating the trained network into UAlbertaBot, an
open source StarCraft bot, the system can significantly outperform the game's
built-in Terran bot, and play competitively against UAlbertaBot with a fixed
rush strategy. To our knowledge, this is the first time macromanagement tasks
are learned directly from replays in StarCraft. While the best hand-crafted
strategies are still the state-of-the-art, the deep network approach is able to
express a wide range of different strategies and thus improving the network's
performance further with deep reinforcement learning is an immediately
promising avenue for future research. Ultimately this approach could lead to
strong StarCraft bots that are less reliant on hard-coded strategies.Comment: 8 pages, to appear in the proceedings of the IEEE Conference on
Computational Intelligence and Games (CIG 2017
Deep learning for video game playing
In this article, we review recent Deep Learning advances in the context of
how they have been applied to play different types of video games such as
first-person shooters, arcade games, and real-time strategy games. We analyze
the unique requirements that different game genres pose to a deep learning
system and highlight important open challenges in the context of applying these
machine learning methods to video games, such as general game playing, dealing
with extremely large decision spaces and sparse rewards
Behavior Trees in Robotics and AI: An Introduction
A Behavior Tree (BT) is a way to structure the switching between different
tasks in an autonomous agent, such as a robot or a virtual entity in a computer
game. BTs are a very efficient way of creating complex systems that are both
modular and reactive. These properties are crucial in many applications, which
has led to the spread of BT from computer game programming to many branches of
AI and Robotics. In this book, we will first give an introduction to BTs, then
we describe how BTs relate to, and in many cases generalize, earlier switching
structures. These ideas are then used as a foundation for a set of efficient
and easy to use design principles. Properties such as safety, robustness, and
efficiency are important for an autonomous system, and we describe a set of
tools for formally analyzing these using a state space description of BTs. With
the new analysis tools, we can formalize the descriptions of how BTs generalize
earlier approaches. We also show the use of BTs in automated planning and
machine learning. Finally, we describe an extended set of tools to capture the
behavior of Stochastic BTs, where the outcomes of actions are described by
probabilities. These tools enable the computation of both success probabilities
and time to completion
Curriculum Generation and Sequencing for Deep Reinforcement Learning in StarCraft II
Reinforcement learning has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model’s ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution for reinforcement learning. Due to limitations involved with automating curriculum learning, curricula are usually manually designed. However, due to a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for reinforcement learning in real-time strategy game, StarCraft II. We propose three generalised methods of manually creating tasks for curriculum learning and verify their effectiveness through experiments. We also experiment with different curricula sequences, in addition to the most commonly used easy-to-hard order. Our results show that all three of our proposed methods can improve a reinforcement learning agent’s learning process when used correctly. We demonstrate that modifying the state space of the tasks is the most effective way to create training samples for StarCraft II and that reversed curricula can be beneficial to an agent’s convergence process under certain circumstances
- …