4 research outputs found
Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithms
We investigate the impact of supervised prediction models on the strength and
efficiency of artificial agents that use the Monte-Carlo Tree Search (MCTS)
algorithm to play a popular video game Hearthstone: Heroes of Warcraft. We
overview our custom implementation of the MCTS that is well-suited for games
with partially hidden information and random effects. We also describe
experiments which we designed to quantify the performance of our Hearthstone
agent's decision making. We show that even simple neural networks can be
trained and successfully used for the evaluation of game states. Moreover, we
demonstrate that by providing a guidance to the game state search heuristic, it
is possible to substantially improve the win rate, and at the same time reduce
the required computations.Comment: Proceedings of the 2018 IEEE Conference on Computational Intelligence
and Games (CIG'18); pages 445-452; ISBN: 978-1-5386-4358-
Traditional Wisdom and Monte Carlo Tree Search Face-to-Face in the Card Game Scopone
We present the design of a competitive artificial intelligence for Scopone, a
popular Italian card game. We compare rule-based players using the most
established strategies (one for beginners and two for advanced players) against
players using Monte Carlo Tree Search (MCTS) and Information Set Monte Carlo
Tree Search (ISMCTS) with different reward functions and simulation strategies.
MCTS requires complete information about the game state and thus implements a
cheating player while ISMCTS can deal with incomplete information and thus
implements a fair player. Our results show that, as expected, the cheating MCTS
outperforms all the other strategies; ISMCTS is stronger than all the
rule-based players implementing well-known and most advanced strategies and it
also turns out to be a challenging opponent for human players.Comment: Preprint. Accepted for publication in the IEEE Transaction on Game
Neuroevolution of Self-Interpretable Agents
Inattentional blindness is the psychological phenomenon that causes one to
miss things in plain sight. It is a consequence of the selective attention in
perception that lets us remain focused on important parts of our world without
distraction from irrelevant details. Motivated by selective attention, we study
the properties of artificial agents that perceive the world through the lens of
a self-attention bottleneck. By constraining access to only a small fraction of
the visual input, we show that their policies are directly interpretable in
pixel space. We find neuroevolution ideal for training self-attention
architectures for vision-based reinforcement learning (RL) tasks, allowing us
to incorporate modules that can include discrete, non-differentiable operations
which are useful for our agent. We argue that self-attention has similar
properties as indirect encoding, in the sense that large implicit weight
matrices are generated from a small number of key-query parameters, thus
enabling our agent to solve challenging vision based tasks with at least 1000x
fewer parameters than existing methods. Since our agent attends to only task
critical visual hints, they are able to generalize to environments where task
irrelevant elements are modified while conventional methods fail. Videos of our
results and source code available at https://attentionagent.github.io/Comment: To appear at the Genetic and Evolutionary Computation Conference
(GECCO 2020) as a full pape