308 research outputs found
Biasing MCTS with Features for General Games
This paper proposes using a linear function approximator, rather than a deep
neural network (DNN), to bias a Monte Carlo tree search (MCTS) player for
general games. This is unlikely to match the potential raw playing strength of
DNNs, but has advantages in terms of generality, interpretability and resources
(time and hardware) required for training. Features describing local patterns
are used as inputs. The features are formulated in such a way that they are
easily interpretable and applicable to a wide range of general games, and might
encode simple local strategies. We gradually create new features during the
same self-play training process used to learn feature weights. We evaluate the
playing strength of an MCTS player biased by learnt features against a standard
upper confidence bounds for trees (UCT) player in multiple different board
games, and demonstrate significantly improved playing strength in the majority
of them after a small number of self-play training games.Comment: Accepted at IEEE CEC 2019, Special Session on Games. Copyright of
final version held by IEE
Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimates
In recent years, state-of-the-art game-playing agents often involve policies
that are trained in self-playing processes where Monte Carlo tree search (MCTS)
algorithms and trained policies iteratively improve each other. The strongest
results have been obtained when policies are trained to mimic the search
behaviour of MCTS by minimising a cross-entropy loss. Because MCTS, by design,
includes an element of exploration, policies trained in this manner are also
likely to exhibit a similar extent of exploration. In this paper, we are
interested in learning policies for a project with future goals including the
extraction of interpretable strategies, rather than state-of-the-art
game-playing performance. For these goals, we argue that such an extent of
exploration is undesirable, and we propose a novel objective function for
training policies that are not exploratory. We derive a policy gradient
expression for maximising this objective function, which can be estimated using
MCTS value estimates, rather than MCTS visit counts. We empirically evaluate
various properties of resulting policies, in a variety of board games.Comment: Accepted at the IEEE Conference on Games (CoG) 201
Ludii -- The Ludemic General Game System
While current General Game Playing (GGP) systems facilitate useful research
in Artificial Intelligence (AI) for game-playing, they are often somewhat
specialised and computationally inefficient. In this paper, we describe the
"ludemic" general game system Ludii, which has the potential to provide an
efficient tool for AI researchers as well as game designers, historians,
educators and practitioners in related fields. Ludii defines games as
structures of ludemes -- high-level, easily understandable game concepts --
which allows for concise and human-understandable game descriptions. We
formally describe Ludii and outline its main benefits: generality,
extensibility, understandability and efficiency. Experimentally, Ludii
outperforms one of the most efficient Game Description Language (GDL)
reasoners, based on a propositional network, in all games available in the
Tiltyard GGP repository. Moreover, Ludii is also competitive in terms of
performance with the more recently proposed Regular Boardgames (RBG) system,
and has various advantages in qualitative aspects such as generality.Comment: Accepted at ECAI 202
General Board Game Concepts
Many games often share common ideas or aspects between them, such as their
rules, controls, or playing area. However, in the context of General Game
Playing (GGP) for board games, this area remains under-explored. We propose to
formalise the notion of "game concept", inspired by terms generally used by
game players and designers. Through the Ludii General Game System, we describe
concepts for several levels of abstraction, such as the game itself, the moves
played, or the states reached. This new GGP feature associated with the ludeme
representation of games opens many new lines of research. The creation of a
hyper-agent selector, the transfer of AI learning between games, or explaining
AI techniques using game terms, can all be facilitated by the use of game
concepts. Other applications which can benefit from game concepts are also
discussed, such as the generation of plausible reconstructed rules for
incomplete ancient games, or the implementation of a board game recommender
system
- …