217 research outputs found
Structured parallel programming for Monte Carlo Tree Search
The thesis is part of a bigger project, the HEPGAME (High Energy Physics Game). The main objective for HEPGAME is the utilization of AI solutions, particularly by using MCTS for simplification of HEP calculations. One of the issues is solving mathematical expressions of interest with millions of terms. These calculations can be solved with the FORM program, which is software for symbolic manipulation. Since these calculations are computationally intensive and take a large amount of time, the FORM program was parallelized to solve them in a reasonable amount of time.Therefore, any new algorithm based on MCTS, should also be parallelized. This requirement was behind the problem statement of the thesis: “How do we design a structured pattern-based parallel programming approach for efficient parallelism of MCTS for both multi-core and manycore shared-memory machines?”.To answer this question, the thesis approached the MCTS parallelization problem in three levels: (1) implementation level, (2) data structure level, and (3) algorithm level.In the implementation level, we proposed task-level parallelization over thread-level parallelization. Task-level parallelization provides us with efficient parallelism for MCTS to utilize cores on both multi-core and manycore machines.In the data structure level, we presented a lock-free data structure that guarantees the correctness. A lock-free data structure (1) removes the synchronization overhead when a parallel program needs many tasks to feed its cores and (2) improves both performance and scalability.In the algorithm level, we first explained how to use pipeline pattern for parallelization of MCTS to overcome search overhead. Then, through a step by step approach, we were able to propose and detail the structured parallel programming approach for Monte Carlo Tree Search.Algorithms and the Foundations of Software technolog
Scaling Monte Carlo Tree Search on Intel Xeon Phi
Many algorithms have been parallelized successfully on the Intel Xeon Phi
coprocessor, especially those with regular, balanced, and predictable data
access patterns and instruction flows. Irregular and unbalanced algorithms are
harder to parallelize efficiently. They are, for instance, present in
artificial intelligence search algorithms such as Monte Carlo Tree Search
(MCTS). In this paper we study the scaling behavior of MCTS, on a highly
optimized real-world application, on real hardware. The Intel Xeon Phi allows
shared memory scaling studies up to 61 cores and 244 hardware threads. We
compare work-stealing (Cilk Plus and TBB) and work-sharing (FIFO scheduling)
approaches. Interestingly, we find that a straightforward thread pool with a
work-sharing FIFO queue shows the best performance. A crucial element for this
high performance is the controlling of the grain size, an approach that we call
Grain Size Controlled Parallel MCTS. Our subsequent comparing with the Xeon
CPUs shows an even more comprehensible distinction in performance between
different threading libraries. We achieve, to the best of our knowledge, the
fastest implementation of a parallel MCTS on the 61 core Intel Xeon Phi using a
real application (47 relative to a sequential run).Comment: 8 pages, 9 figure
A Survey of Monte Carlo Tree Search Methods
Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work
Relational-Grid-World: A Novel Relational Reasoning Environment and An Agent Model for Relational Information Extraction
Reinforcement learning (RL) agents are often designed specifically for a
particular problem and they generally have uninterpretable working processes.
Statistical methods-based agent algorithms can be improved in terms of
generalizability and interpretability using symbolic Artificial Intelligence
(AI) tools such as logic programming. In this study, we present a model-free RL
architecture that is supported with explicit relational representations of the
environmental objects. For the first time, we use the PrediNet network
architecture in a dynamic decision-making problem rather than image-based
tasks, and Multi-Head Dot-Product Attention Network (MHDPA) as a baseline for
performance comparisons. We tested two networks in two environments ---i.e.,
the baseline Box-World environment and our novel environment,
Relational-Grid-World (RGW). With the procedurally generated RGW environment,
which is complex in terms of visual perceptions and combinatorial selections,
it is easy to measure the relational representation performance of the RL
agents. The experiments were carried out using different configurations of the
environment so that the presented module and the environment were compared with
the baselines. We reached similar policy optimization performance results with
the PrediNet architecture and MHDPA; additionally, we achieved to extract the
propositional representation explicitly ---which makes the agent's statistical
policy logic more interpretable and tractable. This flexibility in the agent's
policy provides convenience for designing non-task-specific agent
architectures. The main contributions of this study are two-fold ---an RL agent
that can explicitly perform relational reasoning, and a new environment that
measures the relational reasoning capabilities of RL agents
- …