12,412 research outputs found
Monte Carlo Tree Search with Heuristic Evaluations using Implicit Minimax Backups
Monte Carlo Tree Search (MCTS) has improved the performance of game engines
in domains such as Go, Hex, and general game playing. MCTS has been shown to
outperform classic alpha-beta search in games where good heuristic evaluations
are difficult to obtain. In recent years, combining ideas from traditional
minimax search in MCTS has been shown to be advantageous in some domains, such
as Lines of Action, Amazons, and Breakthrough. In this paper, we propose a new
way to use heuristic evaluations to guide the MCTS search by storing the two
sources of information, estimated win rates and heuristic evaluations,
separately. Rather than using the heuristic evaluations to replace the
playouts, our technique backs them up implicitly during the MCTS simulations.
These minimax values are then used to guide future simulations. We show that
using implicit minimax backups leads to stronger play performance in Kalah,
Breakthrough, and Lines of Action.Comment: 24 pages, 7 figures, 9 tables, expanded version of paper presented at
IEEE Conference on Computational Intelligence and Games (CIG) 2014 conferenc
A Survey of Monte Carlo Tree Search Methods
Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work
An Efficient Interpolation Technique for Jump Proposals in Reversible-Jump Markov Chain Monte Carlo Calculations
Selection among alternative theoretical models given an observed data set is
an important challenge in many areas of physics and astronomy. Reversible-jump
Markov chain Monte Carlo (RJMCMC) is an extremely powerful technique for
performing Bayesian model selection, but it suffers from a fundamental
difficulty: it requires jumps between model parameter spaces, but cannot
efficiently explore both parameter spaces at once. Thus, a naive jump between
parameter spaces is unlikely to be accepted in the MCMC algorithm and
convergence is correspondingly slow. Here we demonstrate an interpolation
technique that uses samples from single-model MCMCs to propose inter-model
jumps from an approximation to the single-model posterior of the target
parameter space. The interpolation technique, based on a kD-tree data
structure, is adaptive and efficient in modest dimensionality. We show that our
technique leads to improved convergence over naive jumps in an RJMCMC, and
compare it to other proposals in the literature to improve the convergence of
RJMCMCs. We also demonstrate the use of the same interpolation technique as a
way to construct efficient "global" proposal distributions for single-model
MCMCs without prior knowledge of the structure of the posterior distribution,
and discuss improvements that permit the method to be used in
higher-dimensional spaces efficiently.Comment: Minor revision to match published versio
CP-violating top quark couplings at future linear colliders
We study the potential of future lepton colliders to probe violation of the
CP symmetry in the top quark sector. In certain extensions of the Standard
Model, such as the two-Higgs-doublet model (2HDM), sizeable anomalous top quark
dipole moments can arise, that may be revealed by a precise measurement of top
quark pair production. We present results from detailed Monte Carlo studies for
the ILC at 500~\GeV{} and CLIC at 380~\gev{} and use parton-level simulations
to explore the potential of high-energy operation. We find that precise
measurements in production with subsequent decay
to lepton plus jets final states can provide sufficient sensitivity to detect
Higgs-boson-induced CP violation in a viable two-Higgs-doublet model. The
potential of a linear collider to detect CP-violating electric and
weak dipole form factors of the top quark exceeds the prospects of the HL-LHC
by over an order of magnitude
Monte-Carlo tree search with heuristic knowledge: A novel way in solving capturing and life and death problems in Go
Monte-Carlo (MC) tree search is a new research field. Its effectiveness in searching large state spaces, such as the Go game tree, is well recognized in the computer Go community. Go domain- specific heuristics and techniques as well as domain-independent heuristics and techniques are sys- tematically investigated in the context of the MC tree search in this dissertation. The search extensions based on these heuristics and techniques can significantly improve the effectiveness and efficiency of the MC tree search.
Two major areas of investigation are addressed in this dissertation research: I. The identification and use of the effective heuristic knowledge in guiding the MC simulations, II. The extension of the MC tree search algorithm with heuristics. Go, the most challenging board game to the machine, serves as the test bed. The effectiveness of the MC tree search extensions is demonstrated through the performances of Go tactic problem solvers using these techniques.
The main contributions of this dissertation include:
1. A heuristics based Monte-Carlo tactic tree search framework is proposed to extend the standard
Monte-Carlo tree search.
2. (Go) Knowledge based heuristics are systematically investigated to improve the Monte-Carlo
tactic tree search.
3. Pattern learning is demonstrated as effective in improving the Monte-Carlo tactic tree search.
4. Domain knowledge independent tree search enhancements are shown as effective in improving
the Monte-Carlo tactic tree search performances.
5. A strong Go Tactic solver based on proposed algorithms outperforms traditional game tree
search algorithms.
The techniques developed in this dissertation research can benefit other game domains and ap-
plication fields
- …