Guiding Monte Carlo Tree Search simulations through Bayesian Opponent Modeling in The Octagon Theory

Os jogos de tabuleiro apresentam um problema de tomada de decisão desafiador na área da Inteligência Artificial. Embora abordagens clássicas baseadas em árvores de pesquisa tenham sido aplicadas com sucesso em diversos jogos de tabuleiro, como o Xadrez, estas mesmas abordagens ainda são limitadas pela tecnologia actual quando aplicadas a jogos de tabuleiro de maior omplexidade, como o Go. Face a isto, os jogos de maior complexidade só se tornaram no foco de pesquisa com o aparecimento de árvores de pesquisa baseadas em métodos de Monte Carlo (Monte Carlo Tree Search - MCTS), uma vez que começaram a surgir perspectivas de solução neste domínio.Este projecto de dissertação tem como objectivo expandir o estado de arte actual relativo a MCTS, através da investigação da integração de modelação de oponentes (Opponent Modeling) com MCTS. O propósito desta integração é guiar as simulações de um algoritmo típico de MCTS através da obtenção de conhecimento acerca do adversário, utilizando modelação de oponentes Bayesiana (Bayesian Opponent Modeling), com o intuito de reduzir o número de computações irrelevantes que são executadas em métodos puramente estocásticos e independentes de domínio. Para esta investigação, foi utilizado o jogo de tabuleiro deterministico The Octagon Theory, pois as suas regras, dimensão fixa do problema e configuração do tabuleiro apresentam não só um complexo desafio na criação de modelos de oponentes e na execução de MCTS em si, mas também um meio claro de classificação e comparação (benchmark) entre algoritmos. Através da análise de um estudo efectuado sobre a complexidade do jogo, acredita-se que o jogo, quando jogado na maior versão do tabuleiro, se encontra na mesma classe de complexidade do Shogi e da versão 19x19 do Go, transformando-se num jogo de tabuleiro adequado para investigação nesta área. Ao longo deste relatório, diversas políticas e melhoramentos relativos a MCTS são apresentados e comparados não só com a variação proposta, mas também com o método básico de Monte Carlo e com a melhor abordagem (greedy) conhecida no contexto do The Octagon Theory. Os resultados desta investigação revelam que a adição de Move Groups, Decisive Moves, Upper Confidence Bounds for Trees (UCT), Limited Simulation Lengths e Opponent Modeling transformam um agente MCTS previamente perdedor no melhor agente, num domínio com uma complexidade da árvore de jogo (game-tree complexity) estimada de 10^293, mesmo quando o orçamento computacional atribuído ao agente é mínimo.Board games present a very challenging problem in the decision-making topic of Artificial Intelligence. Although classical tree search approaches have been successful in various board games, such as Chess, these approaches are still very limited by modern technology when applied to higher complexity games such as Go. In light of this, it was not until the appearance of Monte Carlo Tree Search (MCTS) methods that higher complexity games became the main focus of research, as solution perspectives started to appear in this domain.This thesis builds on the current state-of-the-art in MCTS methods, by investigating the integration of Opponent Modeling with MCTS. The goal of this integration is to guide the simulations of the MCTS algorithm according to knowledge about the opponent, obtained in real-time through Bayesian Opponent Modeling, with the intention of reducing the number of irrelevant computations that are performed in purely stochastic, domain-independent methods. For this research, the two player deterministic board game The Octagon Theory was used, as its rules, fixed problem length and board configuration, present not only a difficult challenge for both the creation of opponent models and the execution of the MCTS method itself, but also a clear benchmark for comparison between algorithms. Through the analysis of a performed computation on the gametree complexity, the large board version of the game is believed to be in the same complexity class of Shogi and the 19x19 version of Go, turning it into a suitable board game for research in this area. Throughout this report, several MCTS policies and enhancements are presented and compared with not only the proposed variation, but also standard Monte Carlo search and the best known greedy approach for The Octagon Theory. The experiments reveal that a combination of Move Groups, Decisive Moves, Upper Confidence Bounds for Trees (UCT), Limited Simulation Lengths and an Opponent Modeling based simulation policy turn a former losing MCTS agent into the best performing one in a domain with estimated game-tree complexity of 10^293, even when the provided computational budget is kept low

Simulation approach to reliability analysis of WAMPAC system

© 2015 IEEE.Wide are monitoring, protection and control (WAMPAC) plays a critical role in smart grid development. Since WAMPAC frequently has the tasks of executing control and protection actions necessary for secure operation of power systems, its reliability is essential. This paper proposes a novel approach to the reliability analysis of WAMPAC systems. WAMPAC system functions are first divided into four subsystems: the measured inputs, the communication, the actuator and the analytic execution subsystems. The reliability indices of the subsystems are computed then using Monte Carlo approach. A sensitivity analysis is also described to illustrate the influence of different components on the system reliability

The heavy-quark hybrid meson spectrum in lattice QCD

Recent findings on the spectrum of heavy-quark mesons from computer simulations of quarks and gluons in lattice QCD are summarized, with particular attention to quark-antiquark states bound by an excited gluon field. The validity of a Born-Oppenheimer treatment for such systems is discussed. Recent results on glueball masses, the light-quark 1-+ hybrid meson mass, and the static three-quark potential are summarized.Comment: 15 pages, 13 figures, talk given at the Workshop on Scalar Mesons: An Interesting Puzzle for QCD, SUNY Institute of Technology, Utica, NY, May 16-18, 2003, submitted to American Institute of Physics Conference Proceedings. After publication, it will be found at http://proceedings.aip.org/proceedings

Investigating and Optimizing the Chiral Properties of Lattice Fermion Actions

We study exceptional modes of both the Wilson and the clover action in order to understand why quenched clover spectroscopy suffers so severely from exceptional configurations. We show that, in contrast to the case of the Wilson action, a large clover coefficient can make the exceptional modes extremely localized and thus very sensitive to short distance fluctuations. We describe a way to optimize the chiral behavior of Wilson-type lattice fermion actions by studying their low energy real eigenmodes. We find a candidate action, the clover action with fat links with a tuned clover term. We present a calculation of spectroscopy and matrix elements at Wilson gauge coupling beta=5.7. When compared to simulations with the standard (nonperturbatively improved) clover action at small lattice spacing, the action shows good scaling behavior, with an apparent great reduction in the number of exceptional configurations.Comment: 29 pages, LaTeX with 24 eps figures; Due to the suggestion of a referee, the previous version of this paper has been merged with hep-lat/9807002. Otherwise no major change in the contents of either pape

Exact sampling with Markov chains

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mathematics, 1996.Includes bibliographical references (p. 79-83).by David Bruce Wilson.Ph.D

Numerical Investigation of Exotic Phases in Quantum Lattice Models

In this thesis we present details of the design, development and application of a large scale exact diagonalisation code named DoQO (Diagonalisation of Quantum Observables). Among the features of this code are its ability to exploit physical symmetries and the fact that it has been designed to run in parallel to take advantage of modern high performance computing resources. The primary motivation for developing this code has been the investigation of exotic phases in quantum lattice models, and in particular of topological phases. A signicant portion of the thesis concerns the investigation of supersymmetric lattice models, which involves signicant use of the developed DoQO code. These are a relatively new (2003) family of models consisting of spinless fermions hopping on a lattice with the interactions tuned to a point where the spectrum exhibits supersymmetry. These models are extremely rich in the physics that they exhibit. Among the phases believed to exist in these models are critical, super-frustrated and super-topological phases. DoQO was also employed to investigate nite size eects in the Kitaev honeycomb lattice model. This is a spin model which exhibits both abelian and non abelian topological phases

Rooting out letters:octagonal symbol alphabets and algebraic number theory

It is widely expected that NMHV amplitudes in planar, maximally supersymmetric Yang-Mills theory require symbol letters that are not rationally expressible in terms of momentum-twistor (or cluster) variables starting at two loops for eight particles. Recent advances in loop integration technology have made this an `experimentally testable' hypothesis: compute the amplitude at some kinematic point, and see if algebraic symbol letters arise. We demonstrate the feasibility of such a test by directly integrating the most difficult of the two-loop topologies required. This integral, together with its rotated image, suffices to determine the simplest NMHV component amplitude: the unique component finite at this order. Although each of these integrals involve algebraic symbol alphabets, the combination contributing to this amplitude is---surprisingly---rational. We describe the steps involved in this analysis, which requires several novel tricks of loop integration and also a considerable degree of algebraic number theory. We find dramatic and unusual simplifications, in which the two symbols initially expressed as almost ten million terms in over two thousand letters combine in a form that can be written in five thousand terms and twenty-five letters.Comment: 25 pages, 4 figures; detailed results available as ancillary file