Learning The Game: Implementations Of Convolutional Networks In Automated Strategy Identification

Games can be used to represent a wide variety of real world problems, giving rise to many applications of game theory. Various computational methods have been proposed for identifying game strategies, including optimized tree search algorithms, game-specific heuristics, and artificial intelligence. In the last decade, systems like AlphaGo and AlphaZero have significantly exceeded the performance of the best human players in Chess, Go, and other games. The most effective game engines to date employ convolutional neural networks (CNNs) to evaluate game boards, extract features, and predict the optimal next move. These engines are trained on billions of simulated games, wherein the strategies become increasingly refined as more games are played. To explore the trade-offs inherent in developing CNNs, we will train them to play the game Connect-4, which is relatively small and has known optimal strategies. In this setting, we experiment with a variety of neural structures and other related factors with only a few hundred thousand simulated games. The results will allow us to compare how different aspects of the neural network impact performance. We propose a framework for this training process which generalizes to any two-player board games meeting some necessary criteria

Complexity, Heuristic, and Search Analysis for the Games of Crossings and Epaminondas

Games provide fertile research domains for algorithmic research. Often, game research helps solve real-world problems through the testing and refinement of search algorithms in game domains. Other times, game research finds limits for certain algorithms. For example, the game of Go proved intractable for the Min-Max with Alpha-Beta pruning algorithm leading to the popularity of Monte-Carlo based search algorithms. Although effective in Go, and game domains once ruled by Alpha-Beta such as Lines of Action, Monte-Carlo methods appear to have limits too as they fall short in tactical domains such as Hex and Chess. In a continuation of this type of research, two new games, Crossings and Epaminondas, are presented, analyzed and used to test two Monte-Carlo based algorithms: Upper Confidence Bounds applied to Trees (UCT) and Heuristic Guided UCT (HUCT). Results indicate that heuristic knowledge can positively affect UCT\u27s performance in the lower complexity domain of Crossings. However, both agents perform worse in the higher complexity domain of Epaminondas. This identifies Epaminondas as another domain that poses difficulties for Monte Carlo agents

Speculative parallelism in Intel Cilk Plus

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 37).Certain algorithms can be effectively parallelized at the cost of performing some redundant work. One example is searching an unordered tree graph for a particular node. Each subtree can be searched in parallel by a separate thread. Once a single thread is successful, however, the work of the others is unneeded and should be ended. This type of computation is known as speculative parallelism. Typically, an abort command is provided in the programming language to provide this functionality, but some languages do not. This thesis shows how support for the abort command can be provided as a user-level library. A parallel version of the alpha beta search algorithm demonstrates its effectivenesss.by Ruben Perez.M.Eng

Designing and Developing an Intelligent Congkak

Congkak is the nation's traditional game which could soon be forgotten if no serious attention is given to it, but literature survey has not yet found any research publication that mentioned the use of neural network algorithm (NN) on Congkak. Therefore the project want to try to rectify this issue by trying to develop an Intelligent Congkak System that also implemented NN and try answer research question such as this: “What is the best Congkak evaluation function for training NN for game playing?” and “Can Min-Max algorithm (MM) be speeded up by using NN as a forward-pruning method?”. This issues can solved by programming the Congkak system based on previous work on Mancala and NN system, and then recording the performance of the related algorithm. As a result: the project had created a Congkak system that had featured 3 Artificial Intelligence (AI) agent, and discovered that the combination of NN and MM is slower than MM alone

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Temoral Difference Learning in Complex Domains

Submitted to the University of London for the Degree of Doctor of Philosophy in Computer Scienc

Learning From Geometry In Learning For Tactical And Strategic Decision Domains

Artificial neural networks (ANNs) are an abstraction of the low-level architecture of biological brains that are often applied in general problem solving and function approximation. Neuroevolution (NE), i.e. the evolution of ANNs, has proven effective at solving problems in a variety of domains. Information from the domain is input to the ANN, which outputs its desired actions. This dissertation presents a new NE algorithm called Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT), based on a novel indirect encoding of ANNs. The key insight in HyperNEAT is to make the algorithm aware of the geometry in which the ANNs are embedded and thereby exploit such domain geometry to evolve ANNs more effectively. The dissertation focuses on applying HyperNEAT to tactical and strategic decision domains. These domains involve simultaneously considering short-term tactics while also balancing long-term strategies. Board games such as checkers and Go are canonical examples of such domains; however, they also include real-time strategy games and military scenarios. The dissertation details three proposed extensions to HyperNEAT designed to work in tactical and strategic decision domains. The first is an action selector ANN architecture that allows the ANN to indicate its judgements on every possible action all at once. The second technique is called substrate extrapolation. It allows learning basic concepts at a low resolution, and then increasing the resolution to learn more advanced concepts. The iii final extension is geometric game-tree pruning, whereby HyperNEAT can endow the ANN the ability to focus on specific areas of a domain (such as a checkers board) that deserve more inspection. The culminating contribution is to demonstrate the ability of HyperNEAT with these extensions to play Go, a most challenging game for artificial intelligence, by combining HyperNEAT with UC

Temporal Difference Learning in Complex Domains

PhDThis thesis adapts and improves on the methods of TD(k) (Sutton 1988) that were successfully used for backgammon (Tesauro 1994) and applies them to other complex games that are less amenable to simple pattem-matching approaches. The games investigated are chess and shogi, both of which (unlike backgammon) require significant amounts of computational effort to be expended on search in order to achieve expert play. The improved methods are also tested in a non-game domain. In the chess domain, the adapted TD(k) method is shown to successfully learn the relative values of the pieces, and matches using these learnt piece values indicate that they perform at least as well as piece values widely quoted in elementary chess books. The adapted TD(X) method is also shown to work well in shogi, considered by many researchers to be the next challenge for computer game-playing, and for which there is no standardised set of piece values. An original method to automatically set and adjust the major control parameters used by TD(k) is presented. The main performance advantage comes from the learning rate adjustment, which is based on a new concept called temporal coherence. Experiments in both chess and a random-walk domain show that the temporal coherence algorithm produces both faster learning and more stable values than both human-chosen parameters and an earlier method for learning rate adjustment. The methods presented in this thesis allow programs to learn with as little input of external knowledge as possible, exploring the domain on their own rather than by being taught. Further experiments show that the method is capable of handling many hundreds of weights, and that it is not necessary to perform deep searches during the leaming phase in order to learn effective weight