Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation

Monte Carlo tree search (MCTS) is extremely popular in computer Go which determines each action by enormous simulations in a broad and deep search tree. However, human experts select most actions by pattern analysis and careful evaluation rather than brute search of millions of future nteractions. In this paper, we propose a computer Go system that follows experts way of thinking and playing. Our system consists of two parts. The first part is a novel deep alternative neural network (DANN) used to generate candidates of next move. Compared with existing deep convolutional neural network (DCNN), DANN inserts recurrent layer after each convolutional layer and stacks them in an alternative manner. We show such setting can preserve more contexts of local features and its evolutions which are beneficial for move prediction. The second part is a long-term evaluation (LTE) module used to provide a reliable evaluation of candidates rather than a single probability from move predictor. This is consistent with human experts nature of playing since they can foresee tens of steps to give an accurate estimation of candidates. In our system, for each candidate, LTE calculates a cumulative reward after several future interactions when local variations are settled. Combining criteria from the two parts, our system determines the optimal choice of next move. For more comprehensive experiments, we introduce a new professional Go dataset (PGD), consisting of 253233 professional records. Experiments on GoGoD and PGD datasets show the DANN can substantially improve performance of move prediction over pure DCNN. When combining LTE, our system outperforms most relevant approaches and open engines based on MCTS.Comment: AAAI 201

Machine Learning in Go: Supervised Learning in Move Prediction

The oriental game of Go is increasingly recognized as the "grand challenge" of Artificial Intelligence (AI). So far, traditional AI approaches have resulted in programs that play at the level of a human amateur. Engineering Go knowledge into a Go playing program has proven to be a difficult task, a machine learning approach might therefore be successful. In this study, a supervised learning approach is used to learn to distinguish good moves from bad moves. This is done by training a neural network on a database of moves played by human players. The network's performance is measured on a prediction task. Three main research directions can be identified in this study. The first direction relates to the features used to encode a position in the game of Go. Specifically, an attempt is made to capture global information into a local area. The second research direction addresses the methodology of supervised learning. In order to gain some insight in the ability of a neural network to extract the knowledge used by human experts, both professional and human amateur games are used in the training process. Furthermore, games used in the training sets are decomposed to test whether knowledge obtained in a specific part of the game can be applied to the entire game. The last research direction is an attempt to uncover the relation between move prediction accuracy and playing strength. Results show that (1) capturing global information leads to a significantly higher prediction performance, (2) professional games do not necessarily provide a better base for achieving a high prediction score than amateur games, (3) knowledge obtained from one part of the game does not generalize over the entire game, and (4) no strong claims can be made regarding the relation between prediction accuracy and playing strength, at least for the program used in this study

Evaluating Go Game Records for Prediction of Player Attributes

We propose a way of extracting and aggregating per-move evaluations from sets of Go game records. The evaluations capture different aspects of the games such as played patterns or statistic of sente/gote sequences. Using machine learning algorithms, the evaluations can be utilized to predict different relevant target variables. We apply this methodology to predict the strength and playing style of the player (e.g. territoriality or aggressivity) with good accuracy. We propose a number of possible applications including aiding in Go study, seeding real-work ranks of internet players or tuning of Go-playing programs

A Dynamical Systems Approach for Static Evaluation in Go

In the paper arguments are given why the concept of static evaluation has the potential to be a useful extension to Monte Carlo tree search. A new concept of modeling static evaluation through a dynamical system is introduced and strengths and weaknesses are discussed. The general suitability of this approach is demonstrated.Comment: IEEE Transactions on Computational Intelligence and AI in Games, vol 3 (2011), no

Protein Structure Prediction Using Basin-Hopping

Associative memory Hamiltonian structure prediction potentials are not overly rugged, thereby suggesting their landscapes are like those of actual proteins. In the present contribution we show how basin-hopping global optimization can identify low-lying minima for the corresponding mildly frustrated energy landscapes. For small systems the basin-hopping algorithm succeeds in locating both lower minima and conformations closer to the experimental structure than does molecular dynamics with simulated annealing. For large systems the efficiency of basin-hopping decreases for our initial implementation, where the steps consist of random perturbations to the Cartesian coordinates. We implemented umbrella sampling using basin-hopping to further confirm when the global minima are reached. We have also improved the energy surface by employing bioinformatic techniques for reducing the roughness or variance of the energy surface. Finally, the basin-hopping calculations have guided improvements in the excluded volume of the Hamiltonian, producing better structures. These results suggest a novel and transferable optimization scheme for future energy function development