An Information Theoretic Analysis of Decision in Computer Chess

The basis of the method proposed in this article is the idea that information is one of the most important factors in strategic decisions, including decisions in computer chess and other strategy games. The model proposed in this article and the algorithm described are based on the idea of a information theoretic basis of decision in strategy games . The model generalizes and provides a mathematical justification for one of the most popular search algorithms used in leading computer chess programs, the fractional ply scheme. However, despite its success in leading computer chess applications, until now few has been published about this method. The article creates a fundamental basis for this method in the axioms of information theory, then derives the principles used in programming the search and describes mathematically the form of the coefficients. One of the most important parameters of the fractional ply search is derived from fundamental principles. Until now this coefficient has been usually handcrafted or determined from intuitive elements or data mining. There is a deep, information theoretical justification for such a parameter. In one way the method proposed is a generalization of previous methods. More important, it shows why the fractional depth ply scheme is so powerful. It is because the algorithm navigates along the lines where the highest information gain is possible. A working and original implementation has been written and tested for this algorithm and is provided in the appendix. The article is essentially self-contained and gives proper background knowledge and references. The assumptions are intuitive and in the direction expected and described intuitively by great champions of chess

Genetic Algorithms for Evolving Computer Chess Programs

This paper demonstrates the use of genetic algorithms for evolving: 1) a grandmaster-level evaluation function, and 2) a search mechanism for a chess program, the parameter values of which are initialized randomly. The evaluation function of the program is evolved by learning from databases of (human) grandmaster games. At first, the organisms are evolved to mimic the behavior of human grandmasters, and then these organisms are further improved upon by means of coevolution. The search mechanism is evolved by learning from tactical test suites. Our results show that the evolved program outperforms a two-time world computer chess champion and is at par with the other leading computer chess programs.Comment: Winner of Gold Award in 11th Annual "Humies" Awards for Human-Competitive Results. arXiv admin note: substantial text overlap with arXiv:1711.06840, arXiv:1711.06841, arXiv:1711.0683

Giraffe: Using Deep Reinforcement Learning to Play Chess

This report presents Giraffe, a chess engine that uses self-play to discover all its domain-specific knowledge, with minimal hand-crafted knowledge given by the programmer. Unlike previous attempts using machine learning only to perform parameter-tuning on hand-crafted evaluation functions, Giraffe's learning system also performs automatic feature extraction and pattern recognition. The trained evaluation function performs comparably to the evaluation functions of state-of-the-art chess engines - all of which containing thousands of lines of carefully hand-crafted pattern recognizers, tuned over many years by both computer chess experts and human chess masters. Giraffe is the most successful attempt thus far at using end-to-end machine learning to play chess.Comment: MSc Dissertatio

Simulating Human Grandmasters: Evolution and Coevolution of Evaluation Functions

This paper demonstrates the use of genetic algorithms for evolving a grandmaster-level evaluation function for a chess program. This is achieved by combining supervised and unsupervised learning. In the supervised learning phase the organisms are evolved to mimic the behavior of human grandmasters, and in the unsupervised learning phase these evolved organisms are further improved upon by means of coevolution. While past attempts succeeded in creating a grandmaster-level program by mimicking the behavior of existing computer chess programs, this paper presents the first successful attempt at evolving a state-of-the-art evaluation function by learning only from databases of games played by humans. Our results demonstrate that the evolved program outperforms a two-time World Computer Chess Champion.Comment: arXiv admin note: substantial text overlap with arXiv:1711.06839, arXiv:1711.0684

Expert-Driven Genetic Algorithms for Simulating Evaluation Functions

In this paper we demonstrate how genetic algorithms can be used to reverse engineer an evaluation function's parameters for computer chess. Our results show that using an appropriate expert (or mentor), we can evolve a program that is on par with top tournament-playing chess programs, outperforming a two-time World Computer Chess Champion. This performance gain is achieved by evolving a program that mimics the behavior of a superior expert. The resulting evaluation function of the evolved program consists of a much smaller number of parameters than the expert's. The extended experimental results provided in this paper include a report of our successful participation in the 2008 World Computer Chess Championship. In principle, our expert-driven approach could be used in a wide range of problems for which appropriate experts are available.Comment: arXiv admin note: substantial text overlap with arXiv:1711.06839, arXiv:1711.0684

Genetic Algorithms for Mentor-Assisted Evaluation Function Optimization

In this paper we demonstrate how genetic algorithms can be used to reverse engineer an evaluation function's parameters for computer chess. Our results show that using an appropriate mentor, we can evolve a program that is on par with top tournament-playing chess programs, outperforming a two-time World Computer Chess Champion. This performance gain is achieved by evolving a program with a smaller number of parameters in its evaluation function to mimic the behavior of a superior mentor which uses a more extensive evaluation function. In principle, our mentor-assisted approach could be used in a wide range of problems for which appropriate mentors are available.Comment: Winner of Best Paper Award in GECCO 2008. arXiv admin note: substantial text overlap with arXiv:1711.06840, arXiv:1711.0684

A More Human Way to Play Computer Chess

This paper suggests a forward-pruning technique for computer chess that uses 'Move Tables', which are like Transposition Tables, but for moves not positions. They use an efficient memory structure and has put the design into the context of long and short-term memories. The long-term memory updates a play path with weight reinforcement, while the short-term memory can be immediately added or removed. With this, 'long branches' can play a short path, before returning to a full search at the resulting leaf nodes. Re-using an earlier search path allows the tree to be forward-pruned, which is known to be dangerous, because it removes part of the search process. Additional checks are therefore made and moves can even be re-added when the search result is unsatisfactory. Automatic feature analysis is now central to the algorithm, where key squares and related squares can be generated automatically and used to guide the search process. Using this analysis, if a search result is inferior, it can re-insert un-played moves that cover these key squares only. On the tactical side, a type of move that the forward-pruning will fail on is recognised and a pattern-based solution to that problem is suggested. This has completed the theory of an earlier paper and resulted in a more human-like approach to searching for a chess move. Tests demonstrate that the obvious blunders associated with forward pruning are no longer present and that it can compete at the top level with regard to playing strength

Evolutionary Search in the Space of Rules for Creation of New Two-Player Board Games

Games have always been a popular test bed for artificial intelligence techniques. Game developers are always in constant search for techniques that can automatically create computer games minimizing the developer's task. In this work we present an evolutionary strategy based solution towards the automatic generation of two player board games. To guide the evolutionary process towards games, which are entertaining, we propose a set of metrics. These metrics are based upon different theories of entertainment in computer games. This work also compares the entertainment value of the evolved games with the existing popular board based games. Further to verify the entertainment value of the evolved games with the entertainment value of the human user a human user survey is conducted. In addition to the user survey we check the learnability of the evolved games using an artificial neural network based controller. The proposed metrics and the evolutionary process can be employed for generating new and entertaining board games, provided an initial search space is given to the evolutionary algorithm

Deep Pepper: Expert Iteration based Chess agent in the Reinforcement Learning Setting

An almost-perfect chess playing agent has been a long standing challenge in the field of Artificial Intelligence. Some of the recent advances demonstrate we are approaching that goal. In this project, we provide methods for faster training of self-play style algorithms, mathematical details of the algorithm used, various potential future directions, and discuss most of the relevant work in the area of computer chess. Deep Pepper uses embedded knowledge to accelerate the training of the chess engine over a "tabula rasa" system such as Alpha Zero. We also release our code to promote further research.Comment: Tabula Rasa, Chess engine, Learning Fast and Slow, Reinforcement Learning, Alpha Zer

Information and Search in Computer Chess

The article describes a model of chess based on information theory. A mathematical model of the partial depth scheme is outlined and a formula for the partial depth added for each ply is calculated from the principles of the model. An implementation of alpha-beta with partial depth is given. The method is tested using an experimental strategy having as objective to show the effect of allocation of a higher amount of search resources on areas of the search tree with higher information. The search proceeds in the direction of lines with higher information gain. The effects on search performance of allocating higher search resources on lines with higher information gain are tested experimentaly and conclusive results are obtained. In order to isolate the effects of the partial depth scheme no other heuristic is used