Development of a Chess AI and an Interactive Learning Game

Chess is one of the most ancient games in humanity's history. It has been studied and explored for centuries, and the resulting knowledge has been stored in books of theory, still used today to teach chess apprentices. Since their introduction in the 20th century, chess algorithms have proved that chess is very computationally intensive at high-level play. However, thanks to the ever-increasing computational prowess of modern computers, as well as software-side optimizations, the best modern chess engines have long surpassed human capabilities. In 1997, the famous computer Deep Blue made history by defeating reigning world champion Garry Kasparov in a series of 6 games, after losing to the world champion the previous year. Being fascinated by the field of AI algorithms, and as a chess beginner myself, I set off to develop a game, called Easy Chess, as the Final Project of my Computer Engineering degree, that would help players hone their chess abilities and allow them to become better players. This article will go over the development of a chess engine within the computer game, by looking at my initial objectives, the methodology I followed to implement the algorithm, some development details, as well as the final results of the chess engine and the game's interface.Els escacs són un dels jocs més antics de la història de la humanitat. Han set estudiats i explorats durant segles, i el coneixement resultant ha sigut emmagatzemat en llibres de teoria, utilitzats encara avui dia per entrenar nous aprenents. Des de la seva introducció en el segle XX, els algorismes d'escacs han demostrat que és un joc molt exigent a escala computacional. Ara bé, gràcies a les constants millores pel que fa a la capacitat de còmput dels ordinadors actuals, i de les progressives optimitzacions quant a software, els millors motors d'escacs actuals són ja superiors als millors jugadors humans. L'any 1997, el famós ordinador Deep Blue va fer història en vèncer Garry Kasparov, el Campió del Món d'Escacs, en una sèrie de 6 partides, després d'haver perdut contra el campió l'any anterior. Degut a la meva fascinació pel món dels algorismes d'intel·ligència artificial i a la meva afició als escacs, vaig decidir desenvolupar un joc, anomenat Easy Chess, que permetés als usuaris millorar les seves habilitats i convertir-se en millors jugadors, en el marc del meu Treball Final de Grau pel grau d'Enginyeria Informàtica. Aquest article tracta sobre el desenvolupament del motor d'escacs i el joc 2D, a través de l'explicació dels objectius inicials, la metodologia seguida, detalls del desenvolupament i els resultats finals del projecte.El ajedrez es uno de los juegos más viejos de la historia de la humanidad. Ha sido estudiado y explorado durante siglos, y el conocimiento resultante ha sido almacenado en libros de teoría, utilizados aun hoy día para entrenar nuevos aprendientes. Desde su introducción en el siglo XX, los algoritmos de ajedrez han demostrado que es un juego muy exigente a escala computacional. Ahora bien, gracias a las constantes mejoras en la capacidad de cómputo de los ordenadores actuales, y de las progresivas optimizaciones a nivel de software, los mejores motores de ajedrez actuales son ya superiores a los mejores jugadores humanos. En el año 1997, el famoso ordenador Deep Blue hizo historia al vencer a Garry Kasparov, el Campeón del Mundo de Ajedrez, en una serie de 6 partidas, después de haber perdido ante el campeón el año anterior. Debido a mi fascinación por el mundo de los algoritmos de inteligencia artificial y como aficionado al ajedrez, decidí desarrollar un juego, llamado Easy Chess, el cual permitiera a los usuarios mejorar sus habilidades y convertirse en mejores jugadores, en el marco de mi Trabajo Final de Grado para el grado de Ingeniería Informática. Este artículo trata sobre el desarrollo del motor de ajedrez y el juego 2D, a través de la explicación de los objetivos iniciales, la metodología seguida, detalles del desarrollo y los resultados finales del proyecto

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Temporal Difference Learning in Complex Domains

PhDThis thesis adapts and improves on the methods of TD(k) (Sutton 1988) that were successfully used for backgammon (Tesauro 1994) and applies them to other complex games that are less amenable to simple pattem-matching approaches. The games investigated are chess and shogi, both of which (unlike backgammon) require significant amounts of computational effort to be expended on search in order to achieve expert play. The improved methods are also tested in a non-game domain. In the chess domain, the adapted TD(k) method is shown to successfully learn the relative values of the pieces, and matches using these learnt piece values indicate that they perform at least as well as piece values widely quoted in elementary chess books. The adapted TD(X) method is also shown to work well in shogi, considered by many researchers to be the next challenge for computer game-playing, and for which there is no standardised set of piece values. An original method to automatically set and adjust the major control parameters used by TD(k) is presented. The main performance advantage comes from the learning rate adjustment, which is based on a new concept called temporal coherence. Experiments in both chess and a random-walk domain show that the temporal coherence algorithm produces both faster learning and more stable values than both human-chosen parameters and an earlier method for learning rate adjustment. The methods presented in this thesis allow programs to learn with as little input of external knowledge as possible, exploring the domain on their own rather than by being taught. Further experiments show that the method is capable of handling many hundreds of weights, and that it is not necessary to perform deep searches during the leaming phase in order to learn effective weight

Temoral Difference Learning in Complex Domains

Submitted to the University of London for the Degree of Doctor of Philosophy in Computer Scienc

Machine Learning Techniques for the Development of a Stratego Bot

Stratego is a two-player, non-stochastic, imperfect-information strategy game in which players try to locate and capture the opponent\u27s flag. At the outset o f each game, players deploy their pieces in any arrangement they choose. Throughout play, each player knows the positions of the opponent’s pieces, but not the specific identities o f the opponent’s pieces. The game therefore involves deduction, bluffing, and a degree o f invention in addition to the sort o f planning familiar to perfect-information games like chess or backgammon. Developing a strong A.l. player presents three major challenges. Firstly, a Stratego program must maintain states o f belief about the opponent’s pieces as well as beliefs about the opponent’s beliefs. Beliefs must be updated according to in-game events. We propose to solve this using Bayesian probability theory and Bayesian networks. Secondly, any turn-based game-playing program must perform tree search as part o f its planning and move-making routine. Search in perfect-information games such as chess has been studied extensively and produced a wealth o f algorithms and heuristics to expedite the process. Stochastic and imperfect-information games, however, have received less general attention, though Schaeffer et al have made a significant effort to revisit this domain. Interestingly, the same family o f algorithms (Ballard’s Star-1 and Star-2) used in the stochastic perfect-information game of backgammon can be used in the deterministic, imperfect-information domain o f Stratego. The technical challenge here, just as in the stochastic domain, is to optimize node cutoffs. Thirdly, a strong Stratego program should have some degree o f inventiveness so that it can avoid predictable play. The game’s intricacy comes from information being concealed from the players. A program which plays too predictably (that is, according to known or obvious tactics) has a significant disadvantage against a more creative opponent. There is a balance, however, between tactics’ being novel and being foolish. Current, strong Stratego programs have been developed by human experts (such as Vincent deBoer), whose tactical preferences are hard-coded into those programs. Since we claim no especial talent for Stratego ourselves, part o f the development challenge will be to allow the program to discover tactical preferences and advantages on its own. Withholding explicitly programmed heuristics and allowing machines to discover tactics on their own has led to original and powerful computer play in the past (note Tesauro’s success with TD-Gammon). We hope our program will likewise learn to play competitively without depending on instruction from a mediocre or predictable player. Various techniques from machine learning, including both supervised and unsupervised learning, are applied to this objective. At our disposal are more than 50,000 match records from an online Stratego site. Part of developing a strong player will involve separating the truly advantageous features in these data from features which are merely frequent. The learning process must be objective enough to avoid bias and predictability, yet robust enough to exploit utility. We introduce a modeling method which allows partial instruction as guidelines for feature detection

Complexity, Heuristic, and Search Analysis for the Games of Crossings and Epaminondas

Games provide fertile research domains for algorithmic research. Often, game research helps solve real-world problems through the testing and refinement of search algorithms in game domains. Other times, game research finds limits for certain algorithms. For example, the game of Go proved intractable for the Min-Max with Alpha-Beta pruning algorithm leading to the popularity of Monte-Carlo based search algorithms. Although effective in Go, and game domains once ruled by Alpha-Beta such as Lines of Action, Monte-Carlo methods appear to have limits too as they fall short in tactical domains such as Hex and Chess. In a continuation of this type of research, two new games, Crossings and Epaminondas, are presented, analyzed and used to test two Monte-Carlo based algorithms: Upper Confidence Bounds applied to Trees (UCT) and Heuristic Guided UCT (HUCT). Results indicate that heuristic knowledge can positively affect UCT\u27s performance in the lower complexity domain of Crossings. However, both agents perform worse in the higher complexity domain of Epaminondas. This identifies Epaminondas as another domain that poses difficulties for Monte Carlo agents