Investigating evolutionary checkers by incorporating individual and social learning, N-tuple systems and a round robin tournament

In recent years, much research attention has been paid to evolving self-learning game players. Fogel's Blondie24 is just one demonstration of a real success in this field and it has inspired many other scientists. In this thesis, artificial neural networks are employed to evolve game playing strategies for the game of checkers by introducing a league structure into the learning phase of a system based on Blondie24. We believe that this helps eliminate some of the randomness in the evolution. The best player obtained is tested against an evolutionary checkers program based on Blondie24. The results obtained are promising. In addition, we introduce an individual and social learning mechanism into the learning phase of the evolutionary checkers system. The best player obtained is tested against an implementation of an evolutionary checkers program, and also against a player, which utilises a round robin tournament. The results are promising. N-tuple systems are also investigated and are used as position value functions for the game of checkers. The architecture of the n-tuple is utilises temporal difference learning. The best player obtained is compared with an implementation of evolutionary checkers program based on Blondie24, and also against a Blondie24 inspired player, which utilises a round robin tournament. The results are promising. We also address the question of whether piece difference and the look-ahead depth are important factors in the Blondie24 architecture. Our experiments show that piece difference and the look-ahead depth have a significant effect on learning abilities

Investigating evolutionary checkers by incorporating individual and social learning, N-tuple systems and a round robin tournament

In recent years, much research attention has been paid to evolving self-learning game players. Fogel's Blondie24 is just one demonstration of a real success in this field and it has inspired many other scientists. In this thesis, artificial neural networks are employed to evolve game playing strategies for the game of checkers by introducing a league structure into the learning phase of a system based on Blondie24. We believe that this helps eliminate some of the randomness in the evolution. The best player obtained is tested against an evolutionary checkers program based on Blondie24. The results obtained are promising. In addition, we introduce an individual and social learning mechanism into the learning phase of the evolutionary checkers system. The best player obtained is tested against an implementation of an evolutionary checkers program, and also against a player, which utilises a round robin tournament. The results are promising. N-tuple systems are also investigated and are used as position value functions for the game of checkers. The architecture of the n-tuple is utilises temporal difference learning. The best player obtained is compared with an implementation of evolutionary checkers program based on Blondie24, and also against a Blondie24 inspired player, which utilises a round robin tournament. The results are promising. We also address the question of whether piece difference and the look-ahead depth are important factors in the Blondie24 architecture. Our experiments show that piece difference and the look-ahead depth have a significant effect on learning abilities

Coevolutionary optimization of fuzzy logic intelligence for strategic decision support

©2005 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.We present a description and initial results of a computer code that coevolves fuzzy logic rules to play a two-sided zero-sum competitive game. It is based on the TEMPO Military Planning Game that has been used to teach resource allocation to over 20 000 students over the past 40 years. No feasible algorithm for optimal play is known. The coevolved rules, when pitted against human players, usually win the first few competitions. For reasons not yet understood, the evolved rules (found in a symmetrical competition) place little value on information concerning the play of the opponent.Rodney W. Johnson, Michael E. Melich, Zbigniew Michalewicz, and Martin Schmid

Learning From Geometry In Learning For Tactical And Strategic Decision Domains

Artificial neural networks (ANNs) are an abstraction of the low-level architecture of biological brains that are often applied in general problem solving and function approximation. Neuroevolution (NE), i.e. the evolution of ANNs, has proven effective at solving problems in a variety of domains. Information from the domain is input to the ANN, which outputs its desired actions. This dissertation presents a new NE algorithm called Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT), based on a novel indirect encoding of ANNs. The key insight in HyperNEAT is to make the algorithm aware of the geometry in which the ANNs are embedded and thereby exploit such domain geometry to evolve ANNs more effectively. The dissertation focuses on applying HyperNEAT to tactical and strategic decision domains. These domains involve simultaneously considering short-term tactics while also balancing long-term strategies. Board games such as checkers and Go are canonical examples of such domains; however, they also include real-time strategy games and military scenarios. The dissertation details three proposed extensions to HyperNEAT designed to work in tactical and strategic decision domains. The first is an action selector ANN architecture that allows the ANN to indicate its judgements on every possible action all at once. The second technique is called substrate extrapolation. It allows learning basic concepts at a low resolution, and then increasing the resolution to learn more advanced concepts. The iii final extension is geometric game-tree pruning, whereby HyperNEAT can endow the ANN the ability to focus on specific areas of a domain (such as a checkers board) that deserve more inspection. The culminating contribution is to demonstrate the ability of HyperNEAT with these extensions to play Go, a most challenging game for artificial intelligence, by combining HyperNEAT with UC

PSO-based coevolutionary Game Learning

Games have been investigated as computationally complex problems since the inception of artificial intelligence in the 1950’s. Originally, search-based techniques were applied to create a competent (and sometimes even expert) game player. The search-based techniques, such as game trees, made use of human-defined knowledge to evaluate the current game state and recommend the best move to make next. Recent research has shown that neural networks can be evolved as game state evaluators, thereby removing the human intelligence factor completely. This study builds on the initial research that made use of evolutionary programming to evolve neural networks in the game learning domain. Particle Swarm Optimisation (PSO) is applied inside a coevolutionary training environment to evolve the weights of the neural network. The training technique is applied to both the zero sum and non-zero sum game domains, with specific application to Tic-Tac-Toe, Checkers and the Iterated Prisoners Dilemma (IPD). The influence of the various PSO parameters on playing performance are experimentally examined, and the overall performance of three different neighbourhood information sharing structures compared. A new coevolutionary scoring scheme and particle dispersement operator are defined, inspired by Formula One Grand Prix racing. Finally, the PSO is applied in three novel ways to evolve strategies for the IPD – the first application of its kind in the PSO field. The PSO-based coevolutionary learning technique described and examined in this study shows promise in evolving intelligent evaluators for the aforementioned games, and further study will be conducted to analyse its scalability to larger search spaces and games of varying complexity.Dissertation (MSc)--University of Pretoria, 2005.Computer Scienceunrestricte

Monte-Carlo Tree Search Algorithm in Pac-Man Identification of commonalities in 2D video games for realisation in AI (Artificial Intelligence)

The research is dedicated to the game strategy, which uses the Monte-Carlo Tree Search algorithm for the Pac-Man agent. Two main strategies were heavily researched for Pac-Man’s behaviour (Next Level priority) and HS (Highest Score priority). The Pacman game best known as STPacman is a 2D maze game that will allow users to play the game using artificial intelligence and smart features such as, Panic buttons (where players can activate on or off when they want and when they do activate it Pacman will be controlled via Artificial intelligence). A Variety of experiments were provided to compare the results to determine the efficiency of every strategy. A lot of intensive research was also put into place to find a variety of 2D games (Chess, Checkers, Go, etc.) which have similar functionalities to the game of Pac-Man. The main idea behind the research was to see how effective 2D games will be if they were to be implemented in the program (Classes/Methods) and how well would the artificial intelligence used in the development of STPacman behave/perform in a variety of different 2D games. A lot of time was also dedicated to researching an ‘AI’ engine that will be able to develop any 2D game based on the users submitted requirements with the use of a spreadsheet functionality (chapter 3, topic 3.3.1 shows an example of the spreadsheet feature) which will contain near enough everything to do with 2D games such as the parameters (The API/Classes/Methods/Text descriptions and more). The spreadsheet feature will act as a tool that will scan/examine all of the users submitted requirements and will give a rough estimation(time) on how long it will take for the chosen 2D game to be developed. It will have a lot of smart functionality and if the game is not unique like chess/checkers it will automatically recognize it and alert the user of it

Artificial Intelligence Techniques Applied To Draughts

This thesis documents the work done to develop a draughts playing program that learns game strategies utilising various Artificial Intelligence (AI) techniques with the goal of being able to play draughts at a reasonably high skill level as a result of having played against itself without external guidance. Context/Background: AI is a fast evolving field of study. The motivation being programming computers to learn from experience should eventually eliminate the need for this detailed, time consuming, and costly programming effort currently required to program solutions to problems. Aims: The aim is to investigate a variety of AI techniques. The program’s effectiveness will be assessed in both evaluating moves and playing a computationally intensive game. Minimax based algorithms together with a basic scoring heuristic are used to evaluate enough of the game tree to pick high utility moves. Later the scoring heuristic is augmented using artificial intelligence techniques. As a result of this training “smart scoring behaviour” the program is expected to learn how to best assign values to each of the squares on the draughts board enabling it to play at an adequately high skill level. Method: In this thesis a version of the board game Draughts is implemented in the Java programming language. Players were developed using a variety of techniques. These algorithms were tested by comparing running times, number of nodes of the game tree searched and the utility of the moves picked. In addition an algorithm is developed to assign scores to given board states using a genetic algorithm. Results: The project was a success for the most part permitting the creation of the game of draughts in the JAVA programming language. Four out of the five proposed move selection techniques were successfully tested in isolation. Finally the genetic algorithm demonstrated the ability to augment the scoring heuristic without the benefit of external guidance in the form of human experience

Evolving Effective Micro Behaviors for Real-Time Strategy Games

Real-Time Strategy games have become a new frontier of artificial intelligence research. Advances in real-time strategy game AI, like with chess and checkers before, will significantly advance the state of the art in AI research. This thesis aims to investigate using heuristic search algorithms to generate effective micro behaviors in combat scenarios for real-time strategy games. Macro and micro management are two key aspects of real-time strategy games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes against equal numbers of opponent units or win even when outnumbered. In this research, we use influence maps and potential fields as a basis representation to evolve micro behaviors. We first compare genetic algorithms against two types of hill climbers for generating competitive unit micro management. Second, we investigated the use of case-injected genetic algorithms to quickly and reliably generate high quality micro behaviors. Then we compactly encoded micro behaviors including influence maps, potential fields, and reactive control into fourteen parameters and used genetic algorithms to search for a complete micro bot, ECSLBot. We compare the performance of our ECSLBot with two state of the art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular real-time strategy game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. In addition, the same approach works to create competitive micro behaviors in another game SeaCraft. Using parallelized genetic algorithms to evolve parameters in SeaCraft we are able to speed up the evolutionary process from twenty one hours to nine minutes. We believe this work provides evidence that genetic algorithms and our representation may be a viable approach to creating effective micro behaviors for winning skirmishes in real-time strategy games

A Refinement-Based Heuristic Method for Decision Making in the Context of Ayo Game

Games of strategy, such as chess have served as a convenient test of skills at devising efficient search algorithms, formalizing knowledge, and bringing the power of computation to bear on “intractable” problems. Generally, minimax search has been the fundamental concept of obtaining solution to game problems. However, there are a number of limitations associated with using minimax search in order to offer solution to Ayo game. Among these limitations are: (i.) improper design of a suitable evaluator for moves before the moves are made, and (ii.) inability to select a correct move without assuming that players will play optimally. This study investigated the extent to which the knowledge of minimax search technique could be enhanced with a refinement-based heuristic method for playing Ayo game. This is complemented by the CDG (an end game strategy) for generating procedures such that only good moves are generated at any instance of playing Ayo game by taking cognizance of the opponent strategy of play. The study was motivated by the need to advance the African board game – Ayo – to see how it could be made to be played by humans across the globe, by creating both theoretical and product-oriented framework. This framework provides local Ayo game promotion initiatives in accordance with state-of-the-art practices in the global game playing domain. In order to accomplish this arduous task, both theoretical and empirical approaches were used. The theoretical approach reveals some mathematical properties of Ayo game with specific emphasis on the CDG as an end game strategy and means of obtaining the minimal and maximal CDG configurations. Similarly, a theoretical analysis of the minimax search was given and was enhanced with the Refinement-based heuristics. For the empirical approach, we simulated Ayo game playing on a digital viii computer and studied the behaviour of the various heuristic metrics used and compared the play strategies of the simulation with AWALE (the world known Ayo game playing standard software). Furthermore, empirical judgment was carried out on how experts play Ayo game as a means of evaluating the performance of the heuristics used to evolve the Ayo player in the simulation which gives room for statistical interpretation. This projects novel means of solving the problem of decision making in move selections in computer game playing of Ayo game. The study shows how an indigenous game like Ayo can generate integer sequence, and consequently obtain some self-replicating patterns that repeat themselves at different iterations. More importantly, the study gives an efficient and usable operation support tools in the prototype simulation of Ayo game playing that has improvement over Awal

Evolving board evaluation fuctions for a complex strategy game

The development of board evaluation functions for complex strategy games has been approached in a variety of ways. The analysis of game interactions is recognized as a valid analogy to common real-world problems, which often present difficulty in designing algorithms to solve them. Genetic programming, as a branch of evolutionary computation,provides advantages over traditional algorithms in solving these complex real-world problems in speed, robustness and flexibility. This thesis attempts to address the problem of applying genetic programming techniques to the evolution of a strategy for evaluating potential moves in a one-step lookahead intelligent agent heuristic for a complex strategy based game. This is meant to continue the work in artificial intelligence which seeks to provide computer systems with the tools they need to learn how to operate within a domain, given only the basic building blocks. The issues surrounding this problem are formulated and techniques are presented within the realm of genetic programming which aim to contribute to the solution of this problem. The domain chosen is the strategy game known as Acquire, whose object is to amass wealth while investing stock in hotel chains and effecting mergers of these chains as they grow. The evolution of the board evaluation functions to be used by agent players of the game is accomplished via genetic programming. Implementation details are discussed, empirical results are presented, and the strategies of some of the best players are analyzed. Future improvements on these techniques within this domain are outlined, as well as implications for artificial intelligence and genetic programming.M.S., Computer Science -- Drexel University, 200