A visual demonstration of convergence properties of cooperative coevolution

We introduce a model for cooperative coevolutionary algorithms (CCEAs) using partial mixing, which allows us to compute the expected long-run convergence of such algorithms when individuals ’ fitness is based on the maximum payoff of some N evaluations with partners chosen at random from the other population. Using this model, we devise novel visualization mechanisms to attempt to qualitatively explain a difficult-to-conceptualize pathology in CCEAs: the tendency for them to converge to suboptimal Nash equilibria. We further demonstrate visually how increasing the size of N, or biasing the fitness to include an ideal-collaboration factor, both improve the likelihood of optimal convergence, and under which initial population configurations they are not much help

Introducción a las técnicas de aprendizaje automático (parte II): entorno de competición

El presente artículo describe la parte práctica de la asignatura de formación en Aprendizaje Automático, en el entorno de nuestro grupo de investigación. Se estructura como una competición entre distintos grupos de trabajo, lo cual sienta las bases para motivar al alumno y establecer un marco común de comparación de distintas metodologías de Aprendizaje Automático

Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Evolving board evaluation fuctions for a complex strategy game

The development of board evaluation functions for complex strategy games has been approached in a variety of ways. The analysis of game interactions is recognized as a valid analogy to common real-world problems, which often present difficulty in designing algorithms to solve them. Genetic programming, as a branch of evolutionary computation,provides advantages over traditional algorithms in solving these complex real-world problems in speed, robustness and flexibility. This thesis attempts to address the problem of applying genetic programming techniques to the evolution of a strategy for evaluating potential moves in a one-step lookahead intelligent agent heuristic for a complex strategy based game. This is meant to continue the work in artificial intelligence which seeks to provide computer systems with the tools they need to learn how to operate within a domain, given only the basic building blocks. The issues surrounding this problem are formulated and techniques are presented within the realm of genetic programming which aim to contribute to the solution of this problem. The domain chosen is the strategy game known as Acquire, whose object is to amass wealth while investing stock in hotel chains and effecting mergers of these chains as they grow. The evolution of the board evaluation functions to be used by agent players of the game is accomplished via genetic programming. Implementation details are discussed, empirical results are presented, and the strategies of some of the best players are analyzed. Future improvements on these techniques within this domain are outlined, as well as implications for artificial intelligence and genetic programming.M.S., Computer Science -- Drexel University, 200

Opponent modeling and exploitation in poker using evolved recurrent neural networks

As a classic example of imperfect information games, poker, in particular, Heads-Up No-Limit Texas Holdem (HUNL), has been studied extensively in recent years. A number of computer poker agents have been built with increasingly higher quality. While agents based on approximated Nash equilibrium have been successful, they lack the ability to exploit their opponents effectively. In addition, the performance of equilibrium strategies cannot be guaranteed in games with more than two players and multiple Nash equilibria. This dissertation focuses on devising an evolutionary method to discover opponent models based on recurrent neural networks. A series of computer poker agents called Adaptive System for Hold’Em (ASHE) were evolved for HUNL. ASHE models the opponent explicitly using Pattern Recognition Trees (PRTs) and LSTM estimators. The default and board-texture-based PRTs maintain statistical data on the opponent strategies at different game states. The Opponent Action Rate Estimator predicts the opponent’s moves, and the Hand Range Estimator evaluates the showdown value of ASHE’s hand. Recursive Utility Estimation is used to evaluate the expected utility/reward for each available action. Experimental results show that (1) ASHE exploits opponents with high to moderate level of exploitability more effectively than Nash-equilibrium-based agents, and (2) ASHE can defeat top-ranking equilibrium-based poker agents. Thus, the dissertation introduces an effective new method to building high-performance computer agents for poker and other imperfect information games. It also provides a promising direction for future research in imperfect information games beyond the equilibrium-based approach.Computer Science