Unobtrusive workstation farming without inconveniencing owners: Learning Backgammon with a genetic algorithm

Most efforts at low-cost parallel computing assume a monopoly on the hardware being used. That all-or-nothing attitude ignores many machines dedicated to other activities, but which sit idle for 16 hours a day. However naive attempts to utilize idle machines can interfere with their primary purpose. This paper describes the successful effort to unobtrusively farm idle machines, for an artificial intelligence system using a genetic algorithm to learn the game Backgammon. It maintains owners' full access to their machines, without causing any detectable interference

Computationally intensive and noisy tasks: Co-evolutionary learning and temporal difference learning

The most difficult but realistic learning tasks are both noisy and computationally intensive. This paper investigates how, for a given solution representation, co-evolutionary learning can achieve the highest ability from the least computation time. Using a population of Backgammon strategies, this paper examines ways to make computational costs reasonable. With the same simple architecture Gerald Tesauro used for Temporal Difference learning to create the Backgammon strategy `Pubeval', co-evolutionary learning here creates a better player

Black magic: Interdependence prevents principled parameter setting, self adapting costs too much computation

The No Free Lunch theorem shows that the best algorithm for a particular problem is one tuned to that problem. Evolutionary Computation (EC) has many parameters to tune. These are usually fixed by hand, more by art than science. A poor choice of any of these can result in unsatisfactory performance. Self-adapting "parameterless" EC aims to avoid this guesswork. However, those efforts have focussed on one or another attribute, never all of them simultaneously. This paper demonstrates that such piecewise attempts are doomed to failure, because the customizable features are so interdependent. To self-adapt one particular attribute (or even a few) while guessing the others still lets unlucky guesses wreak havoc. This paper gives concrete examples of how, in a coevolving population of Backgammon strategies, both EC-specific and problem-specific attributes are so heavily interdependent that they give rise to bizarre side-effects. Even a fully "parameterless" algorithm adjusting all attributes may not be a solution because it vastly enlarges the search space, making EC even more computationally demanding. Will the human virtuoso remain indispensable

How important is your reputation in a multi-agent environment

Most work on the evolutionary approach to the iterated prisoner's dilemma (IPD) game uses a binary model where the choice of each player can only be cooperation or defection. However, we rarely commit ourselves to complete cooperation or defection in the real world. This paper examines the continuous IPD game and similarities and differences between the discrete and continuous games. The paper also studies the issue of reputation of a player, following Nowak and Sigmund's recent work, and how it affects the evolution of cooperation. This study differs from Nowak and Sigmund's in that players in a population can have more than two levels of cooperation (or even continuous). The players are also changing all the time under the influence of selection, crossover and mutation. We think that this is a more realistic model of the evolution of society in the real world

Co-evolution in iterated prisoner's dilemma with intermediate levels of cooperation: Application to missile defense

There is a widespread perception that in conflict situations, more intermediate choices between full peace and total war makes full peace less likely. This view is a motivation for opposing the proposed National Missile Defense. This perception is partly due to research in the abstract game of Iterated Prisoner’s Dilemma. This paper critically evaluates this perception

Co-evolutionary learning on noisy tasks

The paper studies the effect of noise on co-evolutionary learning, using Backgammon as a typical noisy task. It might seem that co-evolutionary learning would be ill-suited to noisy tasks: genetic drift causes convergence to a population of similar individuals, and on noisy tasks it would seem to require many samples (i.e., many evaluations and long computation time) to discern small differences between similar population members. Surprisingly, the paper learns otherwise: for small population sizes, the number of evaluations does have an effect on learning; but for sufficiently large populations, more evaluations do not improve learning at all-population size is the dominant variable. This is because a large population maintains more diversity, so that the larger differences in ability can be discerned with a modest number of evaluations. This counter-intuitive result means that co-evolutionary learning is a feasible method for noisy tasks, such as military situations and investment management

On Evolving Robust Strategies for Iterated Prisoner's Dilemma

Evolution is a fundamental form of adaptation in a dynamic and complex environment. Genetic algorithms are an effective tool in the empirical study of evolution. This paper follows Axelrod's work [2] in using the genetic algorithm to evolve strategies for playing the game of Iterated Prisoner's Dilemma, using co-evolution, where each member of the population (each strategy) is evaluated by how it performs against the other members of the current population. This creates a dynamic environment in which the algorithm is optimising to a moving target instead of the usual evaluation against some fixed set of strategies. The hope is that this will stimulate an "arms race" of innovation [3]. We conduct two sets of experiments. The first set investigates what conditions evolve the best strategies. The second set studies the robustness of the strategies thus evolved, that is, are the strategies useful only in the round robin of its population or are they effective against a wide variety of oppo..

Viability of populations in a landscape

Darwen, P. J. and Green, D. G., Viability of populations in

The Exploitation of Cooperation in Iterated Prisoner's Dilemma

We follow Axelrod [2] in using the genetic algorithm to play Iterated Prisoner's Dilemma. Each member of the population (i.e., each strategy) is evaluated by how it performs against the other members of the current population. This creates a dynamic environment in which the algorithm is optimising to a moving target instead of the usual evaluation against some fixed set of strategies, causing an "arms race" of innovation [3]. We conduct two sets of experiments. The first set investigates what conditions evolve the best strategies. The second set studies the robustness of the strategies thus evolved, that is, are the strategies useful only in the round robin of its population or are they effective against a wide variety of opponents? Our results indicate that the population has nearly always converged by about 250 generations, by which time the bias in the population has almost always stabilised at 85%. Our results confirm that cooperation almost always becomes the dominant strategy [1,..