Evolution of a robotic soccer player

Robotic soccer is a complex domain where, rather than hand-coding computer programs to control the players, it is possible to create them through evolutionary methods. This has been successfully done before by using genetic programming with high-level genes. Such an approach is, however, limiting. This work attempts to reduce that limit by evolving control programs using genetic programming with low-level nodes

Programming Robosoccer agents by modelling human behavior

The Robosoccer simulator is a challenging environment for artificial intelligence, where a human has to program a team of agents and introduce it into a soccer virtual environment. Most usually, Robosoccer agents are programmed by hand. In some cases, agents make use of Machine learning (ML) to adapt and predict the behavior of the opposite team, but the bulk of the agent has been preprogrammed. The main aim of this paper is to transform Robosoccer into an interactive game and let a human control a Robosoccer agent. Then ML techniques can be used to model his/her behavior from training instances generated during the play. This model will be used later to control a Robosoccer agent, thus imitating the human behavior. We have focused our research on low-level behavior, like looking for the ball, conducting the ball towards the goal, or scoring in the presence of opponent players. Results have shown that indeed, Robosoccer agents can be controlled by programs that model human play.Publicad

An approach to evolve and exploit repertoires of general robot behaviours

Recent works in evolutionary robotics have shown the viability of evolution driven by behavioural novelty and diversity. These evolutionary approaches have been successfully used to generate repertoires of diverse and high-quality behaviours, instead of driving evolution towards a single, task-specific solution. Having repertoires of behaviours can enable new forms of robotic control, in which high-level controllers continually decide which behaviour to execute. To date, however, only the use of repertoires of open-loop locomotion primitives has been studied. We propose EvoRBC-II, an approach that enables the evolution of repertoires composed of general closed-loop behaviours, that can respond to the robot's sensory inputs. The evolved repertoire is then used as a basis to evolve a transparent higher-level controller that decides when and which behaviours of the repertoire to execute. Relying on experiments in a simulated domain, we show that the evolved repertoires are composed of highly diverse and useful behaviours. The same repertoire contains sufficiently diverse behaviours to solve a wide range of tasks, and the EvoRBC-II approach can yield a performance that is comparable to the standard tabula-rasa evolution. EvoRBC-II enables automatic generation of hierarchical control through a two-step evolutionary process, thus opening doors for the further exploration of the advantages that can be brought by hierarchical control.info:eu-repo/semantics/acceptedVersio

Co-Evolving Soccer Softbot Team Coordination with Genetic Programming

. In this paper we explain how we applied genetic programming to behavior-based team coordination in the RoboCup Soccer Server domain. Genetic programming is a promising new method for automatically generating functions and algorithms through natural selection. In contrast to other learning methods, genetic programming's automatic programming makes it a natural approach for developing algorithmic robot behaviors. The RoboCup Soccer Server was a very challenging domain for genetic programming, but we were pleased with the results. At the end, genetic programming had produced teams of soccer softbots which had learned to cooperate to play a good game of simulator soccer. 1 Introduction The RoboCup competition pits robots against each other in a simulated soccer tournament [Kitano et al, 1995]. The aim of the RoboCup competition is to foster an interdisciplinary approach to robotics and agent-based Artificial Intelligence by presenting a domain that requires large-scale cooperation and c..

Evolving board evaluation fuctions for a complex strategy game

The development of board evaluation functions for complex strategy games has been approached in a variety of ways. The analysis of game interactions is recognized as a valid analogy to common real-world problems, which often present difficulty in designing algorithms to solve them. Genetic programming, as a branch of evolutionary computation,provides advantages over traditional algorithms in solving these complex real-world problems in speed, robustness and flexibility. This thesis attempts to address the problem of applying genetic programming techniques to the evolution of a strategy for evaluating potential moves in a one-step lookahead intelligent agent heuristic for a complex strategy based game. This is meant to continue the work in artificial intelligence which seeks to provide computer systems with the tools they need to learn how to operate within a domain, given only the basic building blocks. The issues surrounding this problem are formulated and techniques are presented within the realm of genetic programming which aim to contribute to the solution of this problem. The domain chosen is the strategy game known as Acquire, whose object is to amass wealth while investing stock in hotel chains and effecting mergers of these chains as they grow. The evolution of the board evaluation functions to be used by agent players of the game is accomplished via genetic programming. Implementation details are discussed, empirical results are presented, and the strategies of some of the best players are analyzed. Future improvements on these techniques within this domain are outlined, as well as implications for artificial intelligence and genetic programming.M.S., Computer Science -- Drexel University, 200

A Methodology to Evolve Cooperation in Pursuit Domain using Genetic Network Programming

The design of strategies to devise teamwork and cooperation among agents is a central research issue in the field of multi-agent systems (MAS). The complexity of the cooperative strategy design can rise rapidly with increasing number of agents and their behavioral sophistication. The field of cooperative multi-agent learning promises solutions to such problems by attempting to discover agent behaviors as well as suggesting new approaches by applying machine learning techniques. Due to the difficulty in specifying a priori for an effective algorithm for multiple interacting agents, and the inherent adaptability of artificially evolved agents, recently, the use of evolutionary computation as a machining learning technique and a design process has received much attention. In this thesis, we design a methodology using an evolutionary computation technique called Genetic Network Programming (GNP) to automatically evolve teamwork and cooperation among agents in the pursuit domain. Simulation results show that our proposed methodology was effective in evolving teamwork and cooperation among agents. Compared with Genetic Programming approaches, its performance is significantly superior, its computation cost is less and the learning speed is faster. We also provide some analytical results of the proposed approach

Complementary Layered Learning

Layered learning is a machine learning paradigm used to develop autonomous robotic-based agents by decomposing a complex task into simpler subtasks and learns each sequentially. Although the paradigm continues to have success in multiple domains, performance can be unexpectedly unsatisfactory. Using Boolean-logic problems and autonomous agent navigation, we show poor performance is due to the learner forgetting how to perform earlier learned subtasks too quickly (favoring plasticity) or having difficulty learning new things (favoring stability). We demonstrate that this imbalance can hinder learning so that task performance is no better than that of a suboptimal learning technique, monolithic learning, which does not use decomposition. Through the resulting analyses, we have identified factors that can lead to imbalance and their negative effects, providing a deeper understanding of stability and plasticity in decomposition-based approaches, such as layered learning. To combat the negative effects of the imbalance, a complementary learning system is applied to layered learning. The new technique augments the original learning approach with dual storage region policies to preserve useful information from being removed from an agent’s policy prematurely. Through multi-agent experiments, a 28% task performance increase is obtained with the proposed augmentations over the original technique

USING COEVOLUTION IN COMPLEX DOMAINS

Genetic Algorithms is a computational model inspired by Darwin's theory of evolution. It has a broad range of applications from function optimization to solving robotic control problems. Coevolution is an extension of Genetic Algorithms in which more than one population is evolved at the same time. Coevolution can be done in two ways: cooperatively, in which populations jointly try to solve an evolutionary problem, or competitively. Coevolution has been shown to be useful in solving many problems, yet its application in complex domains still needs to be demonstrated.Robotic soccer is a complex domain that has a dynamic and noisy environment. Many Reinforcement Learning techniques have been applied to the robotic soccer domain, since it is a great test bed for many machine learning methods. However, the success of Reinforcement Learning methods has been limited due to the huge state space of the domain. Evolutionary Algorithms have also been used to tackle this domain; nevertheless, their application has been limited to a small subset of the domain, and no attempt has been shown to be successful in acting on solving the whole problem.This thesis will try to answer the question of whether coevolution can be applied successfully to complex domains. Three techniques are introduced to tackle the robotic soccer problem. First, an incremental learning algorithm is used to achieve a desirable performance of some soccer tasks. Second, a hierarchical coevolution paradigm is introduced to allow coevolution to scale up in solving the problem. Third, an orchestration mechanism is utilized to manage the learning processes