Situation based strategic positioning for coordinating a team of homogeneous agents

. In this paper we are proposing an approach for coordinating a team ofhomogeneous agents based on a flexible common Team Strategy as well as onthe concepts of Situation Based Strategic Positioning and Dynamic Positioningand Role Exchange. We also introduce an Agent Architecture including a specifichigh-level decision module capable of implementing this strategy. Ourproposal is based on the formalization of what is a team strategy for competingwith an opponent team having opposite goals. A team strategy is composed of aset of agent types and a set of tactics, which are also composed of several formations.Formations are used for different situations and assign each agent a defaultspatial positioning and an agent type (defining its behaviour at several levels).Agents reactivity is also introduced for appropriate response to the dynamicsof the current situation. However, in our approach this is done in a way thatpreserves team coherence instead of permitting uncoordinated agent behaviour.We have applied, with success, this coordination approach to the RoboSoccersimulated domain. The FC Portugal team, developed using this approach wonthe RoboCup2000 (simulation league) European and World championshipsscoring a total of 180 goals and conceding none

Ambiente de simulação para agentes em futebol robótico

Mestrado em Engenharia de Computadores e TelemáticaO teste de algoritmos na área da robótica pode ser uma tarefa difícil, especialmente se o teste envolver múltipos robots. Neste contexto o uso de um simulador torna-se uma ferramenta importante no teste de algoritmos pois permite ultrapassar algumas limitações e oferece várias vantagens. CAMBADA é a equipa de futebol robótico da liga de tamanho médio da Universidade de Aveiro, Portugal. A equipa está familiarizada com as limitações do uso de robots reais para o teste de algoritmos. Devido a isso o simulador criado pela equipa Brainstormers Tribots foi adaptado para prover um ambiente de simulação ao software CAMBADA e estava em uso aquando do início desta dissertação. O simulador oferecia pouca flexibilidade na modelação dos robots que resultava em comportamentos imprecisos, oferecia também reduzida interacção com a simulação. O objectivo desta dissertação é criar um ambiente de simulação para agentes em futebol robótico com a intenção de melhorar o ambiente de simulação da equipa CAMBADA. O simulador deve ser capaz de simular dinâmica de objectos a três dimensões, sensores e actuadores ao mesmo tempo que oferece visualização do mundo e a possibilidade de interagir com a simulação. Da pesquisa realizada sobre simuladores robóticos o simulador Gazebo respeitava os nossos requisitos e foi escolhido para código base do nosso simulador. Para criar um ambiente simulado adequado à equipa CAMBADA alguns componentes do Gazebo foram alterados e novos sensores e actuadores virtuais foram desenvolvidos. Vários componentes do software CAMBADA tiveram que sofrer alterações de modo a suportar um ambiente simulado. O robot virtual foi modelado de modo a assemelhar-se com o robot real com o objectivo de obter comportamentos mais precisos. O simulador desenvolvido substituiu a solução anteriormente criada pela equipa CAMBADA e foi usado nos testes de preparação para a participação da equipa no RoboCup 2010 em Singapura onde deu o seu contributo na obtenção do terceiro lugar.In the field of robotics, testing algorithms with the real robots can be a di cult task, specially if the test involves more than one robot. In this context a simulator is an important tool for testing algorithms because it helps overcome some limitation and o ers several advantages. CAMBADA is the RoboCup MSL soccer team of the University of Aveiro, Portugal. The team is familiar with the limitations of using the real robots for testing algorithms. Therefore, a simulator created by the Brainstormers Tribots team was adapted to provide a simulated environment for their software and was used for testing at the time of the beginning of this thesis. The simulator offered low flexibility on the modeling of the robots from which resulted inaccurate behaviors, it also o ered reduced interaction with the simulation. The purpose of this thesis is to create a simulation environment for robotic soccer agents with the intention of improving the simulated environment for the CAMBADA team. The simulation must provide three-dimensional dynamics of objects, be capable of simulating sensors and actuators, allow the visualization of the simulation and provide interaction with the simulation. From the conducted survey about robotic simulators, the simulator Gazebo complied with our requirements and was chosen to provide the code base for our simulator. To create an adequate simulation environment for the CAMBADA team some components of Gazebo were modi ed and new sensors and actuator were developed. Several components of the CAMBADA software had to be modified to support the simulated environment. The virtual robot was modeled to resemble the real robot to provide more accurate behaviors. The developed simulator substituted the previous solution created by CAMBADA team and was used in the preparation tests for the participation in the RoboCup 2010 in Singapore where it contributed to obtain of the third-place

Optimizing simulated humanoid robot skills

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Genetic programming for the RoboCup Rescue Simulation System

The Robocup Rescue Simulation System (RCRSS) is a dynamic system of multi-agent interaction, simulating a large-scale urban disaster scenario. Teams of rescue agents are charged with the tasks of minimizing civilian casualties and infrastructure damage while competing against limitations on time, communication, and awareness. This thesis provides the first known attempt of applying Genetic Programming (GP) to the development of behaviours necessary to perform well in the RCRSS. Specifically, this thesis studies the suitability of GP to evolve the operational behaviours required of each type of rescue agent in the RCRSS. The system developed is evaluated in terms of the consistency with which expected solutions are the target of convergence as well as by comparison to previous competition results. The results indicate that GP is capable of converging to some forms of expected behaviour, but that additional evolution in strategizing behaviours must be performed in order to become competitive. An enhancement to the standard GP algorithm is proposed which is shown to simplify the initial search space allowing evolution to occur much quicker. In addition, two forms of population are employed and compared in terms of their apparent effects on the evolution of control structures for intelligent rescue agents. The first is a single population in which each individual is comprised of three distinct trees for the respective control of three types of agents, the second is a set of three co-evolving subpopulations one for each type of agent. Multiple populations of cooperating individuals appear to achieve higher proficiencies in training, but testing on unseen instances raises the issue of overfitting

Exploiting Opponent Modeling For Learning In Multi-agent Adversarial Games

An issue with learning effective policies in multi-agent adversarial games is that the size of the search space can be prohibitively large when the actions of both teammates and opponents are considered simultaneously. Opponent modeling, predicting an opponent’s actions in advance of execution, is one approach for selecting actions in adversarial settings, but it is often performed in an ad hoc way. In this dissertation, we introduce several methods for using opponent modeling, in the form of predictions about the players’ physical movements, to learn team policies. To explore the problem of decision-making in multi-agent adversarial scenarios, we use our approach for both offline play generation and real-time team response in the Rush 2008 American football simulator. Simultaneously predicting the movement trajectories, future reward, and play strategies of multiple players in real-time is a daunting task but we illustrate how it is possible to divide and conquer this problem with an assortment of data-driven models. By leveraging spatio-temporal traces of player movements, we learn discriminative models of defensive play for opponent modeling. With the reward information from previous play matchups, we use a modified version of UCT (Upper Conference Bounds applied to Trees) to create new offensive plays and to learn play repairs to counter predicted opponent actions. iii In team games, players must coordinate effectively to accomplish tasks while foiling their opponents either in a preplanned or emergent manner. An effective team policy must generate the necessary coordination, yet considering all possibilities for creating coordinating subgroups is computationally infeasible. Automatically identifying and preserving the coordination between key subgroups of teammates can make search more productive by pruning policies that disrupt these relationships. We demonstrate that combining opponent modeling with automatic subgroup identification can be used to create team policies with a higher average yardage than either the baseline game or domain-specific heuristics

Learning Motion Skills for a Humanoid Robot

This thesis investigates the learning of motion skills for humanoid robots. As groundwork, a humanoid robot with integrated fall management was developed as an experimental platform. Then, two different approaches for creating motion skills were investigated. First, one that is based on Cartesian quintic splines with optimized parameters. Second, a reinforcement learning-based approach that utilizes the first approach as a reference motion to guide the learning. Both approaches were tested on the developed robot and on further simulated robots to show their generalization. A special focus was set on the locomotion skill, but a standing-up and kick skill are also discussed. Diese Dissertation beschäftigt sich mit dem Lernen von Bewegungsfähigkeiten für humanoide Roboter. Als Grundlage wurde zunächst ein humanoider Roboter mit integriertem Fall Management entwickelt, welcher als Experimentalplatform dient. Dann wurden zwei verschiedene Ansätze für die Erstellung von Bewegungsfähigkeiten untersucht. Zu erst einer der kartesische quintische Splines mit optimierten Parametern nutzt. Danach wurde ein Ansatz basierend auf bestärkendem Lernen untersucht, welcher den ersten Ansatz als Referenzbewegung benutzt. Beide Ansätze wurden sowohl auf der entwickelten Roboterplatform, als auch auf weiteren simulierten Robotern getestet um die Generalisierbarkeit zu zeigen. Ein besonderer Fokus wurde auf die Fähigkeit des Gehens gelegt, aber auch Aufsteh- und Schussfähigkeiten werden diskutiert

Second Workshop on Modelling of Objects, Components and Agents

This report contains the proceedings of the workshop Modelling of Objects, Components, and Agents (MOCA'02), August 26-27, 2002.The workshop is organized by the 'Coloured Petri Net' Group at the University of Aarhus, Denmark and the 'Theoretical Foundations of Computer Science' Group at the University of Hamburg, Germany. The homepage of the workshop is: http://www.daimi.au.dk/CPnets/workshop02