Using Monte Carlo Search With Data Aggregation to Improve Robot Soccer Policies

RoboCup soccer competitions are considered among the most challenging multi-robot adversarial environments, due to their high dynamism and the partial observability of the environment. In this paper we introduce a method based on a combination of Monte Carlo search and data aggregation (MCSDA) to adapt discrete-action soccer policies for a defender robot to the strategy of the opponent team. By exploiting a simple representation of the domain, a supervised learning algorithm is trained over an initial collection of data consisting of several simulations of human expert policies. Monte Carlo policy rollouts are then generated and aggregated to previous data to improve the learned policy over multiple epochs and games. The proposed approach has been extensively tested both on a soccer-dedicated simulator and on real robots. Using this method, our learning robot soccer team achieves an improvement in ball interceptions, as well as a reduction in the number of opponents' goals. Together with a better performance, an overall more efficient positioning of the whole team within the field is achieved

The RoboCup agent behavior modeling challenge

Proceedings of: XI Workshop of Physical Agents 2010 in the framework of the Congreso Español De Informática, CEDI 2010. Valencia, Spain. 9th - 10th September, 2010.RoboCup is an international joint project that aims to foster Arti cial Intelligence (AI) and intelligent robotics research by providing a standard problem. RoboCup offers different challenges for intelligent agent researchers in a dynamic, real-time and multi-agent domain. One of these challenges, especially in the Simulation League, is the opponent modeling, which is crucial for the ultimate goal of the RoboCup project: develop a team of fully autonomous. In order to emphasize opponent-modeling approaches, the RoboCup Coach Competition was created and it was held every year (with some changes) from 2001 to 2006. Although there were several interesting research works about the agent modeling challenge during that time, several considerations were not well de ned and the competition was suspended after RoboCup Coach Competition 2006. In this paper, we propose a new approach for the competition to face the opponent modeling challenge in the RoboCup competition.No publicad

The winning advantage: using opponent models in robot Soccer

Proceeding of: Intelligent Data Engineering and Automated Learning, IDEAL 2009, 10th International Conference, Burgos, Spain, September, 23-26th, 2009.Opponent modeling is a skill in multi-agent systems (MAS) which attempts to create a model of the behavior of the opponent. This model can be used to predict the future actions of the opponent and generate appropriate strategies to play against it. Several researches present different methods to create an opponent model in the RoboCup environment. However, how these models can impact the performance of teams is an essential aspect. This paper introduces a novel approach to use efficiently opponent models in order to improve our own team behavior. The basis of this approach is the research done by CAOS Coach Team for modeling and recognizing behaviors evaluated in the RoboCup Coach Competition 2006. For using these models, it is necessary a special agent (coach) which can model the observed opponent team (based on the previous research) and communicate a counter-strategy to the coached players (using the approach proposed in this paper). The evaluation of this approach is a hard problem, but we have conducted several experiments that can help us to know if we are going in a promising direction.This work has been supported by the Spanish Government under project TRA2007-67374-C02-02.Publicad

Exploiting opponent behavior in multi-agent systems

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

OMBO: An opponent modeling approach

In competitive domains, some knowledge about the opponent can give players a clear advantage. This idea led many people to propose approaches that automatically acquire models of opponents, based only on the observation of their input–output behavior. If opponent outputs could be accessed directly, a model can be constructed by feeding a machine learning method with traces of the behavior of the opponent. However, that is not the case in the RoboCup domain where an agent does not have direct access to the opponent inputs and outputs. Rather, the agent sees the opponent behavior from its own point of view and inputs and outputs (actions) have to be inferred from observation. In this paper, we present an approach to model low-level behavior of individual opponent agents. First, we build a classifier to infer and label opponent actions based on observation. Second, our agent observes an opponent and labels its actions using the previous classifier. From these observations, machine learning techniques generate a model that predicts the opponent actions. Finally, the agent uses the model to anticipate opponent actions. In order to test our ideas, we have created an architecture called OMBO (Opponent Modeling Based on Observation). Using OMBO, a striker agent can anticipate goalie actions. Results show that in this striker-goalie scenario, scores are significantly higher using the acquired opponent's model of actions.This work has been partially supported by the Spanish MCyT under projects TRA2007-67374- C02-02 and TIN-2005-08818-C04.Also, it has been supported under MEC grant by TIN2005-08945- C06-05. We thank anonymous reviewers for their helpful comments.Publicad

Classifying efficiently the behavior of a Soccer team

Proceeding of: 10th International Conference on Intelligent Autonomous Systems (IAS 2008), Baden Baden, Germany, July 23-25th, 2008.In order to make a good decision, humans usually try to predict the behavior of others. By this prediction, many different tasks can be performed, such as to coordinate with them, to assist them or to predict their future behavior. In competitive domains, to recognize the behavior of the opponent can be very advantageous. In this paper, an approach for creating automatically the model of the behavior of a soccer team is presented. This approach is an effective and notable improvement of a previous work. As the actions performed by a soccer team are sequential, this sequentiality should be considered in the modeling process. Therefore, the observations of a soccer team in a dynamic, complex and continuous multi-variate world state are transformed into a sequence of atomic behaviors. Then, this sequence is analyzed in order to find out a model that defines the team behavior. Finally, the classification of an observed team is done by using a statistical test.This work has been supported by the Spanish Ministry of Education and Science under project TRA-2007-67374-C02-02.Publicad