Exploiting opponent behavior in multi-agent systems

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

FC Portugal 3D Simulation Team: Team Description Paper 2020

The FC Portugal 3D team is developed upon the structure of our previous Simulation league 2D/3D teams and our standard platform league team. Our research concerning the robot low-level skills is focused on developing behaviors that may be applied on real robots with minimal adaptation using model-based approaches. Our research on high-level soccer coordination methodologies and team playing is mainly focused on the adaptation of previously developed methodologies from our 2D soccer teams to the 3D humanoid environment and on creating new coordination methodologies based on the previously developed ones. The research-oriented development of our team has been pushing it to be one of the most competitive over the years (World champion in 2000 and Coach Champion in 2002, European champion in 2000 and 2001, Coach 2nd place in 2003 and 2004, European champion in Rescue Simulation and Simulation 3D in 2006, World Champion in Simulation 3D in Bremen 2006 and European champion in 2007, 2012, 2013, 2014 and 2015). This paper describes some of the main innovations of our 3D simulation league team during the last years. A new generic framework for reinforcement learning tasks has also been developed. The current research is focused on improving the above-mentioned framework by developing new learning algorithms to optimize low-level skills, such as running and sprinting. We are also trying to increase student contact by providing reinforcement learning assignments to be completed using our new framework, which exposes a simple interface without sharing low-level implementation details

A Formalization of the Coach Problem

Coordination is an important aspect of multi-agent teamwork. In the context of robot soccer in the RoboCup Standard Platform League, our focus is on the coach as an external observer of the team, aiming to provide his teammates with effective tactical advice during matches. The coach problem can be approached from different angles: in order to adapt the behaviour of his teammates, he should at first be able to perform plan recognition on their observable actions. Furthermore, in providing them with appropriate advice, he should still adhere to the norms and regulations of the match to prevent penalties for his team. Also, when teammates' profiles and attributes are unknown or the system is only partially observable, coordination should be more 'ad hoc' to ensure robustness of the Multi-Agent System (MAS). In this work, we present a formalization of the problem of designing a coach in robot soccer, employing a temporal deontic logical framework. The framework is based on agent organizations[10], in which social coordination and norms play an important part

Applying biological paradigms to emerge behaviour in RoboCup Rescue team

This paper presents a hybrid behaviour process for performing collaborative tasks and coordination capabilities in a rescue team. RoboCup Rescue simulator and its associated international competition are used as the testbed for our proposal. Unlike other published work in this field one of our main concerns is having good results on RoboCup Rescue championships by emerging behaviour in agents using a biological paradigm. The benefit comes from the hierarchic and parallel organisation of the mammalian brain. In our behaviour process, Artificial Neural Networks are used in order to make agents capable of learning information from the environment. This allows agents to improve several algorithms like their Path Finding Algorithm to find the shortest path between two points. Also, we aim to filter the most important messages that arise from the environment, to make the right choice on the best path planning among many alternatives, in a short time. A policy action was implemented using Kohonen's network, Dijkstra's and D* algorithm. This policy has achieved good results in our tests, getting our team classified for RoboCup Rescue Simulation League 2005

ECOLANG - Communications Language for Ecological Simulations Network

O documento descreve a linguagem de comunicação utilizada num ambiente multiagente de simulações ecológicas, centrada na aplicação de simulação EcoDynamo ligada a vários agentes inteligentes e aplicações de visualização, e alarga a definição inicial da linguagem.As acções e percepções dos agentes são traduzidos em mensagens trocadas com o simulador e com outros agentes.As definições de conceitos seguem a notação BNF (Backus et al. 1960) e foram inspiradas na Coach Unilang language (Reis and Lau 2002).The document describes the communication language usedin one multi-agent systems environment for ecological simulations, based on the EcoDynamosimulator application linked with several intelligent agents andvisualisation applications and extends the initial definition of the language.The agents' actions and perceptions are translated into messages exchanged with thesimulator application and other agents.The concepts' definitions used follow the BNF notation (Backus et al. 1960) and it's inspiredin the Coach Unilang language (Reis and Lau 2002)

Vision based referee sign language recognition system for the RoboCup MSL league

In RoboCup Middle Size league (MSL) the main referee uses assisting technology, controlled by a second referee, to support him, in particular for conveying referee decisions for robot players with the help of a wireless communication system. In this paper a vision-based system is introduced, able to interpret dynamic and static gestures of the referee, thus eliminating the need for a second one. The referee's gestures are interpreted by the system and sent directly to the Referee Box, which sends the proper commands to the robots. The system is divided into four modules: a real time hand tracking and feature extraction, a SVM (Support Vector Machine) for static hand posture identification, an HMM (Hidden Markov Model) for dynamic unistroke hand gesture recognition, and a FSM (Finite State Machine) to control the various system states transitions. The experimental results showed that the system works very reliably, being able to recognize the combination of gestures and hand postures in real-time. For the hand posture recognition, with the SVM model trained with the selected features, an accuracy of 98,2% was achieved. Also, the system has many advantages over the current implemented one, like avoiding the necessity of a second referee, working on noisy environments, working on wireless jammed situations. This system is easy to implement and train and may be an inexpensive solution

Rule based strategies for large extensive-form games: A specification language for No-Limit Texas Hold'em agents

Poker is used to measure progresses in extensive-form games research due to its unique characteristics: it is a game where playing agents have to deal with incomplete information and stochastic scenarios and a large number of decision points. The development of Poker agents has seen significant advances in one-on-one matches but there are still no consistent results in multiplayer and in games against human experts. In order to allow for experts to aid the improvement of the agents' performance, we have created a high-level strategy specification language. To support strategy definition, we have also developed an intuitive graphical tool. Additionally, we have also created a strategy inferring system, based on a dynamically weighted Euclidean distance. This approach was validated through the creation of simple agents and by successfully inferring strategies from 10 human players. The created agents were able to beat previously developed mid-level agents by a good profit margin

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time

Definição de uma Linguagem para a Programação do Comportamento de Robôs Dentro do Contexto da RoboCup

A RoboCup Soccer é uma das ligas da Copa Mundial de Robôs. Nela testa-se a capacidade de robôs humanóides autônomos jogarem futebol. No ano de 2015 a Taura Bots, equipe de futebol de robôs da Universidade Federal de Santa Maria, participou pela primeira vez da competição. Assim, surgiu a necessidade de criação de uma linguagem para a programação do comportamento dos robôs que ofereça aos usuários um alto nível de abstração, portabilidade e uma sintaxe simples e intuitiva. O objetivo do projeto foi alcançado, levando em conta que a TauraLang, linguagem construída neste trabalho, já pode ser usada na programação de comportamentos simples e alguns já rodam no simulador