Rhoban Football Club: RoboCup Humanoid KidSize 2019 Champion Team Paper

International audienceIn 2019, Rhoban Football Club reached the first place of the KidSize soccer competition for the fourth time and performed the first in-game throw-in in the history of the Humanoid league. Building on our existing code-base, we improved some specific functionalities, introduced new behaviors and experimented with original methods for labeling videos. This paper presents and reviews our latest changes to both software and hardware, highlighting the lessons learned during RoboCup

CAOS Coach 2006 Simulation Team: An Opponent Modelling Approach

Agent technology represents a very interesting new means for analyzing, designing and building complex software systems. Nowadays, agent modelling in multi-agent systems is increasingly becoming more complex and significant. RoboCup Coach Competition is an exciting competition in the RoboCup Soccer League and its main goal is to encourage research in multii-agent modelling. This paper describes a novel method used by the team CAOS (CAOS Coach 2006 Simulation Team) in this competition. The objective of the team is to model successfully the behaviour of a multi-agent system

CAOS Coach 2006 Simulation Team: an opponent modelling approach

Agent technology l'epresents a vel'Y intel'esting new means for analyzing, designing and building complex software systems. Nowadays, agent modelling in multi-agent systems is increasingly becoming more complex and significant. RoboCup Coach Cornpetition is an exciting competition in the RoboCup Soccel' League and its main goal is to encourage reseal'ch in multiagent modelling. This papel' describes a novel method used by the team CAOS (CAOS Coach 2006 Simulation Tearn) in this competition. The objective of the team is to model successfully the behaviour of a multi-agent system.This work has been supported by the Spanish Ministry of Education and Science under project TRA-2007-67374-C02-02

Hypernetworks Analysis of RoboCup Interactions

Robotic soccer simulations are controlled environments in which the rich variety of interactions among agents make them good candidates to be studied as complex adaptive systems. The challenge is to create an autonomous team of soccer agents that can adapt and improve its behaviour as it plays other teams. By analogy with chess, the movements of the soccer agents and the ball form ever-changing networks as players in one team form structures that give their team an advantage. For example, the Defender’s Dilemma involves relationships between an attacker with the ball, a team-mate and a defender. The defender must choose between tackling the player with the ball, or taking a position to intercept a pass to the other attacker. Since these structures involve more that two interacting entities it is necessary to go beyond networks to multidimensional hypernetworks. In this context, this thesis investigates (i) is it possible to identify patterns of play, that lead a team to obtain an advantage ?, (ii) is it possible to forecast with a good degree of accuracy if a certain game action or sequence of game actions is going to be successful, before it has been completed ?, and (iii) is it possible to make behavioural patterns emerge in the game without specifying the behavioural rules in detail ? To investigate these research questions we devised two methods to analyse the interactions between robotic players, one based on traditional programming and one based on Deep Learning. The first method identified thousands of Defender’s Dilemma configurations from RoboCup 2D simulator games and found a statistically significant association between winning and the creation of the defender’s dilemma by the attackers of the winning team. The second method showed that a feedforward Artificial Neural Network trained on thousands of games can take as input the current game configuration and forecast to a high degree of accuracy if the current action will end up in a goal or not. Finally, we designed our own fast and simple robotic soccer simulator for investigating Reinforcement Learning. This showed that Reinforcement Learning using Proximal Policy Optimization could train two agents in the task of scoring a goal, using only basic actions without using pre-built hand-programmed skills. These experiments provide evidence that it is possible: to identify advantageous patterns of play; to forecast if an action or sequence of actions will be successful; and to make behavioural patterns emerge in the game without specifying the behavioural rules in detail

Perceção e arquitectura de software para robótica móvel

Doutoramento em Ciências da ComputaçãoWhen developing software for autonomous mobile robots, one has to inevitably tackle some kind of perception. Moreover, when dealing with agents that possess some level of reasoning for executing their actions, there is the need to model the environment and the robot internal state in a way that it represents the scenario in which the robot operates. Inserted in the ATRI group, part of the IEETA research unit at Aveiro University, this work uses two of the projects of the group as test bed, particularly in the scenario of robotic soccer with real robots. With the main objective of developing algorithms for sensor and information fusion that could be used e ectively on these teams, several state of the art approaches were studied, implemented and adapted to each of the robot types. Within the MSL RoboCup team CAMBADA, the main focus was the perception of ball and obstacles, with the creation of models capable of providing extended information so that the reasoning of the robot can be ever more e ective. To achieve it, several methodologies were analyzed, implemented, compared and improved. Concerning the ball, an analysis of ltering methodologies for stabilization of its position and estimation of its velocity was performed. Also, with the goal keeper in mind, work has been done to provide it with information of aerial balls. As for obstacles, a new de nition of the way they are perceived by the vision and the type of information provided was created, as well as a methodology for identifying which of the obstacles are team mates. Also, a tracking algorithm was developed, which ultimately assigned each of the obstacles a unique identi er. Associated with the improvement of the obstacles perception, a new algorithm of estimating reactive obstacle avoidance was created. In the context of the SPL RoboCup team Portuguese Team, besides the inevitable adaptation of many of the algorithms already developed for sensor and information fusion and considering that it was recently created, the objective was to create a sustainable software architecture that could be the base for future modular development. The software architecture created is based on a series of di erent processes and the means of communication among them. All processes were created or adapted for the new architecture and a base set of roles and behaviors was de ned during this work to achieve a base functional framework. In terms of perception, the main focus was to de ne a projection model and camera pose extraction that could provide information in metric coordinates. The second main objective was to adapt the CAMBADA localization algorithm to work on the NAO robots, considering all the limitations it presents when comparing to the MSL team, especially in terms of computational resources. A set of support tools were developed or improved in order to support the test and development in both teams. In general, the work developed during this thesis improved the performance of the teams during play and also the e ectiveness of the developers team when in development and test phases.Durante o desenvolvimento de software para robôs autónomos móveis, e inevitavelmente necessário lidar com algum tipo de perceção. Al em disso, ao lidar com agentes que possuem algum tipo de raciocínio para executar as suas ações, há a necessidade de modelar o ambiente e o estado interno do robô de forma a representar o cenário onde o robô opera. Inserido no grupo ATRI, integrado na unidade de investigação IEETA da Universidade de Aveiro, este trabalho usa dois dos projetos do grupo como plataformas de teste, particularmente no cenário de futebol robótico com robôs reais. Com o principal objetivo de desenvolver algoritmos para fusão sensorial e de informação que possam ser usados eficazmente nestas equipas, v arias abordagens de estado da arte foram estudadas, implementadas e adaptadas para cada tipo de robôs. No âmbito da equipa de RoboCup MSL, CAMBADA, o principal foco foi a perceção da bola e obstáculos, com a criação de modelos capazes de providenciar informação estendida para que o raciocino do robô possa ser cada vez mais eficaz. Para o alcançar, v arias metodologias foram analisadas, implementadas, comparadas e melhoradas. Em relação a bola, foi efetuada uma análise de metodologias de filtragem para estabilização da sua posição e estimação da sua velocidade. Tendo o guarda-redes em mente, foi também realizado trabalho para providenciar informação de bolas no ar. Quanto aos obstáculos, foi criada uma nova definição para a forma como são detetados pela visão e para o tipo de informação fornecida, bem como uma metodologia para identificar quais dos obstáculos são colegas de equipa. Além disso foi desenvolvido um algoritmo de rastreamento que, no final, atribui um identicador único a cada obstáculo. Associado a melhoria na perceção dos obstáculos foi criado um novo algoritmo para realizar desvio reativo de obstáculos. No contexto da equipa de RoboCup SPL, Portuguese Team, al em da inevitável adaptação de vários dos algoritmos j a desenvolvidos para fusão sensorial e de informação, tendo em conta que foi recentemente criada, o objetivo foi criar uma arquitetura sustentável de software que possa ser a base para futuro desenvolvimento modular. A arquitetura de software criada e baseada numa série de processos diferentes e métodos de comunicação entre eles. Todos os processos foram criados ou adaptados para a nova arquitetura e um conjunto base de papeis e comportamentos foi definido para obter uma framework funcional base. Em termos de perceção, o principal foco foi a definição de um modelo de projeção e extração de pose da câmara que consiga providenciar informação em coordenadas métricas. O segundo objetivo principal era adaptar o algoritmo de localização da CAMBADA para funcionar nos robôs NAO, considerando todas as limitações apresentadas quando comparando com a equipa MSL, principalmente em termos de recursos computacionais. Um conjunto de ferramentas de suporte foram desenvolvidas ou melhoradas para auxiliar o teste e desenvolvimento em ambas as equipas. Em geral, o trabalho desenvolvido durante esta tese melhorou o desempenho da equipas durante os jogos e também a eficácia da equipa de programação durante as fases de desenvolvimento e teste

Scaled Autonomy for Networked Humanoids

Humanoid robots have been developed with the intention of aiding in environments designed for humans. As such, the control of humanoid morphology and effectiveness of human robot interaction form the two principal research issues for deploying these robots in the real world. In this thesis work, the issue of humanoid control is coupled with human robot interaction under the framework of scaled autonomy, where the human and robot exchange levels of control depending on the environment and task at hand. This scaled autonomy is approached with control algorithms for reactive stabilization of human commands and planned trajectories that encode semantically meaningful motion preferences in a sequential convex optimization framework. The control and planning algorithms have been extensively tested in the field for robustness and system verification. The RoboCup competition provides a benchmark competition for autonomous agents that are trained with a human supervisor. The kid-sized and adult-sized humanoid robots coordinate over a noisy network in a known environment with adversarial opponents, and the software and routines in this work allowed for five consecutive championships. Furthermore, the motion planning and user interfaces developed in the work have been tested in the noisy network of the DARPA Robotics Challenge (DRC) Trials and Finals in an unknown environment. Overall, the ability to extend simplified locomotion models to aid in semi-autonomous manipulation allows untrained humans to operate complex, high dimensional robots. This represents another step in the path to deploying humanoids in the real world, based on the low dimensional motion abstractions and proven performance in real world tasks like RoboCup and the DRC

Exploiting Opponent Modeling For Learning In Multi-agent Adversarial Games

An issue with learning effective policies in multi-agent adversarial games is that the size of the search space can be prohibitively large when the actions of both teammates and opponents are considered simultaneously. Opponent modeling, predicting an opponent’s actions in advance of execution, is one approach for selecting actions in adversarial settings, but it is often performed in an ad hoc way. In this dissertation, we introduce several methods for using opponent modeling, in the form of predictions about the players’ physical movements, to learn team policies. To explore the problem of decision-making in multi-agent adversarial scenarios, we use our approach for both offline play generation and real-time team response in the Rush 2008 American football simulator. Simultaneously predicting the movement trajectories, future reward, and play strategies of multiple players in real-time is a daunting task but we illustrate how it is possible to divide and conquer this problem with an assortment of data-driven models. By leveraging spatio-temporal traces of player movements, we learn discriminative models of defensive play for opponent modeling. With the reward information from previous play matchups, we use a modified version of UCT (Upper Conference Bounds applied to Trees) to create new offensive plays and to learn play repairs to counter predicted opponent actions. iii In team games, players must coordinate effectively to accomplish tasks while foiling their opponents either in a preplanned or emergent manner. An effective team policy must generate the necessary coordination, yet considering all possibilities for creating coordinating subgroups is computationally infeasible. Automatically identifying and preserving the coordination between key subgroups of teammates can make search more productive by pruning policies that disrupt these relationships. We demonstrate that combining opponent modeling with automatic subgroup identification can be used to create team policies with a higher average yardage than either the baseline game or domain-specific heuristics

A usage-based model for the acquisition of syntactic constructions and its application in spoken language understanding

Gaspers J. A usage-based model for the acquisition of syntactic constructions and its application in spoken language understanding. Bielefeld: Universitätsbibliothek Bielefeld; 2014

Neural Probabilistic Methods for Event Sequence Modeling

This thesis focuses on modeling event sequences, namely, sequences of discrete events in continuous time. We build a family of generative probabilistic models that is able to reason about what events will happen in the future and when, given the history of previous events. Under our models, each event—as it happens—is allowed to update the future intensities of multiple event types, and the intensity of each event type—as nothing happens—is allowed to evolve with time along a trajectory. We use neural networks to allow the “updates” and “trajectories” to be complex and realistic. In the purely neural version of our model, all future event intensities are conditioned on the hidden state of a continuous-time LSTM, which has consumed every past event as it happened. To exploit domain-specific knowledge of how an event might only affect a few—but not all—future event intensities, we propose to introduce domain-specific structure into the model. We design a modeling language, by which a domain expert can write down the rules of a temporal deductive database. The database tracks facts over time; the rules deduce facts from other facts and from past events. Each fact has a time-varying state, computed by a neural network whose topology is determined by the fact’s provenance, including its experience of the past events that have contributed to deducing it. The possible event types at any time are given by special facts, whose intensities are neurally modeled alongside their states. We develop efficient methods for training our models, and doing inference with them. Applying the general principle of noise-contrastive estimation, we work out a stochastic training objective that is less expensive to optimize than the log-likelihood, which people typically maximize for parameter estimation. As in the discrete-time case that inspired us, the parameters that maximize our objective will provably maximize the log-likelihood as well. For the scenarios where we are given incomplete sequences, we propose particle smoothing—a form of sequential importance sampling—to impute the missing events. This thesis includes extensive experiments, demonstrating the effectiveness of our models and algorithms. On many synthetic and real-world datasets, on held-out sequences, we show empirically: (1) our purely neural model achieves competitive likelihood and predictive accuracy; (2) our neural-symbolic model improves prediction by encoding appropriate domain knowledge in the architecture; (3) for models to achieve the same level of log-likelihood, our noise-contrastive estimation needs considerably fewer function evaluations and less wall-clock time than maximum likelihood estimation; (4) our particle smoothing method is effective at inferring the ground-truth unobserved events. In this thesis, I will also discuss a few future research directions, including embedding our models within a reinforcement learner to discover causal structure and learn an intervention policy