Analysing the behaviour of robot teams through relational sequential pattern mining

This report outlines the use of a relational representation in a Multi-Agent domain to model the behaviour of the whole system. A desired property in this systems is the ability of the team members to work together to achieve a common goal in a cooperative manner. The aim is to define a systematic method to verify the effective collaboration among the members of a team and comparing the different multi-agent behaviours. Using external observations of a Multi-Agent System to analyse, model, recognize agent behaviour could be very useful to direct team actions. In particular, this report focuses on the challenge of autonomous unsupervised sequential learning of the team's behaviour from observations. Our approach allows to learn a symbolic sequence (a relational representation) to translate raw multi-agent, multi-variate observations of a dynamic, complex environment, into a set of sequential behaviours that are characteristic of the team in question, represented by a set of sequences expressed in first-order logic atoms. We propose to use a relational learning algorithm to mine meaningful frequent patterns among the relational sequences to characterise team behaviours. We compared the performance of two teams in the RoboCup four-legged league environment, that have a very different approach to the game. One uses a Case Based Reasoning approach, the other uses a pure reactive behaviour.Comment: 25 page

Interactive co-construction to study dynamical collaborative situations.

International audienceThe purpose of this paper is to present the principle of our framework CoPeFoot dedicated to the study of dynamic and collaborative situations. This research work aims to instate learning by the co-construction of such situations. The article starts by recalling constraints induced by such situations. Next, it introduces interactive co-construction assumption and their implementation in CoPeFoot. In fact, this implementation is based on two steps in CoPeFoot: firstly, machine learning for behavior modeling, using imitation of real users and secondly, refining this behavior by using interaction between the user and the simulation, enhanced by additional information called augmented virtuality. In order to do that, CoPeFoot lies on context base reasoning which is presented. The article ends by a first evaluation of this work

Real-time retrieval for case-based reasoning in interactive multiagent-based simulations

The aim of this paper is to present the principles and results about case-based reasoning adapted to real- time interactive simulations, more precisely concerning retrieval mechanisms. The article begins by introducing the constraints involved in interactive multiagent-based simulations. The second section pre- sents a framework stemming from case-based reasoning by autonomous agents. Each agent uses a case base of local situations and, from this base, it can choose an action in order to interact with other auton- omous agents or users' avatars. We illustrate this framework with an example dedicated to the study of dynamic situations in football. We then go on to address the difficulties of conducting such simulations in real-time and propose a model for case and for case base. Using generic agents and adequate case base structure associated with a dedicated recall algorithm, we improve retrieval performance under time pressure compared to classic CBR techniques. We present some results relating to the performance of this solution. The article concludes by outlining future development of our project

A Survey of Distributed and Data Intensive CBR Systems

Case-Based Reasoning is a methodology that uses information that has been considered as valid in previous situations to solve new problems. That use of the information allows CBR systems to be applied to different fields where the reutilization of past good solutions is a key factor. In this paper some of the most modern applications of the CBR methodology are revised in order to obtain a global vision of the techniques used to develop functional systems. In order to analyze the systems, the four main phases of the CBR cycled are considered as the key elements to organize an application based on CBR

Multi-robot coordination using flexible setplays : applications in RoboCup's simulation and middle-size leagues

Tese de Doutoramento. Engenharia Informática. Faculdade de Engenharia. Universidade do Porto. 201

Dynamic behavior-based control and world-embedded knowledge for interactive artificial intelligence

Video game designers depend on artificial intelligence to drive player experience in modern games. Therefore it is critical that AI not only be fast and computation- ally inexpensive, but also easy to incorporate with the design process. We address the problem of building computationally inexpensive AI that eases the game de- sign process and provides strategic and tactical behavior comparable with current industry-standard techniques. Our central hypothesis is that behavior-based characters in games can exhibit effec- tive strategy and coordinate in teams through the use of knowledge embedded in the world and a new dynamic approach to behavior-based control that enables charac- ters to transfer behavioral knowledge. We use dynamic extensions for behavior-based subsumption and world-embedded knowledge to simplify and enhance game character intelligence. We find that the use of extended affordances to embed knowledge in the world can greatly reduce the effort required to build characters and AI engines while increasing the effectiveness of the behavior controllers. In addition, we find that the technique of multi-character affordances can provide a simple mechanism for enabling team coordination. We also show that reactive teaming, enabled by dynamic extensions to the subsumption architecture, is effective in creating large adaptable teams of characters. Finally, we show that the command policy for reactive teaming can be used to improve performance of reactive teams for tactical situations

Mobile Robots

The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task