Apprentissage Intelligent des Robots Mobiles dans la Navigation Autonome

Modern robots are designed for assisting or replacing human beings to perform complicated planning and control operations, and the capability of autonomous navigation in a dynamic environment is an essential requirement for mobile robots. In order to alleviate the tedious task of manually programming a robot, this dissertation contributes to the design of intelligent robot control to endow mobile robots with a learning ability in autonomous navigation tasks. First, we consider the robot learning from expert demonstrations. A neural network framework is proposed as the inference mechanism to learn a policy offline from the dataset extracted from experts. Then we are interested in the robot self-learning ability without expert demonstrations. We apply reinforcement learning techniques to acquire and optimize a control strategy during the interaction process between the learning robot and the unknown environment. A neural network is also incorporated to allow a fast generalization, and it helps the learning to converge in a number of episodes that is greatly smaller than the traditional methods. Finally, we study the robot learning of the potential rewards underneath the states from optimal or suboptimal expert demonstrations. We propose an algorithm based on inverse reinforcement learning. A nonlinear policy representation is designed and the max-margin method is applied to refine the rewards and generate an optimal control policy. The three proposed methods have been successfully implemented on the autonomous navigation tasks for mobile robots in unknown and dynamic environments.Les robots modernes sont appelés à effectuer des opérations ou tâches complexes et la capacité de navigation autonome dans un environnement dynamique est un besoin essentiel pour les robots mobiles. Dans l’objectif de soulager de la fastidieuse tâche de préprogrammer un robot manuellement, cette thèse contribue à la conception de commande intelligente afin de réaliser l’apprentissage des robots mobiles durant la navigation autonome. D’abord, nous considérons l’apprentissage des robots via des démonstrations d’experts. Nous proposons d’utiliser un réseau de neurones pour apprendre hors-ligne une politique de commande à partir de données utiles extraites d’expertises. Ensuite, nous nous intéressons à l’apprentissage sans démonstrations d’experts. Nous utilisons l’apprentissage par renforcement afin que le robot puisse optimiser une stratégie de commande pendant le processus d’interaction avec l’environnement inconnu. Un réseau de neurones est également incorporé et une généralisation rapide permet à l’apprentissage de converger en un certain nombre d’épisodes inférieur à la littérature. Enfin, nous étudions l’apprentissage par fonction de récompenses potentielles compte rendu des démonstrations d’experts optimaux ou non-optimaux. Nous proposons un algorithme basé sur l’apprentissage inverse par renforcement. Une représentation non-linéaire de la politique est désignée et la méthode du max-margin est appliquée permettant d’affiner les récompenses et de générer la politique de commande. Les trois méthodes proposées sont évaluées sur des robots mobiles afin de leurs permettre d’acquérir les compétences de navigation autonome dans des environnements dynamiques et inconnu

Mobile Robots

The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations

Reactive with tags classifier system applied to real robot navigation

7th IEEE International Conference on Emerging Technologies and Factory Automation. Barcelona, 18-21 October 1999.A reactive with tags classifier system (RTCS) is a special classifier system. This system combines the execution capabilities of symbolic systems and the learning capabilities of genetic algorithms. A RTCS is able to learn symbolic rules that allow to generate sequence of actions, chaining rules among different time instants, and react to new environmental situations, considering the last environmental situation to take a decision. The capacity of RTCS to learn good rules has been prove in robotics navigation problem. Results show the suitability of this approximation to the navigation problem and the coherence of extracted rules

Efficient and secure real-time mobile robots cooperation using visual servoing

This paper deals with the challenging problem of navigation in formation of mobiles robots fleet. For that purpose, a secure approach is used based on visual servoing to control velocities (linear and angular) of the multiple robots. To construct our system, we develop the interaction matrix which combines the moments in the image with robots velocities and we estimate the depth between each robot and the targeted object. This is done without any communication between the robots which eliminate the problem of the influence of each robot errors on the whole. For a successful visual servoing, we propose a powerful mechanism to execute safely the robots navigation, exploiting a robot accident reporting system using raspberry Pi3. In addition, in case of problem, a robot accident detection reporting system testbed is used to send an accident notification, in the form of a specifical message. Experimental results are presented using nonholonomic mobiles robots with on-board real time cameras, to show the effectiveness of the proposed method

Resilient middleware for a multi-robot team

Tese de mestrado em Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2010Actualmente, equipas de robôs móveis intervém em diversos contextos e ambientes onde a intervenção humana é perigosa ou mesmo impossível, podemos mencionar como exemplo a vigilância de espaços físicos, como zonas militares ou nucleares. Devido à crescente complexidade inserida nos seus sistemas, esses robôs ficam mais poderosos mas paradoxalmente mais susceptíveis a falhas de hardware e software. Além disso, a incerteza na comunicação wireless pode privá-los temporariamente do seu suporte de informação remoto. Este tipo de problema pode ser causado pelo alcance limitado do emissor wireless e pelas zonas de sombra criadas pelo terreno. Por todas essas razões, desenhar arquitecturas capazes de oferecer mais resiliência para controlo das aplicações, tornou-se um verdadeiro desafio. Este documento aborda um motor cooperativo e resiliente para equipas de robôs que lhes permite partilharem uma vista comum e lidar com novos eventos de uma forma fiável e resiliente. Este middleware tem como função estabelecer a guarda de uma qualquer zona física e detectar eventos inabituais como os intrusos. Neste ultimo caso, um robô tem que encontrar uma maneira de bloquear o intruso para o impedir de fugir. O sistema apoia-se em duas características chave, a primeira é uma camada de controlo baseado em dois sub-módulos de controlo, o payload e o wormhole, a segunda é uma arquitectura baseada em eventos que executam tarefas do payload. Em relação à camada de controlo, o payload pode ser complexo e acede à informação partilhada pelos robôs enquanto que o wormhole é confiável mas apenas utiliza a informação local. O payload utiliza uma estrutura de dados chamada “promessa” na qual fornece o deadline correspondente ao momento mais tarde onde deve enviar a próxima promessa. No caso de receber esta promessa depois do deadline, o wormhole considera que o payload falhou, toma o controlo e executa as tarefas criticas no lugar do payload. Os eventos são propagados às traves de uma estrutura em forma de alvor, da raiz até as folhas. Cada folha do alvor é um módulo que pode ser executado e produz eventos. A produção dos eventos no alvor pode ser assimilado a uma reacção em cadeia. Durante o ciclo dos eventos as traves do alvor não são possíveis, o que permite evitar as reacções não controladas e garantem assim a estabilidade do sistema. A juntar a essa arquitectura, propomos também neste documento alguns mecanismos de sincronizações resilientes, para manter uma vista coerente num mundo ou de navegação para dar ao robô a possibilidade de se mover no mundo e de encontrar o melhor caminho. Guardar uma vista homogénea do mundo é um ponto fundamental que pode não ser fácil em caso de uma reunião de dois grupos. Introduzimos três implementações de middleware, uma versão simulada usada para validar arquitectura e testar a sincronização dos algoritmos num ambiente multi-robô, uma versão móvel apontada para ser implementado em plataformas de hardware compostas por robôs móveis reais e finalmente uma versão de posição capaz de comunicar com robôs móveis, recolher informação e enviar ordens remotas.Nowadays, teams of mobile robots are involved in many contexts and environments where human intervention would be risky or even impossible, we can mention the surveillance of physical areas as military zones or nuclear plants. Due to increasing complexity in their embedded systems, these robots become more powerful but paradoxically more susceptible to face a hardware or software failure. What’s more, unreliability in the wireless communication could deprive them temporally of their remote information support. For all these reasons, designing architectures able to offer more resilience to the control application has become a real challenge. In this document, we present a middleware architecture for the robots to share a common view and to handle new events in a safe and resilient way. The system relies on two key features, first a control layer based on two sub-modules, the payload and the wormhole, and secondly a cycle-proof event-based architecture used to run critical tasks in the payload. Regarding the control layer, the payload could be complex and has access to information shared among robots, while the wormhole is reliable but only uses local information. The wormhole controls the timely execution of the critical tasks by the payload. In case of timing failure, the wormhole takes control and runs these tasks in place of the payload. In addition to this architecture, we propose as well in this document some resilient synchronization mechanisms to maintain a coherent view of the world when two groups of robots are merging. We introduce three implementations of the middleware, a simulation version used to validate the architecture and test the synchronization algorithms in a multi-robot environment, a mobile version aimed to be ported to hardware platforms composed by real mobile robots and finally a station version able to communicate with mobiles, collect information and send remote orders

RTCS: a reactive with tags classifier system

In this work, a new Classifier System is proposed (CS). The system, a Reactive with Tags Classifier System (RTCS), is able to take into account environmental situations in intermediate decisions. CSs are special production systems, where conditions and actions are codified in order to learn new rules by means of Genetic Algorithms (GA). The RTCS has been designed to generate sequences of actions like the traditional classifier systems, but RTCS also has the capability of chaining rules among different time instants and reacting to new environmental situations, considering the last environmental situation to take a decision. In addition to the capability to react and generate sequences of actions, the design of a new rule codification allows the evolution of groups of specialized rules. This new codification is based on the inclusion of several bits, named tags, in conditions and actions, which evolve by means of GA. RTCS has been tested in robotic navigation. Results show the suitability of this approximation to the navigation problem and the coherence of tag values in rules classification.Publicad

Will the Driver Seat Ever Be Empty?

Self-driving technologies have matured and improved to the point that, in the past few years, self-driving cars have been able to safely drive an impressive number of kilometers. It should be noted though that, in all cases, the driver seat was never empty: a human driver was behind the wheel, ready to take over whenever the situation dictated it. This is an interesting paradox since the point of a self-driving car is to remove the most unreliable part of the car, namely the human driver. So, the question naturally arises: will the driver seat ever be empty? Besides legal liability issues, the answer to that question may lie in our ability to improve the self-driving technologies to the point that the human driver can safely be removed from the driving loop altogether. However, things are not that simple. Motion safety, i.e. the ability to avoid collisions, is the critical aspect concerning self-driving cars and autonomous vehicles in general. Before letting self-driving cars transport people around (and move among them) in a truly autonomous way, it is crucial to assess their ability to avoid collision, and to seek to characterize the levels of motion safety that can be achieved and the conditions under which they can be guaranteed. All these issues are explored in this article.Les technologies de conduite automatique ont mûries et se sont améliorées au point que, au cours des dernières années, les voitures automatiques ont été en mesure de conduire en toute sécurité un nombre impressionnant de kilomètres. Il convient cependant de noter que, dans tous les cas, le siège du conducteur n'était jamais vide : un conducteur humain était au volant, prêt à prendre le relais dès que la situation dictée. C'est un paradoxe intéressant car le point d'une voiture automatique est d'enlever la partie la plus sensible de la voiture, à savoir le conducteur humain. Ainsi, la question se pose naturellement: le siège du conducteur sera t'il vide un jour? Outre les questions de responsabilité juridique, la réponse à cette question réside peut-être dans notre capacité à améliorer les technologies de la conduite automatique, au point que le pilote humain peut en toute sécurité être retiré de la boucle de conduite. Toutefois, les choses ne sont pas aussi simple que cela. La sécurité de mouvement, i.e. la capacité à éviter les collisions, est l'aspect critique à l'égard de voitures automatiques et les véhicules autonomes en général. Avant de laisser les voitures automatiques transporter des personnes (et se déplacer parmi eux) d'une manière réellement autonome, il est crucial d'évaluer leur capacité à éviter la collision, et de chercher à caractériser les niveaux de sécurité de mouvement qui peuvent être atteints et les conditions dans lesquelles elles peuvent être garanties. Toutes ces questions sont examinées dans cet article