Distributed Lazy Q-learning for Cooperative Mobile Robots

International audienceCompared to single robot learning, cooperative learning adds the challenge of a much larger search space (combined individual search spaces), awareness of other team members, and also the synthesis of the individual behaviors with respect to the task given to the group. Over the years, reinforcement learning has emerged as the main learning approach in autonomous robotics, and lazy learning has become the leading bias, allowing the reduction of the time required by an experiment to the time needed to test the learned behavior performance. These two approaches have been combined together in what is now called lazy Q-learning, a very efficient single robot learning paradigm. We propose a derivation of this learning to team of robots : the «pessimistic» algorithm able to compute for each team member a lower bound of the utility of executing an action in a given situation. We use the cooperative multi-robot observation of multiple moving targets (CMOMMT) application as an illustrative example, and study the efficiency of the Pessimistic Algorithm in its task of inducing learning of cooperation

Algoritmo bioinspirado a redes de robots para la asistencia en operaciones de busqueda y rescate

ilustraciones, diagramas, fotografíasThis thesis proposes a bio-inspired algorithm for robot networks assisting in the operations of search and rescue scenarios. We consider ants as social animals to study and abstract beha- viors that can be useful in the framework of search and rescue using robots. We consider three main topics to address when using robots to assist rescuers. First, the exploration and mapping of the disaster zones. For this, we consider the mecha- nisms and interactions of ants to explore their environment, look for food, avoid predators, and explore better places to establish a nest. Then, we deploy robots to explore the en- vironment and discourage robots from entering regions other robots have explored using pheromones as markers for the robots. We also abstract the randomness ants use to explore and implement a Q-learning algorithm that allows robots to explore unvisited regions. Second, the navigation and victim detection. Once the environment has been explored, we vi use Reynolds rules to allow the navigation of robots to create cohesion, attraction to target goals, and repulsion to obstacles and inter-agent collisions. Then, we use a neural network to determine whether what robots are detecting is a victim. Lastly, we use a consensus-like approach to classify victims or no victims based on distributed information. Lastly, ants have been famous for carrying loads that surpass their size and payload capacity by cooperating. We consider quadrotors to carry loads cooperatively that can be medical supplies or victims in search and rescue (Texto tomado de la fuente)Esta tesis propone un algoritmo bioinspirado para redes de robots que asisten en las operaciones de escenarios de busqueda y rescate. Consideramos a las hormigas como animales sociales para estudiar y abstraer comportamientos que pueden ser utiles en el marco de la busqueda y rescate mediante robots. Consideramos tres temas principales para abordar cuando se utilizan robots para ayudar a los rescatistas. Primero, la exploracion y mapeo de las zonas de desastre. Para esto, consideramos los mecanismos e interacciones de las hormigas para explorar su entorno, buscar comida, evitar depredadores y explorar mejores lugares para establecer un nido. Luego, desplegamos robots para explorar el entorno y disuadimos a los robots de ingresar a regiones que otros robots han explorado usando feromonas como marcadores para los robots. Tambien abstraemos la aleatoriedad que usan las hormigas para explorar e implementar un algoritmo Q-learning que permite a los robots explorar regiones no visitadas. En segundo lugar, la navegacion y deteccion de vıctimas. Una vez que se ha explorado el entorno, usamos las reglas de Reynolds para permitir que la navegacion de los robots cree cohesion, atraccion hacia los objetivos y repulsion hacia los obstaculos y las colisiones entre agentes. Luego, usamos una red neuronal para determinar si lo que detectan los robots es una vıctima. Por ultimo, utilizamos un enfoque de consenso para clasificar a las vıctimas o no vıctimas en funcion de la informacion distribuida. Por ultimo, las hormigas han sido famosas por llevar cargas que superan su tamano y capacidad de carga al cooperar. Consideramos quadrotors para transportar cargas de manera cooperativa que pueden ser suministros medicos o vıctimas en busqueda y rescate.MaestríaMagister en Ingenieria - Automatizacion IndustrialRobotic

Human aware robot navigation

Abstract. Human aware robot navigation refers to the navigation of a robot in an environment shared with humans in such a way that the humans should feel comfortable, and natural with the presence of the robot. On top of that, the robot navigation should comply with the social norms of the environment. The robot can interact with humans in the environment, such as avoiding them, approaching them, or following them. In this thesis, we specifically focus on the approach behavior of the robot, keeping the other use cases still in mind. Studying and analyzing how humans move around other humans gives us the idea about the kind of navigation behaviors that we expect the robots to exhibit. Most of the previous research does not focus much on understanding such behavioral aspects while approaching people. On top of that, a straightforward mathematical modeling of complex human behaviors is very difficult. So, in this thesis, we proposed an Inverse Reinforcement Learning (IRL) framework based on Guided Cost Learning (GCL) to learn these behaviors from demonstration. After analyzing the CongreG8 dataset, we found that the incoming human tends to make an O-space (circle) with the rest of the group. Also, the approaching velocity slows down when the approaching human gets closer to the group. We utilized these findings in our framework that can learn the optimal reward and policy from the example demonstrations and imitate similar human motion

Mobile Robots

The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic