Engineering evolutionary control for real-world robotic systems

Evolutionary Robotics (ER) is the field of study concerned with the application of evolutionary computation to the design of robotic systems. Two main issues have prevented ER from being applied to real-world tasks, namely scaling to complex tasks and the transfer of control to real-robot systems. Finding solutions to complex tasks is challenging for evolutionary approaches due to the bootstrap problem and deception. When the task goal is too difficult, the evolutionary process will drift in regions of the search space with equally low levels of performance and therefore fail to bootstrap. Furthermore, the search space tends to get rugged (deceptive) as task complexity increases, which can lead to premature convergence. Another prominent issue in ER is the reality gap. Behavioral control is typically evolved in simulation and then only transferred to the real robotic hardware when a good solution has been found. Since simulation is an abstraction of the real world, the accuracy of the robot model and its interactions with the environment is limited. As a result, control evolved in a simulator tends to display a lower performance in reality than in simulation. In this thesis, we present a hierarchical control synthesis approach that enables the use of ER techniques for complex tasks in real robotic hardware by mitigating the bootstrap problem, deception, and the reality gap. We recursively decompose a task into sub-tasks, and synthesize control for each sub-task. The individual behaviors are then composed hierarchically. The possibility of incrementally transferring control as the controller is composed allows transferability issues to be addressed locally in the controller hierarchy. Our approach features hybridity, allowing different control synthesis techniques to be combined. We demonstrate our approach in a series of tasks that go beyond the complexity of tasks where ER has been successfully applied. We further show that hierarchical control can be applied in single-robot systems and in multirobot systems. Given our long-term goal of enabling the application of ER techniques to real-world tasks, we systematically validate our approach in real robotic hardware. For one of the demonstrations in this thesis, we have designed and built a swarm robotic platform, and we show the first successful transfer of evolved and hierarchical control to a swarm of robots outside of controlled laboratory conditions.A Robótica Evolutiva (RE) é a área de investigação que estuda a aplicação de computação evolutiva na conceção de sistemas robóticos. Dois principais desafios têm impedido a aplicação da RE em tarefas do mundo real: a dificuldade em solucionar tarefas complexas e a transferência de controladores evoluídos para sistemas robóticos reais. Encontrar soluções para tarefas complexas é desafiante para as técnicas evolutivas devido ao bootstrap problem e à deception. Quando o objetivo é demasiado difícil, o processo evolutivo tende a permanecer em regiões do espaço de procura com níveis de desempenho igualmente baixos, e consequentemente não consegue inicializar. Por outro lado, o espaço de procura tende a enrugar à medida que a complexidade da tarefa aumenta, o que pode resultar numa convergência prematura. Outro desafio na RE é a reality gap. O controlo robótico é tipicamente evoluído em simulação, e só é transferido para o sistema robótico real quando uma boa solução tiver sido encontrada. Como a simulação é uma abstração da realidade, a precisão do modelo do robô e das suas interações com o ambiente é limitada, podendo resultar em controladores com um menor desempenho no mundo real. Nesta tese, apresentamos uma abordagem de síntese de controlo hierárquica que permite o uso de técnicas de RE em tarefas complexas com hardware robótico real, mitigando o bootstrap problem, a deception e a reality gap. Decompomos recursivamente uma tarefa em sub-tarefas, e sintetizamos controlo para cada subtarefa. Os comportamentos individuais são então compostos hierarquicamente. A possibilidade de transferir o controlo incrementalmente à medida que o controlador é composto permite que problemas de transferibilidade possam ser endereçados localmente na hierarquia do controlador. A nossa abordagem permite o uso de diferentes técnicas de síntese de controlo, resultando em controladores híbridos. Demonstramos a nossa abordagem em várias tarefas que vão para além da complexidade das tarefas onde a RE foi aplicada. Também mostramos que o controlo hierárquico pode ser aplicado em sistemas de um robô ou sistemas multirobô. Dado o nosso objetivo de longo prazo de permitir o uso de técnicas de RE em tarefas no mundo real, concebemos e desenvolvemos uma plataforma de robótica de enxame, e mostramos a primeira transferência de controlo evoluído e hierárquico para um exame de robôs fora de condições controladas de laboratório.This work has been supported by the Portuguese Foundation for Science and Technology (Fundação para a Ciência e Tecnologia) under the grants SFRH/BD/76438/2011, EXPL/EEI-AUT/0329/2013, and by Instituto de Telecomunicações under the grant UID/EEA/50008/2013

Fault Recovery in Swarm Robotics Systems using Learning Algorithms

When faults occur in swarm robotic systems they can have a detrimental effect on collective behaviours, to the point that failed individuals may jeopardise the swarm's ability to complete its task. Although fault tolerance is a desirable property of swarm robotic systems, fault recovery mechanisms have not yet been thoroughly explored. Individual robots may suffer a variety of faults, which will affect collective behaviours in different ways, therefore a recovery process is required that can cope with many different failure scenarios. In this thesis, we propose a novel approach for fault recovery in robot swarms that uses Reinforcement Learning and Self-Organising Maps to select the most appropriate recovery strategy for any given scenario. The learning process is evaluated in both centralised and distributed settings. Additionally, we experimentally evaluate the performance of this approach in comparison to random selection of fault recovery strategies, using simulated collective phototaxis, aggregation and foraging tasks as case studies. Our results show that this machine learning approach outperforms random selection, and allows swarm robotic systems to recover from faults that would otherwise prevent the swarm from completing its mission. This work builds upon existing research in fault detection and diagnosis in robot swarms, with the aim of creating a fully fault-tolerant swarm capable of long-term autonomy

Gaining Insight into Determinants of Physical Activity using Bayesian Network Learning

Contains fulltext : 228326pre.pdf (preprint version ) (Open Access) Contains fulltext : 228326pub.pdf (publisher's version ) (Open Access)BNAIC/BeneLearn 202

Communication Free Robot Swarming

As the military use of unmanned aerial vehicles increases, a growing need for novel strategies to control these systems exists. One such method for controlling many unmanned aerial vehicles simultaneously is the through the use of swarm algorithms. This research explores a swarm robotic algorithm developed by Kadrovach implemented on Pioneer Robots in a real-world environment. An adaptation of his visual sensor is implemented using stereo vision as the primary method of sensing the environment. The swarm members are prohibited from explicitly communicating other than passively through the environment. The resulting implementation produces a communication free swarming algorithm. The algorithm is tested for performance of the visual sensor, performance of the algorithm against stationary targets, and finally, performance against dynamic targets. The results show expected behavior of the swarm model as implemented on the Pioneer robots providing a foundation for future research in swarm algorithms

Commande coopérative reconfigurable pour la recherche d'extremum

This thesis addresses the localisation of the maximum of an unknown spatial field in a delimited area. The search is performed by a multi-agent system composed of autonomous vehicles. The mission can be divided in two parts, the first one focuses on the estimation methods for optimisation, and the second one concerns the control law to move the fleet of agents.Two solutions have been proposed for the estimation part. The first one relies on a local search strategy that estimates the gradient of the unknown field and moves the agents along the gradient direction. The optimal sensor placement of the agents has been investigated and three criteria have been proposed to find the formation shape required for efficient estimation. Moreover, a sensor fault detection and isolation scheme using an adaptive threshold has been presented. The second estimation solution is a global search strategy based on a Kriging model of the field. A new sampling criterion is defined for the multi-agent system to locate the position of the global maximum while limiting the number of measurements and taking into account the agent dynamics.Both solutions provide a set of desired sampling positions to the agents. A distributed control law has been designed to guide the agents toward these locations. This control law is also used in the local approach to gather the agents in a desired formation and reconfigure it when a fault has been detected, following the optimal sensor placement analysis. The same control law has been adapted to reach the positions specified iteratively by the Kriging-based global search strategy.Le problème traité dans cette thèse concerne la recherche coopérative de la position du maximum d'un champ spatial initialement inconnu dans une zone prédéfinie avec un système multi-agent composé de véhicules autonomes. Ce problème se décompose en deux parties, la première s'intéresse aux méthodes d'estimation du champ utilisé pour l'optimisation, et la seconde concerne la conception de lois de commande pour le déplacement de la flotte d'agents.Deux solutions ont été proposées en ce qui concerne les méthodes d'estimation. La première approche s'appuie sur une stratégie de recherche locale qui cherche à estimer le gradient du champ inconnu dans le but de déplacer les agents selon cette direction. La problématique du placement optimal des agents a été abordée et trois critères ont été proposés afin de déterminer les formations qui fournissent la meilleure qualité d'estimation du champ. Une méthode coopérative de détection et d'identification de défauts de mesure utilisant un seuil adaptatif a également été proposée. La deuxième solution d'estimation s'appuie sur une stratégie de recherche globale du maximum. Le champ est modélisé par krigeage et la recherche est effectuée en utilisant les propriétés statistiques de ce méta-modèle. Un nouveau critère d'échantillonnage a été développé pour permettre au système multi-agent de localiser la position du maximum global tout en limitant le nombre de mesures et en tenant compte des contraintes dynamiques des véhicules.Les deux méthodes d'estimation fournissent les positions où effectuer les mesures du champ. Une loi de commande distribuée a donc été conçue pour permettre aux agents d'atteindre leurs positions désirées. Cette loi permet de reconfigurer la formation tel que recommandé par l'analyse de placement optimal lorsqu'un capteur est détecté comme défaillant dans le cas de l'estimation locale. La même loi de commande a été adaptée pour rallier les positions désignées itérativement par la stratégie de recherche globale

A Hormone Inspired System for On-line Adaptation in Swarm Robotic Systems

Individual robots, while providing the opportunity to develop a bespoke and specialised system, suffer in terms of performance when it comes to executing a large number of concurrent tasks. In some cases it is possible to drastically increase the speed of task execution by adding more agents to a system, however this comes at a cost. By mass producing relatively simple robots, costs can be kept low while still gaining the benefit of large scale multi-tasking. This approach sits at the core of swarm robotics. Robot swarms excel in tasks that rely heavily on their ability to multi-task, rather than applications that require bespoke actuation. Swarm suited tasks include: exploration, transportation or operation in dangerous environments. Swarms are particularly suited to hazardous environments due to the inherent expendability that comes with having multiple, decentralised agents. However, due to the variance in the environments a swarm may explore and their need to remain decentralised, a level of adaptability is required of them that can't be provided before a task begins. Methods of novel hormone-inspired robotic control are proposed in this thesis, offering solutions to these problems. These hormone inspired systems, or virtual hormones, provide an on-line method for adaptation that operates while a task is executed. These virtual hormones respond to environmental interactions. Then, through a mixture of decay and stimulant, provide values that grant contextually relevant information to individual robots. These values can then be used in decision making regarding parameters and behavioural changes. The hormone inspired systems presented in this thesis are found to be effective in mid-task adaptation, allowing robots to improve their effectiveness with minimal user interaction. It is also found that it is possible to deploy amalgamations of multiple hormone systems, controlling robots at multiple levels, enabling swarms to achieve strong, energy-efficient, performance

On the development and enhancement of artificial intelligence algorithms for swarm robots in real world applications

Swarm robotics is an area where using artificial intelligence (AI) can show a great deal of improvement. Obstacle avoidance, object detection, mapping and navigation are some the major algorithms required for successful execution of various tasks in the field of robotics. There is a challenge in applying these algorithms in a manner that swarm robots can use effectively. These five areas can be further researched to provide a platform for real world applications. This research aims to tackle the challenges involved in applying the aforementioned algorithms to swarm robotics and comparing the results with single robot systems. These techniques can be optimized by leveraging the advantage of swarm robots communication and scalability. The proposed algorithms were tested and validated using swarm robots along with profiling and simulations. For obstacle avoidance, two algorithms were devoloped. The first used a novel and modified force field method and the second used artificial neural networks (ANN). The results showed that the modified force field method performed better for static environments while ANNs worked better for dynamic environments. For object detection, the proposed algorithm uses an image classifier developed using ANN. The image classifier was trained to identify blocks of various colours using a convolutional neural network technique. This algorithm was then applied to swarm robotics using two proposed methods and results showed that multiple robots viewing objects from different angles provided better results as compared to single robot systems. This was validated with a 97% accuracy. In two dimension (2D) mapping, the proposed algorithm was developed using simultaneous localization and mapping (SLAM). The results showed that a single robot can require upto 3.5x more time for covering a given area compared to a swarm size of ten robots. This research shows a great deal of contribution in applying swarm robotics for surveilance purposes by showcasing the ability for swarm robotics to coordinate and execute the required task in an efficient time frame. The proposed three-dimension (3D) mapping algorithm used octomaps and occupancy grids to map out an image taken from a camera mounted on swarm robots. The images were obtained from various angles using multiple swarm robots. AI algorithms with a focus on swarm robotics are developed and enhanced for real world applications including fire-fighting, surveillance, fault analysis and construction. Results showed that swarm robots were able to complete a given task by up to six times faster as compared to a single robot. The overall contribution of this research lays a platform for further applications by showcasing the effectiveness of robotic algorithms in a swarm robot environment.Heriot-Watt University Fee Scholarshi

Exogenous Fault Detection in Swarm Robotic Systems

Swarm robotic systems comprise many individual robots, and exhibit a degree of innate fault tolerance due to this built-in redundancy. They are robust in the sense that the complete failure of individual robots will have little detrimental effect on a swarm's overall collective behaviour. However, it has recently been shown that partially failed individuals may be harmful, and cause problems that cannot be solved by simply adding more robots to the swarm. Instead, an active approach to dealing with failed individuals is required for a swarm to continue operation in the face of partial failures. This thesis presents a novel method of exogenous fault detection that allows robots to detect the presence of faults in each other, via the comparison of expected and observed behaviour. Each robot predicts the expected behaviour of its neighbours by simulating them online in an internal replica of the real world. This expected behaviour is then compared against observations of their true behaviour, and any significant discrepancy is detected as a fault. This work represents the first step towards a distributed fault detection, diagnosis, and recovery process that would afford robot swarms a high degree of fault tolerance, and facilitate long-term autonomy