The Evolution of Complexity in Autonomous Robots

Evolutionary robotics–the use of evolutionary algorithms to automate the production of autonomous robots–has been an active area of research for two decades. However, previous work in this domain has been limited by the simplicity of the evolved robots and the task environments within which they are able to succeed. This dissertation aims to address these challenges by developing techniques for evolving more complex robots. Particular focus is given to methods which evolve not only the control policies of manually-designed robots, but instead evolve both the control policy and physical form of the robot. These techniques are presented along with their application to investigating previously unexplored relationships between the complexity of evolving robots and the task environments within which they evolve

Environmental Influence on the Evolution of Morphological Complexity in Machines

Whether, when, how, and why increased complexity evolves in biological populations is a longstanding open question. In this work we combine a recently developed method for evolving virtual organisms with an information-theoretic metric of morphological complexity in order to investigate how the complexity of morphologies, which are evolved for locomotion, varies across different environments. We first demonstrate that selection for locomotion results in the evolution of organisms with morphologies that increase in complexity over evolutionary time beyond what would be expected due to random chance. This provides evidence that the increase in complexity observed is a result of a driven rather than a passive trend. In subsequent experiments we demonstrate that morphologies having greater complexity evolve in complex environments, when compared to a simple environment when a cost of complexity is imposed. This suggests that in some niches, evolution may act to complexify the body plans of organisms while in other niches selection favors simpler body plans

Neuroevolution of Actively Controlled Virtual Characters

Master's thesis Information- and communication technology IKT590 - University of Agder 2017Physics-based character animation offer an attractive alternative to traditional animation techniques, however, physics-based approaches often struggle to incorporate active user control of these characters. This thesis suggests a different approach to the problem of actively controlled virtual characters. The proposed solution takes a neuroevolutionary approach, using HyperNEAT to evolve neural controllers for a simulated eight-legged character, a previously untested character morphology for this algorithm. Using these controllers this thesis aims to evaluate the robustness and responsiveness of a control strategy that changes between them based on simulated user input. The results show that HyperNEAT is quite capable of evolving long walking controllers for this character, but also suggests a need for further refinement when operated in tandem

Enabling Machine Science through Distributed Human Computing

Distributed human computing techniques have been shown to be effective ways of accessing the problem-solving capabilities of a large group of anonymous individuals over the World Wide Web. They have been successfully applied to such diverse domains as computer security, biology and astronomy. The success of distributed human computing in various domains suggests that it can be utilized for complex collaborative problem solving. Thus it could be used for machine science : utilizing machines to facilitate the vetting of disparate human hypotheses for solving scientific and engineering problems. In this thesis, we show that machine science is possible through distributed human computing methods for some tasks. By enabling anonymous individuals to collaborate in a way that parallels the scientific method -- suggesting hypotheses, testing and then communicating them for vetting by other participants -- we demonstrate that a crowd can together define robot control strategies, design robot morphologies capable of fast-forward locomotion and contribute features to machine learning models for residential electric energy usage. We also introduce a new methodology for empowering a fully automated robot design system by seeding it with intuitions distilled from the crowd. Our findings suggest that increasingly large, diverse and complex collaborations that combine people and machines in the right way may enable problem solving in a wide range of fields

Engineering evolutionary control for real-world robotic systems

Evolutionary Robotics (ER) is the field of study concerned with the application of evolutionary computation to the design of robotic systems. Two main issues have prevented ER from being applied to real-world tasks, namely scaling to complex tasks and the transfer of control to real-robot systems. Finding solutions to complex tasks is challenging for evolutionary approaches due to the bootstrap problem and deception. When the task goal is too difficult, the evolutionary process will drift in regions of the search space with equally low levels of performance and therefore fail to bootstrap. Furthermore, the search space tends to get rugged (deceptive) as task complexity increases, which can lead to premature convergence. Another prominent issue in ER is the reality gap. Behavioral control is typically evolved in simulation and then only transferred to the real robotic hardware when a good solution has been found. Since simulation is an abstraction of the real world, the accuracy of the robot model and its interactions with the environment is limited. As a result, control evolved in a simulator tends to display a lower performance in reality than in simulation. In this thesis, we present a hierarchical control synthesis approach that enables the use of ER techniques for complex tasks in real robotic hardware by mitigating the bootstrap problem, deception, and the reality gap. We recursively decompose a task into sub-tasks, and synthesize control for each sub-task. The individual behaviors are then composed hierarchically. The possibility of incrementally transferring control as the controller is composed allows transferability issues to be addressed locally in the controller hierarchy. Our approach features hybridity, allowing different control synthesis techniques to be combined. We demonstrate our approach in a series of tasks that go beyond the complexity of tasks where ER has been successfully applied. We further show that hierarchical control can be applied in single-robot systems and in multirobot systems. Given our long-term goal of enabling the application of ER techniques to real-world tasks, we systematically validate our approach in real robotic hardware. For one of the demonstrations in this thesis, we have designed and built a swarm robotic platform, and we show the first successful transfer of evolved and hierarchical control to a swarm of robots outside of controlled laboratory conditions.A Robótica Evolutiva (RE) é a área de investigação que estuda a aplicação de computação evolutiva na conceção de sistemas robóticos. Dois principais desafios têm impedido a aplicação da RE em tarefas do mundo real: a dificuldade em solucionar tarefas complexas e a transferência de controladores evoluídos para sistemas robóticos reais. Encontrar soluções para tarefas complexas é desafiante para as técnicas evolutivas devido ao bootstrap problem e à deception. Quando o objetivo é demasiado difícil, o processo evolutivo tende a permanecer em regiões do espaço de procura com níveis de desempenho igualmente baixos, e consequentemente não consegue inicializar. Por outro lado, o espaço de procura tende a enrugar à medida que a complexidade da tarefa aumenta, o que pode resultar numa convergência prematura. Outro desafio na RE é a reality gap. O controlo robótico é tipicamente evoluído em simulação, e só é transferido para o sistema robótico real quando uma boa solução tiver sido encontrada. Como a simulação é uma abstração da realidade, a precisão do modelo do robô e das suas interações com o ambiente é limitada, podendo resultar em controladores com um menor desempenho no mundo real. Nesta tese, apresentamos uma abordagem de síntese de controlo hierárquica que permite o uso de técnicas de RE em tarefas complexas com hardware robótico real, mitigando o bootstrap problem, a deception e a reality gap. Decompomos recursivamente uma tarefa em sub-tarefas, e sintetizamos controlo para cada subtarefa. Os comportamentos individuais são então compostos hierarquicamente. A possibilidade de transferir o controlo incrementalmente à medida que o controlador é composto permite que problemas de transferibilidade possam ser endereçados localmente na hierarquia do controlador. A nossa abordagem permite o uso de diferentes técnicas de síntese de controlo, resultando em controladores híbridos. Demonstramos a nossa abordagem em várias tarefas que vão para além da complexidade das tarefas onde a RE foi aplicada. Também mostramos que o controlo hierárquico pode ser aplicado em sistemas de um robô ou sistemas multirobô. Dado o nosso objetivo de longo prazo de permitir o uso de técnicas de RE em tarefas no mundo real, concebemos e desenvolvemos uma plataforma de robótica de enxame, e mostramos a primeira transferência de controlo evoluído e hierárquico para um exame de robôs fora de condições controladas de laboratório.This work has been supported by the Portuguese Foundation for Science and Technology (Fundação para a Ciência e Tecnologia) under the grants SFRH/BD/76438/2011, EXPL/EEI-AUT/0329/2013, and by Instituto de Telecomunicações under the grant UID/EEA/50008/2013

Learning From Geometry In Learning For Tactical And Strategic Decision Domains

Artificial neural networks (ANNs) are an abstraction of the low-level architecture of biological brains that are often applied in general problem solving and function approximation. Neuroevolution (NE), i.e. the evolution of ANNs, has proven effective at solving problems in a variety of domains. Information from the domain is input to the ANN, which outputs its desired actions. This dissertation presents a new NE algorithm called Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT), based on a novel indirect encoding of ANNs. The key insight in HyperNEAT is to make the algorithm aware of the geometry in which the ANNs are embedded and thereby exploit such domain geometry to evolve ANNs more effectively. The dissertation focuses on applying HyperNEAT to tactical and strategic decision domains. These domains involve simultaneously considering short-term tactics while also balancing long-term strategies. Board games such as checkers and Go are canonical examples of such domains; however, they also include real-time strategy games and military scenarios. The dissertation details three proposed extensions to HyperNEAT designed to work in tactical and strategic decision domains. The first is an action selector ANN architecture that allows the ANN to indicate its judgements on every possible action all at once. The second technique is called substrate extrapolation. It allows learning basic concepts at a low resolution, and then increasing the resolution to learn more advanced concepts. The iii final extension is geometric game-tree pruning, whereby HyperNEAT can endow the ANN the ability to focus on specific areas of a domain (such as a checkers board) that deserve more inspection. The culminating contribution is to demonstrate the ability of HyperNEAT with these extensions to play Go, a most challenging game for artificial intelligence, by combining HyperNEAT with UC

Training Neural Networks Through the Integration of Evolution and Gradient Descent

Neural networks have achieved widespread adoption due to both their applicability to a wide range of problems and their success relative to other machine learning algorithms. The training of neural networks is achieved through any of several paradigms, most prominently gradient-based approaches (including deep learning), but also through up-and-coming approaches like neuroevolution. However, while both of these neural network training paradigms have seen major improvements over the past decade, little work has been invested in developing algorithms that incorporate the advances from both deep learning and neuroevolution. This dissertation introduces two new algorithms that are steps towards the integration of gradient descent and neuroevolution for training neural networks. The first is (1) the Limited Evaluation Evolutionary Algorithm (LEEA), which implements a novel form of evolution where individuals are partially evaluated, allowing rapid learning and enabling the evolutionary algorithm to behave more like gradient descent. This conception provides a critical stepping stone to future algorithms that more tightly couple evolutionary and gradient descent components. The second major algorithm (2) is Divergent Discriminative Feature Accumulation (DDFA), which combines a neuroevolution phase, where features are collected in an unsupervised manner, with a gradient descent phase for fine tuning of the neural network weights. The neuroevolution phase of DDFA utilizes an indirect encoding and novelty search, which are sophisticated neuroevolution components rarely incorporated into gradient descent-based systems. Further contributions of this work that build on DDFA include (3) an empirical analysis to identify an effective distance function for novelty search in high dimensions and (4) the extension of DDFA for the purpose of discovering convolutional features. The results of these DDFA experiments together show that DDFA discovers features that are effective as a starting point for gradient descent, with significant improvement over gradient descent alone. Additionally, the method of collecting features in an unsupervised manner allows DDFA to be applied to domains with abundant unlabeled data and relatively sparse labeled data. This ability is highlighted in the STL-10 domain, where DDFA is shown to make effective use of unlabeled data

Design for an Increasingly Protean Machine

Data-driven, rather than hypothesis-driven, approaches to robot design are becoming increasingly widespread, but they remain narrowly focused on tuning the parameters of control software (neural network synaptic weights) inside an overwhelmingly static and presupposed body. Meanwhile, an efflorescence of new actuators and metamaterials continue to broaden the ways in which machines are free to move and morph, but they have yet to be adopted by useful robots because the design and control of metamorphosing body plans is extremely non-intuitive. This thesis unites these converging yet previously segregated technologies by automating the design of robots with physically malleable hardware, which we will refer to as protean machines, named after Proteus of Greek mythology. This thesis begins by proposing an ontology of embodied agents, their physical features, and their potential ability to purposefully change each one in space and time. A series of experiments are then documented in which increasingly more of these features (structure, shape, and material properties) were allowed to vary across increasingly more timescales (evolution, development, and physiology), and collectively optimized to facilitate adaptive behavior in a simulated physical environment. The utility of increasingly protean machines is demonstrated by a concomitant increase in both the performance and robustness of the final, optimized system. This holds true even if its ability to change is temporarily removed by fabricating the system in reality, or by “canalization”: the tendency for plasticity to be supplanted by good static traits (an inductive bias) for the current environment. Further, if physical flexibility is retained rather than canalized, it is shown how protean machines can, under certain conditions, achieve a form of hyper-robustness: the ability to self-edit their own anatomy to “undo” large deviations from the environments in which their control policy was originally optimized. Some of the designs that evolved in simulation were manufactured in reality using hundreds of highly deformable silicone building blocks, yielding shapeshifting robots. Others were built entirely out of biological tissues, derived from pluripotent Xenopus laevis stem cells, yielding computer-designed organisms (dubbed “xenobots”). Overall, the results shed unique light on questions about the evolution of development, simulation-to-reality transfer of physical artifacts, and the capacity for bioengineering new organisms with useful functions

Evolutionary Legged Robotics

Due to the technological advance, robotic systems become more and more interesting for industrial and home applications. Popular examples are given by robotic lawn mower, robot vacuum cleaner, and package drones. Beside the toy industry, legged robots are not as popular, although they have some clear advantages compared to wheeled systems. With their flexibility concerning the locomotion, they are able to adapt their walking pattern to different environments. For instance they can walk over obstacles and gaps or climb over rubble and stairs. Another possible advantage could be a redundancy for locomotion. A faulty motor in one limb could be compensated by other motors in the kinematic chain. As well, multiple failing legs can be compensated by an adapted walking pattern. Compared to this, the more complex mechatronic systems represent a major challenge to the construction and the control. This thesis is dedicated to the control of complex walking robots. Genetic algorithms are applied to generate walking patterns for different robots. The evolutionary development of walking patterns is done in a simulation software. Results of various approaches are transferred and tested on existing systems which have been developed at RIC/DFKI. Different robotic systems are used to evaluate the generality of the applied methods. Eventually, a method is developed that can be utilized, with a few system specific modifications, for a variety of legged robots. As basis for the development and investigation of several methods, software tools are designed to generalize the application of applying genetic algorithms to legged locomotion. These tools include a simulation environment, a behavior representation, a genetic algorithm and a learning and benchmark framework. The simulation environment is adapted to the behavior of real robotic systems via reference experiments. In addition, the simulation is extended by a foot contact model for loose surfaces. The evaluation of the genetic algorithm is done on several benchmark problems and compared to three existing algorithms. This thesis contributes to the state of the art in many areas. The developed methodology can easily be applied to several complex robotic systems due to its transferability. The genetic algorithm and the hierarchical behavior representation provide a new opportunity to control the generation of the offspring in an evolutionary process. In addition, the developed software tools are an important contribution for their respective research fields

A Practical Investigation into Achieving Bio-Plausibility in Evo-Devo Neural Microcircuits Feasible in an FPGA

Many researchers has conjectured, argued, or in some cases demonstrated, that bio-plausibility can bring about emergent properties such as adaptability, scalability, fault-tolerance, self-repair, reliability, and autonomy to bio-inspired intelligent systems. Evolutionary-developmental (evo-devo) spiking neural networks are a very bio-plausible mixture of such bio-inspired intelligent systems that have been proposed and studied by a few researchers. However, the general trend is that the complexity and thus the computational cost grow with the bio-plausibility of the system. FPGAs (Field- Programmable Gate Arrays) have been used and proved to be one of the flexible and cost efficient hardware platforms for research' and development of such evo-devo systems. However, mapping a bio-plausible evo-devo spiking neural network to an FPGA is a daunting task full of different constraints and trade-offs that makes it, if not infeasible, very challenging. This thesis explores the challenges, trade-offs, constraints, practical issues, and some possible approaches in achieving bio-plausibility in creating evolutionary developmental spiking neural microcircuits in an FPGA through a practical investigation along with a series of case studies. In this study, the system performance, cost, reliability, scalability, availability, and design and testing time and complexity are defined as measures for feasibility of a system and structural accuracy and consistency with the current knowledge in biology as measures for bio-plausibility. Investigation of the challenges starts with the hardware platform selection and then neuron, cortex, and evo-devo models and integration of these models into a whole bio-inspired intelligent system are examined one by one. For further practical investigation, a new PLAQIF Digital Neuron model, a novel Cortex model, and a new multicellular LGRN evo-devo model are designed, implemented and tested as case studies. Results and their implications for the researchers, designers of such systems, and FPGA manufacturers are discussed and concluded in form of general trends, trade-offs, suggestions, and recommendations