1,224 research outputs found
Scalable Co-Optimization of Morphology and Control in Embodied Machines
Evolution sculpts both the body plans and nervous systems of agents together
over time. In contrast, in AI and robotics, a robot's body plan is usually
designed by hand, and control policies are then optimized for that fixed
design. The task of simultaneously co-optimizing the morphology and controller
of an embodied robot has remained a challenge. In psychology, the theory of
embodied cognition posits that behavior arises from a close coupling between
body plan and sensorimotor control, which suggests why co-optimizing these two
subsystems is so difficult: most evolutionary changes to morphology tend to
adversely impact sensorimotor control, leading to an overall decrease in
behavioral performance. Here, we further examine this hypothesis and
demonstrate a technique for "morphological innovation protection", which
temporarily reduces selection pressure on recently morphologically-changed
individuals, thus enabling evolution some time to "readapt" to the new
morphology with subsequent control policy mutations. We show the potential for
this method to avoid local optima and converge to similar highly fit
morphologies across widely varying initial conditions, while sustaining fitness
improvements further into optimization. While this technique is admittedly only
the first of many steps that must be taken to achieve scalable optimization of
embodied machines, we hope that theoretical insight into the cause of
evolutionary stagnation in current methods will help to enable the automation
of robot design and behavioral training -- while simultaneously providing a
testbed to investigate the theory of embodied cognition
Exploring the Modularity and Structure of Robots Evolved in Multiple Environments
Traditional techniques for the design of robots require human engineers to plan every aspect of the system, from body to controller. In contrast, the field of evolu- tionary robotics uses evolutionary algorithms to create optimized morphologies and neural controllers with minimal human intervention. In order to expand the capability of an evolved agent, it must be exposed to a variety of conditions and environments.
This thesis investigates the design and benefits of virtual robots which can reflect the structure and modularity in the world around them. I show that when a robot’s morphology and controller enable it to perceive each environment as a collection of independent components, rather than a monolithic entity, evolution only needs to optimize on a subset of environments in order to maintain performance in the overall larger environmental space. I explore previously unused methods in evolutionary robotics to aid in the evolution of modularity, including using morphological and neurological cost.
I utilize a tree morphology which makes my results generalizable to other mor- phologies while also allowing in depth theoretical analysis about the properties rel- evant to modularity in embodied agents. In order to better frame the question of modularity in an embodied context, I provide novel definitions of morphological and neurological modularity as well as create the sub-goal interference metric which mea- sures how much independence a robot exhibits with regards to environmental stimu- lus.
My work extends beyond evolutionary robotics and can be applied to the opti- mization of embodied systems in general as well as provides insight into the evolution of form in biological organisms
Exploring the effects of robotic design on learning and neural control
The ongoing deep learning revolution has allowed computers to outclass humans
in various games and perceive features imperceptible to humans during
classification tasks. Current machine learning techniques have clearly
distinguished themselves in specialized tasks. However, we have yet to see
robots capable of performing multiple tasks at an expert level. Most work in
this field is focused on the development of more sophisticated learning
algorithms for a robot's controller given a largely static and presupposed
robotic design. By focusing on the development of robotic bodies, rather than
neural controllers, I have discovered that robots can be designed such that
they overcome many of the current pitfalls encountered by neural controllers in
multitask settings. Through this discovery, I also present novel metrics to
explicitly measure the learning ability of a robotic design and its resistance
to common problems such as catastrophic interference.
Traditionally, the physical robot design requires human engineers to plan
every aspect of the system, which is expensive and often relies on human
intuition. In contrast, within the field of evolutionary robotics, evolutionary
algorithms are used to automatically create optimized designs, however, such
designs are often still limited in their ability to perform in a multitask
setting. The metrics created and presented here give a novel path to automated
design that allow evolved robots to synergize with their controller to improve
the computational efficiency of their learning while overcoming catastrophic
interference.
Overall, this dissertation intimates the ability to automatically design
robots that are more general purpose than current robots and that can perform
various tasks while requiring less computation.Comment: arXiv admin note: text overlap with arXiv:2008.0639
Improving Scalability of Evolutionary Robotics with Reformulation
Creating systems that can operate autonomously in complex environments is a challenge for contemporary engineering techniques. Automatic design methods offer a promising alternative, but so far they have not been able to produce agents that outperform manual designs. One such method is evolutionary robotics. It has been shown to be a robust and versatile tool for designing robots to perform simple tasks, but more challenging tasks at present remain out of reach of the method.
In this thesis I discuss and attack some problems underlying the scalability issues associated with the method. I present a new technique for evolving modular networks. I show that the performance of modularity-biased evolution depends heavily on the morphology of the robot’s body and present a new method for co-evolving morphology and modular control.
To be able to reason about the new technique I develop reformulation framework: a general way to describe and reason about metaoptimization approaches. Within this framework I describe a new heuristic for developing metaoptimization approaches that is based on the technique for co-evolving morphology and modularity. I validate the framework by applying it to a practical task of zero-g autonomous assembly of structures with a fleet of small robots.
Although this work focuses on the evolutionary robotics, methods and approaches developed within it can be applied to optimization problems in any domain
N-LIMB: Neural Limb Optimization for Efficient Morphological Design
A robot's ability to complete a task is heavily dependent on its physical
design. However, identifying an optimal physical design and its corresponding
control policy is inherently challenging. The freedom to choose the number of
links, their type, and how they are connected results in a combinatorial design
space, and the evaluation of any design in that space requires deriving its
optimal controller. In this work, we present N-LIMB, an efficient approach to
optimizing the design and control of a robot over large sets of morphologies.
Central to our framework is a universal, design-conditioned control policy
capable of controlling a diverse sets of designs. This policy greatly improves
the sample efficiency of our approach by allowing the transfer of experience
across designs and reducing the cost to evaluate new designs. We train this
policy to maximize expected return over a distribution of designs, which is
simultaneously updated towards higher performing designs under the universal
policy. In this way, our approach converges towards a design distribution
peaked around high-performing designs and a controller that is effectively
fine-tuned for those designs. We demonstrate the potential of our approach on a
series of locomotion tasks across varying terrains and show the discovery novel
and high-performing design-control pairs.Comment: For code and videos, see https://sites.google.com/ttic.edu/nlim
Locomotion through morphology, evolution and learning for legged and limbless robots
Mención Internacional en el título de doctorRobot locomotion is concerned with providing autonomous locomotion capabilities to mobile robots. Most current day robots feature some form of locomotion for navigating in their environment.
Modalities of robot locomotion includes: (i) aerial locomotion, (ii) terrestrial locomotion, and (iii) aquatic locomotion (on or under water). Three main forms of terrestrial locomotion are, legged locomotion, limbless locomotion and wheel-based locomotion. A Modular Robot (MR), on the other hand, is a robotic system composed of several independent unit modules, where, each module is a robot by itself. The objective in this thesis is to develop legged locomotion in a humanoid robot, as well as, limbless locomotion in modular robotic configurations. Taking inspiration from biology, robot locomotion from the perspective of robot’s morphology, through evolution, and through learning are investigated in this thesis.
Locomotion is one of the key distinguishing characteristics of a zoological organism. Almost all animal species, and even some plant species, produce some form of locomotion. In the past few years, robots have been “moving out” of the factory floor and research labs, and are becoming increasingly common in everyday life. So, providing stable and agile locomotion capabilities for robots to navigate a wide range of environments becomes pivotal. Developing locomotion in robots through biologically inspired methods, also facilitates furthering our understanding on how biological processes may function.
Connected modules in a configuration, exert force on each other as a result of interaction between each other and their environment. This phenomenon is studied and quantified, and then used as implicit communication between robot modules for producing locomotion coordination in MRs. Through this, a strong link between robot morphology and the gait that emerge in it is established.
A variety of locomotion controller, some periodic-function based and some morphology based, are developed for MR locomotion and bipedal gait generation. A hybrid Evolutionary
Algorithm (EA) is implemented for evolving gaits, both in simulation as well as in the real-world on a physical modular robotic configuration. Limbless gaits in MRs are also learnt by learning optimal control policies, through Reinforcement Learning (RL).En robótica, la locomoción trata de proporcionar capacidades de locomoción autónoma a robots móviles. La mayoría de los robots actuales tiene alguna forma de locomoción para navegar en su entorno. Los modos de locomoción robótica se pueden repartir entre: (i) locomoción aérea, (ii) locomoción terrestre, y (iii) locomoción acuática (sobre o bajo el agua). Las tres formas básicas de locomoción terrestre son la locomoción mediante piernas, la locomoción sin miembros, y la locomoción basada en ruedas. Un Robot Modular, por otra parte, es un sistema robótico compuesto por varios módulos independientes, donde cada módulo es un robot en sí mismo.
El objetivo de esta tesis es el desarrollo de la locomoción mediante piernas para un robot humanoide, así como el de la locomoción sin miembros para varias configuraciones de robots modulares. Inspirándose en la biología, también se investiga en esta tesis el desarrollo de la locomoción del robot según su morfología, gracias a técnicas de evolución y de aprendizaje.
La locomoción es una de las características distintivas de un organismo zoológico. Casi todas las especies animales, e incluso algunas especies de plantas, poseen algún tipo de locomoción. En los últimos años, los robots han “migrado” desde las fábricas y los laboratorios de investigación, y se están integrando cada vez más en nuestra vida diaria. Por estas razones, es crucial proporcionar capacidades de locomoción estables y ágiles a los robots para que puedan navegar por todo tipo de entornos. El uso de métodos de inspiración biológica para alcanzar esta meta también nos ayuda a entender mejor cómo pueden funcionar los procesos biológicos equivalentes.
En una configuración de módulos conectados, puesto que cada uno interacciona con su entorno, los módulos ejercen fuerza los unos sobre los otros. Este fenómeno se ha estudiado y cuantificado, y luego se ha usado como comunicación implícita entre los módulos para producir la coordinación en la locomoción de este robot. De esta manera, se establece un fuerte vínculo entre la morfología de un robot y el modo de andar que este desarrolla.
Se han desarrollado varios controladores de locomoción para robots modulares y robots bípedos, algunos basados en funciones periódicas, otros en la morfología del robot. Un algoritmo evolutivo híbrido se ha implementado para la evolución de locomociones, tanto en simulación como en el mundo real en una configuración física de robot modular. También se pueden generar locomociones sin miembros para robots modulares, determinando las políticas de control óptimo gracias a técnicas de aprendizaje por refuerzo.
Se presenta en primer lugar en esta tesis el estado del arte de la robótica modular, enfocándose en la locomoción de robots modulares, los controladores, la locomoción bípeda y la computación morfológica. A continuación se describen cinco configuraciones diferentes de robot modular que se utilizan en esta tesis, seguido de cuatro controladores de locomoción. Estos controladores son el controlador heterogéneo, el controlador basado en funciones periódicas, el controlador homogéneo y el controlador basado en la morfología del robot.
Se desarrolla como parte de este trabajo un controlador de locomoción lineal, periódico, basado en features, para la locomoción bípeda de robots humanoides. Los parámetros de control se ajustan primero a mano para reproducir un modelo cart-table, y el controlador se evalúa en un robot humanoide simulado. A continuación, gracias a un algoritmo evolutivo, la optimización de los parámetros de control permite desarrollar una locomoción sin modelo predeterminado.
Se desarrolla como parte de esta tesis un enfoque sobre algoritmos de Embodied Evolución, en otras palabras el uso de robots modulares físicos en la fase de evolución. La implementación material, la configuración experimental, y el Algoritmo Evolutivo implementado para Embodied Evolución, se explican detalladamente.
El trabajo también incluye una visión general de las técnicas de aprendizaje por refuerzo y de los Procesos de Decisión de Markov. A continuación se presenta un algoritmo popular de aprendizaje por refuerzo, llamado Q-Learning, y su adaptación para aprender locomociones de robots modulares. Se proporcionan una implementación del algoritmo de aprendizaje y la evaluación experimental de la locomoción generada.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Antonio Barrientos Cruz.- Secretario: Luis Santiago Garrido Bullón.- Vocal: Giuseppe Carbon
Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution
Throughout long history, natural species have learned to survive by evolving
their physical structures adaptive to the environment changes. In contrast,
current reinforcement learning (RL) studies mainly focus on training an agent
with a fixed morphology (e.g., skeletal structure and joint attributes) in a
fixed environment, which can hardly generalize to changing environments or new
tasks. In this paper, we optimize an RL agent and its morphology through
``morphology-environment co-evolution (MECE)'', in which the morphology keeps
being updated to adapt to the changing environment, while the environment is
modified progressively to bring new challenges and stimulate the improvement of
the morphology. This leads to a curriculum to train generalizable RL, whose
morphology and policy are optimized for different environments. Instead of
hand-crafting the curriculum, we train two policies to automatically change the
morphology and the environment. To this end, (1) we develop two novel and
effective rewards for the two policies, which are solely based on the learning
dynamics of the RL agent; (2) we design a scheduler to automatically determine
when to change the environment and the morphology. In experiments on two
classes of tasks, the morphology and RL policies trained via MECE exhibit
significantly better generalization performance in unseen test environments
than SOTA morphology optimization methods. Our ablation studies on the two MECE
policies further show that the co-evolution between the morphology and
environment is the key to the success
Co-optimising Robot Morphology and Controller in a Simulated Open-Ended Environment
Designing robots by hand can be costly and time consuming, especially if the
robots have to be created with novel materials, or be robust to internal or
external changes. In order to create robots automatically, without the need for
human intervention, it is necessary to optimise both the behaviour and the body
design of the robot. However, when co-optimising the morphology and controller
of a locomoting agent the morphology tends to converge prematurely, reaching a
local optimum. Approaches such as explicit protection of morphological
innovation have been used to reduce this problem, but it might also be possible
to increase exploration of morphologies using a more indirect approach. We
explore how changing the environment, where the agent locomotes, affects the
convergence of morphologies. The agents' morphologies and controllers are
co-optimised, while the environments the agents locomote in are evolved
open-endedly with the Paired Open-Ended Trailblazer (POET). We compare the
diversity, fitness and robustness of agents evolving in environments generated
by POET to agents evolved in handcrafted curricula of environments. Our agents
each contain of a population of individuals being evolved with a genetic
algorithm. This population is called the agent-population. We show that
agent-populations evolving in open-endedly evolving environments exhibit larger
morphological diversity than agent-populations evolving in hand crafted
curricula of environments. POET proved capable of creating a curriculum of
environments which encouraged both diversity and quality in the populations.
This suggests that POET may be capable of reducing premature convergence in
co-optimisation of morphology and controllers.Comment: 17 pages, 8 figure
- …