386 research outputs found
The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System
Natural evolution has produced a tremendous diversity of functional
organisms. Many believe an essential component of this process was the
evolution of evolvability, whereby evolution speeds up its ability to innovate
by generating a more adaptive pool of offspring. One hypothesized mechanism for
evolvability is developmental canalization, wherein certain dimensions of
variation become more likely to be traversed and others are prevented from
being explored (e.g. offspring tend to have similarly sized legs, and mutations
affect the length of both legs, not each leg individually). While ubiquitous in
nature, canalization almost never evolves in computational simulations of
evolution. Not only does that deprive us of in silico models in which to study
the evolution of evolvability, but it also raises the question of which
conditions give rise to this form of evolvability. Answering this question
would shed light on why such evolvability emerged naturally and could
accelerate engineering efforts to harness evolution to solve important
engineering challenges. In this paper we reveal a unique system in which
canalization did emerge in computational evolution. We document that genomes
entrench certain dimensions of variation that were frequently explored during
their evolutionary history. The genetic representation of these organisms also
evolved to be highly modular and hierarchical, and we show that these
organizational properties correlate with increased fitness. Interestingly, the
type of computational evolutionary experiment that produced this evolvability
was very different from traditional digital evolution in that there was no
objective, suggesting that open-ended, divergent evolutionary processes may be
necessary for the evolution of evolvability.Comment: SI can be found at: http://www.evolvingai.org/files/SI_0.zi
Lifelike Agility and Play on Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models
Summarizing knowledge from animals and human beings inspires robotic
innovations. In this work, we propose a framework for driving legged robots act
like real animals with lifelike agility and strategy in complex environments.
Inspired by large pre-trained models witnessed with impressive performance in
language and image understanding, we introduce the power of advanced deep
generative models to produce motor control signals stimulating legged robots to
act like real animals. Unlike conventional controllers and end-to-end RL
methods that are task-specific, we propose to pre-train generative models over
animal motion datasets to preserve expressive knowledge of animal behavior. The
pre-trained model holds sufficient primitive-level knowledge yet is
environment-agnostic. It is then reused for a successive stage of learning to
align with the environments by traversing a number of challenging obstacles
that are rarely considered in previous approaches, including creeping through
narrow spaces, jumping over hurdles, freerunning over scattered blocks, etc.
Finally, a task-specific controller is trained to solve complex downstream
tasks by reusing the knowledge from previous stages. Enriching the knowledge
regarding each stage does not affect the usage of other levels of knowledge.
This flexible framework offers the possibility of continual knowledge
accumulation at different levels. We successfully apply the trained multi-level
controllers to the MAX robot, a quadrupedal robot developed in-house, to mimic
animals, traverse complex obstacles, and play in a designed challenging
multi-agent Chase Tag Game, where lifelike agility and strategy emerge on the
robots. The present research pushes the frontier of robot control with new
insights on reusing multi-level pre-trained knowledge and solving highly
complex downstream tasks in the real world
Learning to stop: a unifying principle for legged locomotion in varying environments.
Evolutionary studies have unequivocally proven the transition of living organisms from water to land. Consequently, it can be deduced that locomotion strategies must have evolved from one environment to the other. However, the mechanism by which this transition happened and its implications on bio-mechanical studies and robotics research have not been explored in detail. This paper presents a unifying control strategy for locomotion in varying environments based on the principle of 'learning to stop'. Using a common reinforcement learning framework, deep deterministic policy gradient, we show that our proposed learning strategy facilitates a fast and safe methodology for transferring learned controllers from the facile water environment to the harsh land environment. Our results not only propose a plausible mechanism for safe and quick transition of locomotion strategies from a water to land environment but also provide a novel alternative for safer and faster training of robots
- …