9 research outputs found
Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization
Genetic algorithms constitute a family of black-box optimization algorithms,
which take inspiration from the principles of biological evolution. While they
provide a general-purpose tool for optimization, their particular
instantiations can be heuristic and motivated by loose biological intuition. In
this work we explore a fundamentally different approach: Given a sufficiently
flexible parametrization of the genetic operators, we discover entirely new
genetic algorithms in a data-driven fashion. More specifically, we parametrize
selection and mutation rate adaptation as cross- and self-attention modules and
use Meta-Black-Box-Optimization to evolve their parameters on a set of diverse
optimization tasks. The resulting Learned Genetic Algorithm outperforms
state-of-the-art adaptive baseline genetic algorithms and generalizes far
beyond its meta-training settings. The learned algorithm can be applied to
previously unseen optimization problems, search dimensions & evaluation
budgets. We conduct extensive analysis of the discovered operators and provide
ablation experiments, which highlight the benefits of flexible module
parametrization and the ability to transfer (`plug-in') the learned operators
to conventional genetic algorithms.Comment: 14 pages, 31 figure
RoboCat: A Self-Improving Foundation Agent for Robotic Manipulation
The ability to leverage heterogeneous robotic experience from different
robots and tasks to quickly master novel skills and embodiments has the
potential to transform robot learning. Inspired by recent advances in
foundation models for vision and language, we propose a foundation agent for
robotic manipulation. This agent, named RoboCat, is a visual goal-conditioned
decision transformer capable of consuming multi-embodiment action-labelled
visual experience. This data spans a large repertoire of motor control skills
from simulated and real robotic arms with varying sets of observations and
actions. With RoboCat, we demonstrate the ability to generalise to new tasks
and robots, both zero-shot as well as through adaptation using only 100--1000
examples for the target task. We also show how a trained model itself can be
used to generate data for subsequent training iterations, thus providing a
basic building block for an autonomous improvement loop. We investigate the
agent's capabilities, with large-scale evaluations both in simulation and on
three different real robot embodiments. We find that as we grow and diversify
its training data, RoboCat not only shows signs of cross-task transfer, but
also becomes more efficient at adapting to new tasks