376 research outputs found
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
A long-term goal of AI is to produce agents that can learn a diversity of
skills throughout their lifetimes and continuously improve those skills via
experience. A longstanding obstacle towards that goal is catastrophic
forgetting, which is when learning new information erases previously learned
information. Catastrophic forgetting occurs in artificial neural networks
(ANNs), which have fueled most recent advances in AI. A recent paper proposed
that catastrophic forgetting in ANNs can be reduced by promoting modularity,
which can limit forgetting by isolating task information to specific clusters
of nodes and connections (functional modules). While the prior work did show
that modular ANNs suffered less from catastrophic forgetting, it was not able
to produce ANNs that possessed task-specific functional modules, thereby
leaving the main theory regarding modularity and forgetting untested. We
introduce diffusion-based neuromodulation, which simulates the release of
diffusing, neuromodulatory chemicals within an ANN that can modulate (i.e. up
or down regulate) learning in a spatial region. On the simple diagnostic
problem from the prior work, diffusion-based neuromodulation 1) induces
task-specific learning in groups of nodes and connections (task-specific
localized learning), which 2) produces functional modules for each subtask, and
3) yields higher performance by eliminating catastrophic forgetting. Overall,
our results suggest that diffusion-based neuromodulation promotes task-specific
localized learning and functional modularity, which can help solve the
challenging, but important problem of catastrophic forgetting
Combating catastrophic forgetting with developmental compression
Generally intelligent agents exhibit successful behavior across problems in
several settings. Endemic in approaches to realize such intelligence in
machines is catastrophic forgetting: sequential learning corrupts knowledge
obtained earlier in the sequence, or tasks antagonistically compete for system
resources. Methods for obviating catastrophic forgetting have sought to
identify and preserve features of the system necessary to solve one problem
when learning to solve another, or to enforce modularity such that minimally
overlapping sub-functions contain task specific knowledge. While successful,
both approaches scale poorly because they require larger architectures as the
number of training instances grows, causing different parts of the system to
specialize for separate subsets of the data. Here we present a method for
addressing catastrophic forgetting called developmental compression. It
exploits the mild impacts of developmental mutations to lessen adverse changes
to previously-evolved capabilities and `compresses' specialized neural networks
into a generalized one. In the absence of domain knowledge, developmental
compression produces systems that avoid overt specialization, alleviating the
need to engineer a bespoke system for every task permutation and suggesting
better scalability than existing approaches. We validate this method on a robot
control problem and hope to extend this approach to other machine learning
domains in the future
Exploring the Modularity and Structure of Robots Evolved in Multiple Environments
Traditional techniques for the design of robots require human engineers to plan every aspect of the system, from body to controller. In contrast, the field of evolu- tionary robotics uses evolutionary algorithms to create optimized morphologies and neural controllers with minimal human intervention. In order to expand the capability of an evolved agent, it must be exposed to a variety of conditions and environments.
This thesis investigates the design and benefits of virtual robots which can reflect the structure and modularity in the world around them. I show that when a robot’s morphology and controller enable it to perceive each environment as a collection of independent components, rather than a monolithic entity, evolution only needs to optimize on a subset of environments in order to maintain performance in the overall larger environmental space. I explore previously unused methods in evolutionary robotics to aid in the evolution of modularity, including using morphological and neurological cost.
I utilize a tree morphology which makes my results generalizable to other mor- phologies while also allowing in depth theoretical analysis about the properties rel- evant to modularity in embodied agents. In order to better frame the question of modularity in an embodied context, I provide novel definitions of morphological and neurological modularity as well as create the sub-goal interference metric which mea- sures how much independence a robot exhibits with regards to environmental stimu- lus.
My work extends beyond evolutionary robotics and can be applied to the opti- mization of embodied systems in general as well as provides insight into the evolution of form in biological organisms
Improving Scalability of Evolutionary Robotics with Reformulation
Creating systems that can operate autonomously in complex environments is a challenge for contemporary engineering techniques. Automatic design methods offer a promising alternative, but so far they have not been able to produce agents that outperform manual designs. One such method is evolutionary robotics. It has been shown to be a robust and versatile tool for designing robots to perform simple tasks, but more challenging tasks at present remain out of reach of the method.
In this thesis I discuss and attack some problems underlying the scalability issues associated with the method. I present a new technique for evolving modular networks. I show that the performance of modularity-biased evolution depends heavily on the morphology of the robot’s body and present a new method for co-evolving morphology and modular control.
To be able to reason about the new technique I develop reformulation framework: a general way to describe and reason about metaoptimization approaches. Within this framework I describe a new heuristic for developing metaoptimization approaches that is based on the technique for co-evolving morphology and modularity. I validate the framework by applying it to a practical task of zero-g autonomous assembly of structures with a fleet of small robots.
Although this work focuses on the evolutionary robotics, methods and approaches developed within it can be applied to optimization problems in any domain
- …