1,406 research outputs found
Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks
Autonomous robots need to interact with unknown, unstructured and changing
environments, constantly facing novel challenges. Therefore, continuous online
adaptation for lifelong-learning and the need of sample-efficient mechanisms to
adapt to changes in the environment, the constraints, the tasks, or the robot
itself are crucial. In this work, we propose a novel framework for
probabilistic online motion planning with online adaptation based on a
bio-inspired stochastic recurrent neural network. By using learning signals
which mimic the intrinsic motivation signalcognitive dissonance in addition
with a mental replay strategy to intensify experiences, the stochastic
recurrent network can learn from few physical interactions and adapts to novel
environments in seconds. We evaluate our online planning and adaptation
framework on an anthropomorphic KUKA LWR arm. The rapid online adaptation is
shown by learning unknown workspace constraints sample-efficiently from few
physical interactions while following given way points.Comment: accepted in Neural Network
Adaptive motor control and learning in a spiking neural network realised on a mixed-signal neuromorphic processor
Neuromorphic computing is a new paradigm for design of both the computing
hardware and algorithms inspired by biological neural networks. The event-based
nature and the inherent parallelism make neuromorphic computing a promising
paradigm for building efficient neural network based architectures for control
of fast and agile robots. In this paper, we present a spiking neural network
architecture that uses sensory feedback to control rotational velocity of a
robotic vehicle. When the velocity reaches the target value, the mapping from
the target velocity of the vehicle to the correct motor command, both
represented in the spiking neural network on the neuromorphic device, is
autonomously stored on the device using on-chip plastic synaptic weights. We
validate the controller using a wheel motor of a miniature mobile vehicle and
inertia measurement unit as the sensory feedback and demonstrate online
learning of a simple 'inverse model' in a two-layer spiking neural network on
the neuromorphic chip. The prototype neuromorphic device that features 256
spiking neurons allows us to realise a simple proof of concept architecture for
the purely neuromorphic motor control and learning. The architecture can be
easily scaled-up if a larger neuromorphic device is available.Comment: 6+1 pages, 4 figures, will appear in one of the Robotics conference
Closed loop interactions between spiking neural network and robotic simulators based on MUSIC and ROS
In order to properly assess the function and computational properties of
simulated neural systems, it is necessary to account for the nature of the
stimuli that drive the system. However, providing stimuli that are rich and yet
both reproducible and amenable to experimental manipulations is technically
challenging, and even more so if a closed-loop scenario is required. In this
work, we present a novel approach to solve this problem, connecting robotics
and neural network simulators. We implement a middleware solution that bridges
the Robotic Operating System (ROS) to the Multi-Simulator Coordinator (MUSIC).
This enables any robotic and neural simulators that implement the corresponding
interfaces to be efficiently coupled, allowing real-time performance for a wide
range of configurations. This work extends the toolset available for
researchers in both neurorobotics and computational neuroscience, and creates
the opportunity to perform closed-loop experiments of arbitrary complexity to
address questions in multiple areas, including embodiment, agency, and
reinforcement learning
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Biological plastic neural networks are systems of extraordinary computational
capabilities shaped by evolution, development, and lifetime learning. The
interplay of these elements leads to the emergence of adaptive behavior and
intelligence. Inspired by such intricate natural phenomena, Evolved Plastic
Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed
plastic neural networks with a large variety of dynamics, architectures, and
plasticity rules: these artificial systems are composed of inputs, outputs, and
plastic components that change in response to experiences in an environment.
These systems may autonomously discover novel adaptive algorithms, and lead to
hypotheses on the emergence of biological adaptation. EPANNs have seen
considerable progress over the last two decades. Current scientific and
technological advances in artificial neural networks are now setting the
conditions for radically new approaches and results. In particular, the
limitations of hand-designed networks could be overcome by more flexible and
innovative solutions. This paper brings together a variety of inspiring ideas
that define the field of EPANNs. The main methods and results are reviewed.
Finally, new opportunities and developments are presented
A neural circuit for navigation inspired by C. elegans Chemotaxis
We develop an artificial neural circuit for contour tracking and navigation
inspired by the chemotaxis of the nematode Caenorhabditis elegans. In order to
harness the computational advantages spiking neural networks promise over their
non-spiking counterparts, we develop a network comprising 7-spiking neurons
with non-plastic synapses which we show is extremely robust in tracking a range
of concentrations. Our worm uses information regarding local temporal gradients
in sodium chloride concentration to decide the instantaneous path for foraging,
exploration and tracking. A key neuron pair in the C. elegans chemotaxis
network is the ASEL & ASER neuron pair, which capture the gradient of
concentration sensed by the worm in their graded membrane potentials. The
primary sensory neurons for our network are a pair of artificial spiking
neurons that function as gradient detectors whose design is adapted from a
computational model of the ASE neuron pair in C. elegans. Simulations show that
our worm is able to detect the set-point with approximately four times higher
probability than the optimal memoryless Levy foraging model. We also show that
our spiking neural network is much more efficient and noise-resilient while
navigating and tracking a contour, as compared to an equivalent non-spiking
network. We demonstrate that our model is extremely robust to noise and with
slight modifications can be used for other practical applications such as
obstacle avoidance. Our network model could also be extended for use in
three-dimensional contour tracking or obstacle avoidance
Bio-Inspired Autonomous Learning Algorithm With Application to Mobile Robot Obstacle Avoidance
Spiking Neural Networks (SNNs) are often considered the third generation of Artificial Neural Networks (ANNs), owing to their high information processing capability and the accurate simulation of biological neural network behaviors. Though the research for SNNs has been quite active in recent years, there are still some challenges to applying SNNs to various potential applications, especially for robot control. In this study, a biologically inspired autonomous learning algorithm based on reward modulated spike-timing-dependent plasticity is proposed, where a novel rewarding generation mechanism is used to generate the reward signals for both learning and decision-making processes. The proposed learning algorithm is evaluated by a mobile robot obstacle avoidance task and experimental results show that the mobile robot with the proposed algorithm exhibits a good learning ability. The robot can successfully avoid obstacles in the environment after some learning trials. This provides an alternative method to design and apply the bio-inspired robot with autonomous learning capability in the typical robotic task scenario
Learning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients
Artificial Neural Networks (ANNs) are currently being used as function
approximators in many state-of-the-art Reinforcement Learning (RL) algorithms.
Spiking Neural Networks (SNNs) have been shown to drastically reduce the energy
consumption of ANNs by encoding information in sparse temporal binary spike
streams, hence emulating the communication mechanism of biological neurons. Due
to their low energy consumption, SNNs are considered to be important candidates
as co-processors to be implemented in mobile devices. In this work, the use of
SNNs as stochastic policies is explored under an energy-efficient
first-to-spike action rule, whereby the action taken by the RL agent is
determined by the occurrence of the first spike among the output neurons. A
policy gradient-based algorithm is derived considering a Generalized Linear
Model (GLM) for spiking neurons. Experimental results demonstrate the
capability of online trained SNNs as stochastic policies to gracefully trade
energy consumption, as measured by the number of spikes, and control
performance. Significant gains are shown as compared to the standard approach
of converting an offline trained ANN into an SNN.Comment: Submitted for conference publicatio
- …