464 research outputs found
Neural Expectation Maximization
Many real world tasks such as reasoning and physical interaction require
identification and manipulation of conceptual entities. A first step towards
solving these tasks is the automated discovery of distributed symbol-like
representations. In this paper, we explicitly formalize this problem as
inference in a spatial mixture model where each component is parametrized by a
neural network. Based on the Expectation Maximization framework we then derive
a differentiable clustering method that simultaneously learns how to group and
represent individual entities. We evaluate our method on the (sequential)
perceptual grouping task and find that it is able to accurately recover the
constituent objects. We demonstrate that the learned representations are useful
for next-step prediction.Comment: Accepted to NIPS 201
Neural Correlates of Social Behavior in Mushroom Body Extrinsic Neurons of the Honeybee Apis mellifera
The social behavior of honeybees (Apis mellifera) has been extensively investigated, but little is known about its neuronal correlates. We developed a method that allowed us to record extracellularly from mushroom body extrinsic neurons (MB ENs) in a freely moving bee within a small but functioning mini colony of approximately 1,000 bees. This study aimed to correlate the neuronal activity of multimodal high-order MB ENs with social behavior in a close to natural setting. The behavior of all bees in the colony was video recorded. The behavior of the recorded animal was compared with other hive mates and no significant differences were found. Changes in the spike rate appeared before, during or after social interactions. The time window of the strongest effect on spike rate changes ranged from 1 s to 2 s before and after the interaction, depending on the individual animal and recorded neuron. The highest spike rates occurred when the experimental animal was situated close to a hive mate. The variance of the spike rates was analyzed as a proxy for high order multi-unit processing. Comparing randomly selected time windows with those in which the recorded animal performed social interactions showed a significantly increased spike rate variance during social interactions. The experimental set-up employed for this study offers a powerful opportunity to correlate neuronal activity with intrinsically motivated behavior of socially interacting animals. We conclude that the recorded MB ENs are potentially involved in initiating and controlling social interactions in honeybees
Harnessing function from form: towards bio-inspired artificial intelligence in neuronal substrates
Despite the recent success of deep learning, the mammalian brain is still unrivaled when it comes
to interpreting complex, high-dimensional data streams like visual, auditory and somatosensory stimuli.
However, the underlying computational principles allowing the brain to deal with unreliable, high-dimensional
and often incomplete data while having a power consumption on the order of a few watt are still mostly
unknown.
In this work, we investigate how specific functionalities emerge from simple structures observed in the
mammalian cortex, and how these might be utilized in non-von Neumann devices like “neuromorphic
hardware”. Firstly, we show that an ensemble of deterministic, spiking neural networks can be shaped by
a simple, local learning rule to perform sampling-based Bayesian inference. This suggests a coding scheme
where spikes (or “action potentials”) represent samples of a posterior distribution, constrained by sensory
input, without the need for any source of stochasticity. Secondly, we introduce a top-down framework where
neuronal and synaptic dynamics are derived using a least action principle and gradient-based minimization.
Combined, neurosynaptic dynamics approximate real-time error backpropagation, mappable to mechanistic
components of cortical networks, whose dynamics can again be described within the proposed framework.
The presented models narrow the gap between well-defined, functional algorithms and their biophysical
implementation, improving our understanding of the computational principles the brain might employ.
Furthermore, such models are naturally translated to hardware mimicking the vastly parallel neural
structure of the brain, promising a strongly accelerated and energy-efficient implementation of powerful
learning and inference algorithms, which we demonstrate for the physical model system “BrainScaleS–1”
Random Neural Networks and Optimisation
In this thesis we introduce new models and learning algorithms for the Random
Neural Network (RNN), and we develop RNN-based and other approaches for the
solution of emergency management optimisation problems.
With respect to RNN developments, two novel supervised learning algorithms are
proposed. The first, is a gradient descent algorithm for an RNN extension model
that we have introduced, the RNN with synchronised interactions (RNNSI), which
was inspired from the synchronised firing activity observed in brain neural circuits.
The second algorithm is based on modelling the signal-flow equations in RNN as a
nonnegative least squares (NNLS) problem. NNLS is solved using a limited-memory
quasi-Newton algorithm specifically designed for the RNN case.
Regarding the investigation of emergency management optimisation problems,
we examine combinatorial assignment problems that require fast, distributed and
close to optimal solution, under information uncertainty. We consider three different
problems with the above characteristics associated with the assignment of
emergency units to incidents with injured civilians (AEUI), the assignment of assets
to tasks under execution uncertainty (ATAU), and the deployment of a robotic
network to establish communication with trapped civilians (DRNCTC).
AEUI is solved by training an RNN tool with instances of the optimisation problem
and then using the trained RNN for decision making; training is achieved using
the developed learning algorithms. For the solution of ATAU problem, we introduce
two different approaches. The first is based on mapping parameters of the
optimisation problem to RNN parameters, and the second on solving a sequence of
minimum cost flow problems on appropriately constructed networks with estimated
arc costs. For the exact solution of DRNCTC problem, we develop a mixed-integer
linear programming formulation, which is based on network flows. Finally, we design
and implement distributed heuristic algorithms for the deployment of robots
when the civilian locations are known or uncertain
Stochasticity from function -- why the Bayesian brain may need no noise
An increasing body of evidence suggests that the trial-to-trial variability
of spiking activity in the brain is not mere noise, but rather the reflection
of a sampling-based encoding scheme for probabilistic computing. Since the
precise statistical properties of neural activity are important in this
context, many models assume an ad-hoc source of well-behaved, explicit noise,
either on the input or on the output side of single neuron dynamics, most often
assuming an independent Poisson process in either case. However, these
assumptions are somewhat problematic: neighboring neurons tend to share
receptive fields, rendering both their input and their output correlated; at
the same time, neurons are known to behave largely deterministically, as a
function of their membrane potential and conductance. We suggest that spiking
neural networks may, in fact, have no need for noise to perform sampling-based
Bayesian inference. We study analytically the effect of auto- and
cross-correlations in functionally Bayesian spiking networks and demonstrate
how their effect translates to synaptic interaction strengths, rendering them
controllable through synaptic plasticity. This allows even small ensembles of
interconnected deterministic spiking networks to simultaneously and
co-dependently shape their output activity through learning, enabling them to
perform complex Bayesian computation without any need for noise, which we
demonstrate in silico, both in classical simulation and in neuromorphic
emulation. These results close a gap between the abstract models and the
biology of functionally Bayesian spiking networks, effectively reducing the
architectural constraints imposed on physical neural substrates required to
perform probabilistic computing, be they biological or artificial
- …