1,592 research outputs found
Blindfold learning of an accurate neural metric
The brain has no direct access to physical stimuli, but only to the spiking
activity evoked in sensory organs. It is unclear how the brain can structure
its representation of the world based on differences between those noisy,
correlated responses alone. Here we show how to build a distance map of
responses from the structure of the population activity of retinal ganglion
cells, allowing for the accurate discrimination of distinct visual stimuli from
the retinal response. We introduce the Temporal Restricted Boltzmann Machine to
learn the spatiotemporal structure of the population activity, and use this
model to define a distance between spike trains. We show that this metric
outperforms existing neural distances at discriminating pairs of stimuli that
are barely distinguishable. The proposed method provides a generic and
biologically plausible way to learn to associate similar stimuli based on their
spiking responses, without any other knowledge of these stimuli
Spatio-temporal spike trains analysis for large scale networks using maximum entropy principle and Monte-Carlo method
Understanding the dynamics of neural networks is a major challenge in
experimental neuroscience. For that purpose, a modelling of the recorded
activity that reproduces the main statistics of the data is required. In a
first part, we present a review on recent results dealing with spike train
statistics analysis using maximum entropy models (MaxEnt). Most of these
studies have been focusing on modelling synchronous spike patterns, leaving
aside the temporal dynamics of the neural activity. However, the maximum
entropy principle can be generalized to the temporal case, leading to Markovian
models where memory effects and time correlations in the dynamics are properly
taken into account. In a second part, we present a new method based on
Monte-Carlo sampling which is suited for the fitting of large-scale
spatio-temporal MaxEnt models. The formalism and the tools presented here will
be essential to fit MaxEnt spatio-temporal models to large neural ensembles.Comment: 41 pages, 10 figure
Neural Networks and Contagion
We analyze local as well as global interaction and contagion in population games, using the formalism of neural networks. In contrast to much of the literature, a state encodes not only the frequency of play, but also the spatial pattern of play. Stochastic best response dynamics with logistic noise gives rise to a log-linear or logit response model. The stationary distribution is of the Gibbs-Boltzmann type. The long-run equilibria are the maxima of a potential function
TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning
Combining deep model-free reinforcement learning with on-line planning is a
promising approach to building on the successes of deep RL. On-line planning
with look-ahead trees has proven successful in environments where transition
models are known a priori. However, in complex environments where transition
models need to be learned from data, the deficiencies of learned models have
limited their utility for planning. To address these challenges, we propose
TreeQN, a differentiable, recursive, tree-structured model that serves as a
drop-in replacement for any value function network in deep RL with discrete
actions. TreeQN dynamically constructs a tree by recursively applying a
transition model in a learned abstract state space and then aggregating
predicted rewards and state-values using a tree backup to estimate Q-values. We
also propose ATreeC, an actor-critic variant that augments TreeQN with a
softmax layer to form a stochastic policy network. Both approaches are trained
end-to-end, such that the learned model is optimised for its actual use in the
tree. We show that TreeQN and ATreeC outperform n-step DQN and A2C on a
box-pushing task, as well as n-step DQN and value prediction networks (Oh et
al. 2017) on multiple Atari games. Furthermore, we present ablation studies
that demonstrate the effect of different auxiliary losses on learning
transition models
Spin glass systems as collective active inference
An open question in the study of emergent behaviour in multi-agent Bayesian
systems is the relationship, if any, between individual and collective
inference. In this paper we explore the correspondence between generative
models that exist at two distinct scales, using spin glass models as a sandbox
system to investigate this question. We show that the collective dynamics of a
specific type of active inference agent is equivalent to sampling from the
stationary distribution of a spin glass system. A collective of
specifically-designed active inference agents can thus be described as
implementing a form of sampling-based inference (namely, from a Boltzmann
machine) at the higher level. However, this equivalence is very fragile,
breaking upon simple modifications to the generative models of the individual
agents or the nature of their interactions. We discuss the implications of this
correspondence and its fragility for the study of multiscale systems composed
of Bayesian agents.Comment: Accepted for publication: 3rd International Workshop on Active
Inferenc
- …