68 research outputs found
How Gibbs distributions may naturally arise from synaptic adaptation mechanisms. A model-based argumentation
This paper addresses two questions in the context of neuronal networks
dynamics, using methods from dynamical systems theory and statistical physics:
(i) How to characterize the statistical properties of sequences of action
potentials ("spike trains") produced by neuronal networks ? and; (ii) what are
the effects of synaptic plasticity on these statistics ? We introduce a
framework in which spike trains are associated to a coding of membrane
potential trajectories, and actually, constitute a symbolic coding in important
explicit examples (the so-called gIF models). On this basis, we use the
thermodynamic formalism from ergodic theory to show how Gibbs distributions are
natural probability measures to describe the statistics of spike trains, given
the empirical averages of prescribed quantities. As a second result, we show
that Gibbs distributions naturally arise when considering "slow" synaptic
plasticity rules where the characteristic time for synapse adaptation is quite
longer than the characteristic time for neurons dynamics.Comment: 39 pages, 3 figure
Improving Associative Memory in a Network of Spiking Neurons
In this thesis we use computational neural network models to examine the dynamics and functionality of the CA3 region of the mammalian hippocampus. The emphasis of the project is to investigate how the dynamic control structures provided by inhibitory circuitry and cellular modification may effect the CA3 region during the recall of previously stored information. The CA3 region is commonly thought to work as a recurrent auto-associative neural network due to the neurophysiological characteristics found, such as, recurrent collaterals, strong and sparse synapses from external inputs and plasticity between coactive cells. Associative memory models have been developed using various configurations of mathematical artificial neural networks which were first developed over 40 years ago. Within these models we can store information via changes in the strength of connections between simplified model neurons (two-state). These memories can be recalled when a cue (noisy or partial) is instantiated upon the net. The type of information they can store is quite limited due to restrictions caused by the simplicity of the hard-limiting nodes which are commonly associated with a binary activation threshold. We build a much more biologically plausible model with complex spiking cell models and with realistic synaptic properties between cells. This model is based upon some of the many details we now know of the neuronal circuitry of the CA3 region. We implemented the model in computer software using Neuron and Matlab and tested it by running simulations of storage and recall in the network. By building this model we gain new insights into how different types of neurons, and the complex circuits they form, actually work.
The mammalian brain consists of complex resistive-capacative electrical circuitry which is formed by the interconnection of large numbers of neurons. A principal cell type is the pyramidal cell within the cortex, which is the main information processor in our neural networks. Pyramidal cells are surrounded by diverse populations of interneurons which have proportionally smaller numbers compared to the pyramidal cells and these form connections with pyramidal cells and other inhibitory cells. By building detailed computational models of recurrent neural circuitry we explore how these microcircuits of interneurons control the flow of information through pyramidal cells and regulate the efficacy of the network. We also explore the effect of cellular modification due to neuronal activity and the effect of incorporating spatially dependent connectivity on the network during recall of previously stored information.
In particular we implement a spiking neural network proposed by Sommer and Wennekers (2001). We consider methods for improving associative memory recall using methods inspired by the work by Graham and Willshaw (1995) where they apply mathematical transforms to an artificial neural network to improve the recall quality within the network. The networks tested contain either 100 or 1000 pyramidal cells with 10% connectivity applied and a partial cue instantiated, and with a global pseudo-inhibition.We investigate three methods. Firstly, applying localised disynaptic inhibition which will proportionalise the excitatory post synaptic potentials and provide a fast acting reversal potential which should help to reduce the variability in signal propagation between cells and provide further inhibition to help synchronise the network activity. Secondly, implementing a persistent sodium channel to the cell body which will act to non-linearise the activation threshold where after a given membrane potential the amplitude of the excitatory postsynaptic potential (EPSP) is boosted to push cells which receive slightly more excitation (most likely high units) over the firing threshold. Finally, implementing spatial characteristics of the dendritic tree will allow a greater probability of a modified synapse existing after 10% random connectivity has been applied throughout the network. We apply spatial characteristics by scaling the conductance weights of excitatory synapses which simulate the loss in potential in synapses found in the outer dendritic regions due to increased resistance.
To further increase the biological plausibility of the network we remove the pseudo-inhibition and apply realistic basket cell models with differing configurations for a global inhibitory circuit. The networks are configured with; 1 single basket cell providing feedback inhibition, 10% basket cells providing feedback inhibition where 10 pyramidal cells connect to each basket cell and finally, 100% basket cells providing feedback inhibition. These networks are compared and contrasted for efficacy on recall quality and the effect on the network behaviour. We have found promising results from applying biologically plausible recall strategies and network configurations which suggests the role of inhibition and cellular dynamics are pivotal in learning and memory
The importance of space and time in neuromorphic cognitive agents
Artificial neural networks and computational neuroscience models have made
tremendous progress, allowing computers to achieve impressive results in
artificial intelligence (AI) applications, such as image recognition, natural
language processing, or autonomous driving. Despite this remarkable progress,
biological neural systems consume orders of magnitude less energy than today's
artificial neural networks and are much more agile and adaptive. This
efficiency and adaptivity gap is partially explained by the computing substrate
of biological neural processing systems that is fundamentally different from
the way today's computers are built. Biological systems use in-memory computing
elements operating in a massively parallel way rather than time-multiplexed
computing units that are reused in a sequential fashion. Moreover, activity of
biological neurons follows continuous-time dynamics in real, physical time,
instead of operating on discrete temporal cycles abstracted away from
real-time. Here, we present neuromorphic processing devices that emulate the
biological style of processing by using parallel instances of mixed-signal
analog/digital circuits that operate in real time. We argue that this approach
brings significant advantages in efficiency of computation. We show examples of
embodied neuromorphic agents that use such devices to interact with the
environment and exhibit autonomous learning
Spike-based local synaptic plasticity: A survey of computational models and neuromorphic circuits
Understanding how biological neural networks carry out learning using
spike-based local plasticity mechanisms can lead to the development of
powerful, energy-efficient, and adaptive neuromorphic processing systems. A
large number of spike-based learning models have recently been proposed
following different approaches. However, it is difficult to assess if and how
they could be mapped onto neuromorphic hardware, and to compare their features
and ease of implementation. To this end, in this survey, we provide a
comprehensive overview of representative brain-inspired synaptic plasticity
models and mixed-signal CMOS neuromorphic circuits within a unified framework.
We review historical, bottom-up, and top-down approaches to modeling synaptic
plasticity, and we identify computational primitives that can support
low-latency and low-power hardware implementations of spike-based learning
rules. We provide a common definition of a locality principle based on pre- and
post-synaptic neuron information, which we propose as a fundamental requirement
for physical implementations of synaptic plasticity. Based on this principle,
we compare the properties of these models within the same framework, and
describe the mixed-signal electronic circuits that implement their computing
primitives, pointing out how these building blocks enable efficient on-chip and
online learning in neuromorphic processing systems
- …