650 research outputs found
Expressivity of Spiking Neural Networks
This article studies the expressive power of spiking neural networks where
information is encoded in the firing time of neurons. The implementation of
spiking neural networks on neuromorphic hardware presents a promising choice
for future energy-efficient AI applications. However, there exist very few
results that compare the computational power of spiking neurons to arbitrary
threshold circuits and sigmoidal neurons. Additionally, it has also been shown
that a network of spiking neurons is capable of approximating any continuous
function. By using the Spike Response Model as a mathematical model of a
spiking neuron and assuming a linear response function, we prove that the
mapping generated by a network of spiking neurons is continuous piecewise
linear. We also show that a spiking neural network can emulate the output of
any multi-layer (ReLU) neural network. Furthermore, we show that the maximum
number of linear regions generated by a spiking neuron scales exponentially
with respect to the input dimension, a characteristic that distinguishes it
significantly from an artificial (ReLU) neuron. Our results further extend the
understanding of the approximation properties of spiking neural networks and
open up new avenues where spiking neural networks can be deployed instead of
artificial neural networks without any performance loss
Integration of continuous-time dynamics in a spiking neural network simulator
Contemporary modeling approaches to the dynamics of neural networks consider
two main classes of models: biologically grounded spiking neurons and
functionally inspired rate-based units. The unified simulation framework
presented here supports the combination of the two for multi-scale modeling
approaches, the quantitative validation of mean-field approaches by spiking
network simulations, and an increase in reliability by usage of the same
simulation code and the same network model specifications for both model
classes. While most efficient spiking simulations rely on the communication of
discrete events, rate models require time-continuous interactions between
neurons. Exploiting the conceptual similarity to the inclusion of gap junctions
in spiking network simulations, we arrive at a reference implementation of
instantaneous and delayed interactions between rate-based models in a spiking
network simulator. The separation of rate dynamics from the general connection
and communication infrastructure ensures flexibility of the framework. We
further demonstrate the broad applicability of the framework by considering
various examples from the literature ranging from random networks to neural
field models. The study provides the prerequisite for interactions between
rate-based and spiking models in a joint simulation
A Survey on Continuous Time Computations
We provide an overview of theories of continuous time computation. These
theories allow us to understand both the hardness of questions related to
continuous time dynamical systems and the computational power of continuous
time analog models. We survey the existing models, summarizing results, and
point to relevant references in the literature
A point process framework for modeling electrical stimulation of the auditory nerve
Model-based studies of auditory nerve responses to electrical stimulation can
provide insight into the functioning of cochlear implants. Ideally, these
studies can identify limitations in sound processing strategies and lead to
improved methods for providing sound information to cochlear implant users. To
accomplish this, models must accurately describe auditory nerve spiking while
avoiding excessive complexity that would preclude large-scale simulations of
populations of auditory nerve fibers and obscure insight into the mechanisms
that influence neural encoding of sound information. In this spirit, we develop
a point process model of the auditory nerve that provides a compact and
accurate description of neural responses to electric stimulation. Inspired by
the framework of generalized linear models, the proposed model consists of a
cascade of linear and nonlinear stages. We show how each of these stages can be
associated with biophysical mechanisms and related to models of neuronal
dynamics. Moreover, we derive a semi-analytical procedure that uniquely
determines each parameter in the model on the basis of fundamental statistics
from recordings of single fiber responses to electric stimulation, including
threshold, relative spread, jitter, and chronaxie. The model also accounts for
refractory and summation effects that influence the responses of auditory nerve
fibers to high pulse rate stimulation. Throughout, we compare model predictions
to published physiological data and explain differences in auditory nerve
responses to high and low pulse rate stimulation. We close by performing an
ideal observer analysis of simulated spike trains in response to sinusoidally
amplitude modulated stimuli and find that carrier pulse rate does not affect
modulation detection thresholds.Comment: 1 title page, 27 manuscript pages, 14 figures, 1 table, 1 appendi
Quantized State Simulation of Spiking Neural Networks
In this work, we explore the usage of quantized state system (QSS) methods in the simulation of networks of spiking neurons. We compare the simulation results obtained by these discrete-event algorithms with the results of the discrete time methods in use by the neuroscience community. We found that the computational costs of the QSS methods grow almost linearly with the size of the network, while they grows at least quadratically in the discrete time algorithms. We show that this advantage is mainly due to the fact that QSS methods only perform calculations in the components of the system that experience activity. © 2012, Simulation Councils Inc. All rights reserved.Fil: Grinblat, Guillermo Luis. Consejo Nacional de Investigaciones CientÃficas y Técnicas. Centro CientÃfico Tecnológico Conicet - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas. Universidad Nacional de Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas; ArgentinaFil: Ahumada, Hernán. Consejo Nacional de Investigaciones CientÃficas y Técnicas. Centro CientÃfico Tecnológico Conicet - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas. Universidad Nacional de Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas; ArgentinaFil: Kofman, Ernesto Javier. Consejo Nacional de Investigaciones CientÃficas y Técnicas. Centro CientÃfico Tecnológico Conicet - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas. Universidad Nacional de Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas; Argentin
Sample Path Analysis of Integrate-and-Fire Neurons
Computational neuroscience is concerned with answering two intertwined questions that are based on the assumption that spatio-temporal patterns of spikes form the universal language of the nervous system. First, what function does a specific neural circuitry perform in the elaboration of a behavior? Second, how do neural circuits process behaviorally-relevant information? Non-linear system analysis has proven instrumental in understanding the coding strategies of early neural processing in various sensory modalities. Yet, at higher levels of integration, it fails to help in deciphering the response of assemblies of neurons to complex naturalistic stimuli. If neural activity can be assumed to be primarily driven by the stimulus at early stages of processing, the intrinsic activity of neural circuits interacts with their high-dimensional input to transform it in a stochastic non-linear fashion at the cortical level. As a consequence, any attempt to fully understand the brain through a system analysis approach becomes illusory. However, it is increasingly advocated that neural noise plays a constructive role in neural processing, facilitating information transmission. This prompts to gain insight into the neural code by studying the stochasticity of neuronal activity, which is viewed as biologically relevant. Such an endeavor requires the design of guiding theoretical principles to assess the potential benefits of neural noise. In this context, meeting the requirements of biological relevance and computational tractability, while providing a stochastic description of neural activity, prescribes the adoption of the integrate-and-fire model. In this thesis, founding ourselves on the path-wise description of neuronal activity, we propose to further the stochastic analysis of the integrate-and fire model through a combination of numerical and theoretical techniques. To begin, we expand upon the path-wise construction of linear diffusions, which offers a natural setting to describe leaky integrate-and-fire neurons, as inhomogeneous Markov chains. Based on the theoretical analysis of the first-passage problem, we then explore the interplay between the internal neuronal noise and the statistics of injected perturbations at the single unit level, and examine its implications on the neural coding. At the population level, we also develop an exact event-driven implementation of a Markov network of perfect integrate-and-fire neurons with both time delayed instantaneous interactions and arbitrary topology. We hope our approach will provide new paradigms to understand how sensory inputs perturb neural intrinsic activity and accomplish the goal of developing a new technique for identifying relevant patterns of population activity. From a perturbative perspective, our study shows how injecting frozen noise in different flavors can help characterize internal neuronal noise, which is presumably functionally relevant to information processing. From a simulation perspective, our event-driven framework is amenable to scrutinize the stochastic behavior of simple recurrent motifs as well as temporal dynamics of large scale networks under spike-timing-dependent plasticity
Decorrelation of neural-network activity by inhibitory feedback
Correlations in spike-train ensembles can seriously impair the encoding of
information by their spatio-temporal structure. An inevitable source of
correlation in finite neural networks is common presynaptic input to pairs of
neurons. Recent theoretical and experimental studies demonstrate that spike
correlations in recurrent neural networks are considerably smaller than
expected based on the amount of shared presynaptic input. By means of a linear
network model and simulations of networks of leaky integrate-and-fire neurons,
we show that shared-input correlations are efficiently suppressed by inhibitory
feedback. To elucidate the effect of feedback, we compare the responses of the
intact recurrent network and systems where the statistics of the feedback
channel is perturbed. The suppression of spike-train correlations and
population-rate fluctuations by inhibitory feedback can be observed both in
purely inhibitory and in excitatory-inhibitory networks. The effect is fully
understood by a linear theory and becomes already apparent at the macroscopic
level of the population averaged activity. At the microscopic level,
shared-input correlations are suppressed by spike-train correlations: In purely
inhibitory networks, they are canceled by negative spike-train correlations. In
excitatory-inhibitory networks, spike-train correlations are typically
positive. Here, the suppression of input correlations is not a result of the
mere existence of correlations between excitatory (E) and inhibitory (I)
neurons, but a consequence of a particular structure of correlations among the
three possible pairings (EE, EI, II)
Approximating nonlinear functions with latent boundaries in low-rank excitatory-inhibitory spiking networks
Deep feedforward and recurrent rate-based neural networks have become
successful functional models of the brain, but they neglect obvious biological
details such as spikes and Dale's law. Here we argue that these details are
crucial in order to understand how real neural circuits operate. Towards this
aim, we put forth a new framework for spike-based computation in low-rank
excitatory-inhibitory spiking networks. By considering populations with rank-1
connectivity, we cast each neuron's spiking threshold as a boundary in a
low-dimensional input-output space. We then show how the combined thresholds of
a population of inhibitory neurons form a stable boundary in this space, and
those of a population of excitatory neurons form an unstable boundary.
Combining the two boundaries results in a rank-2 excitatory-inhibitory (EI)
network with inhibition-stabilized dynamics at the intersection of the two
boundaries. The computation of the resulting networks can be understood as the
difference of two convex functions, and is thereby capable of approximating
arbitrary non-linear input-output mappings. We demonstrate several properties
of these networks, including noise suppression and amplification, irregular
activity and synaptic balance, as well as how they relate to rate network
dynamics in the limit that the boundary becomes soft. Finally, while our work
focuses on small networks (5-50 neurons), we discuss potential avenues for
scaling up to much larger networks. Overall, our work proposes a new
perspective on spiking networks that may serve as a starting point for a
mechanistic understanding of biological spike-based computation
- …