7,754 research outputs found
Intrinsically-generated fluctuating activity in excitatory-inhibitory networks
Recurrent networks of non-linear units display a variety of dynamical regimes
depending on the structure of their synaptic connectivity. A particularly
remarkable phenomenon is the appearance of strongly fluctuating, chaotic
activity in networks of deterministic, but randomly connected rate units. How
this type of intrinsi- cally generated fluctuations appears in more realistic
networks of spiking neurons has been a long standing question. To ease the
comparison between rate and spiking networks, recent works investigated the
dynami- cal regimes of randomly-connected rate networks with segregated
excitatory and inhibitory populations, and firing rates constrained to be
positive. These works derived general dynamical mean field (DMF) equations
describing the fluctuating dynamics, but solved these equations only in the
case of purely inhibitory networks. Using a simplified excitatory-inhibitory
architecture in which DMF equations are more easily tractable, here we show
that the presence of excitation qualitatively modifies the fluctuating activity
compared to purely inhibitory networks. In presence of excitation,
intrinsically generated fluctuations induce a strong increase in mean firing
rates, a phenomenon that is much weaker in purely inhibitory networks.
Excitation moreover induces two different fluctuating regimes: for moderate
overall coupling, recurrent inhibition is sufficient to stabilize fluctuations,
for strong coupling, firing rates are stabilized solely by the upper bound
imposed on activity, even if inhibition is stronger than excitation. These
results extend to more general network architectures, and to rate networks
receiving noisy inputs mimicking spiking activity. Finally, we show that
signatures of the second dynamical regime appear in networks of
integrate-and-fire neurons
When do correlations increase with firing rates in recurrent networks?
A central question in neuroscience is to understand how noisy firing patterns are used to transmit information. Because neural spiking is noisy, spiking patterns are often quantified via pairwise correlations, or the probability that two cells will spike coincidentally, above and beyond their baseline firing rate. One observation frequently made in experiments, is that correlations can increase systematically with firing rate. Theoretical studies have determined that stimulus-dependent correlations that increase with firing rate can have beneficial effects on information coding; however, we still have an incomplete understanding of what circuit mechanisms do, or do not, produce this correlation-firing rate relationship. Here, we studied the relationship between pairwise correlations and firing rates in recurrently coupled excitatory-inhibitory spiking networks with conductance-based synapses. We found that with stronger excitatory coupling, a positive relationship emerged between pairwise correlations and firing rates. To explain these findings, we used linear response theory to predict the full correlation matrix and to decompose correlations in terms of graph motifs. We then used this decomposition to explain why covariation of correlations with firing rate—a relationship previously explained in feedforward networks driven by correlated input—emerges in some recurrent networks but not in others. Furthermore, when correlations covary with firing rate, this relationship is reflected in low-rank structure in the correlation matrix
Localized activity profiles and storage capacity of rate-based autoassociative networks
We study analytically the effect of metrically structured connectivity on the
behavior of autoassociative networks. We focus on three simple rate-based model
neurons: threshold-linear, binary or smoothly saturating units. For a
connectivity which is short range enough the threshold-linear network shows
localized retrieval states. The saturating and binary models also exhibit
spatially modulated retrieval states if the highest activity level that they
can achieve is above the maximum activity of the units in the stored patterns.
In the zero quenched noise limit, we derive an analytical formula for the
critical value of the connectivity width below which one observes spatially
non-uniform retrieval states. Localization reduces storage capacity, but only
by a factor of 2~3. The approach that we present here is generic in the sense
that there are no specific assumptions on the single unit input-output function
nor on the exact connectivity structure.Comment: 4 pages, 4 figure
NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations
This paper introduces Non-Autonomous Input-Output Stable Network (NAIS-Net),
a very deep architecture where each stacked processing block is derived from a
time-invariant non-autonomous dynamical system. Non-autonomy is implemented by
skip connections from the block input to each of the unrolled processing stages
and allows stability to be enforced so that blocks can be unrolled adaptively
to a pattern-dependent processing depth. NAIS-Net induces non-trivial,
Lipschitz input-output maps, even for an infinite unroll length. We prove that
the network is globally asymptotically stable so that for every initial
condition there is exactly one input-dependent equilibrium assuming tanh units,
and multiple stable equilibria for ReL units. An efficient implementation that
enforces the stability under derived conditions for both fully-connected and
convolutional layers is also presented. Experimental results show how NAIS-Net
exhibits stability in practice, yielding a significant reduction in
generalization gap compared to ResNets.Comment: NIPS 201
Locking of correlated neural activity to ongoing oscillations
Population-wide oscillations are ubiquitously observed in mesoscopic signals
of cortical activity. In these network states a global oscillatory cycle
modulates the propensity of neurons to fire. Synchronous activation of neurons
has been hypothesized to be a separate channel of signal processing information
in the brain. A salient question is therefore if and how oscillations interact
with spike synchrony and in how far these channels can be considered separate.
Experiments indeed showed that correlated spiking co-modulates with the static
firing rate and is also tightly locked to the phase of beta-oscillations. While
the dependence of correlations on the mean rate is well understood in
feed-forward networks, it remains unclear why and by which mechanisms
correlations tightly lock to an oscillatory cycle. We here demonstrate that
such correlated activation of pairs of neurons is qualitatively explained by
periodically-driven random networks. We identify the mechanisms by which
covariances depend on a driving periodic stimulus. Mean-field theory combined
with linear response theory yields closed-form expressions for the
cyclostationary mean activities and pairwise zero-time-lag covariances of
binary recurrent random networks. Two distinct mechanisms cause time-dependent
covariances: the modulation of the susceptibility of single neurons (via the
external input and network feedback) and the time-varying variances of single
unit activities. For some parameters, the effectively inhibitory recurrent
feedback leads to resonant covariances even if mean activities show
non-resonant behavior. Our analytical results open the question of
time-modulated synchronous activity to a quantitative analysis.Comment: 57 pages, 12 figures, published versio
- …