76 research outputs found
Shared inputs, entrainment, and desynchrony in elliptic bursters: from slow passage to discontinuous circle maps
What input signals will lead to synchrony vs. desynchrony in a group of
biological oscillators? This question connects with both classical dynamical
systems analyses of entrainment and phase locking and with emerging studies of
stimulation patterns for controlling neural network activity. Here, we focus on
the response of a population of uncoupled, elliptically bursting neurons to a
common pulsatile input. We extend a phase reduction from the literature to
capture inputs of varied strength, leading to a circle map with discontinuities
of various orders. In a combined analytical and numerical approach, we apply
our results to both a normal form model for elliptic bursting and to a
biophysically-based neuron model from the basal ganglia. We find that,
depending on the period and amplitude of inputs, the response can either appear
chaotic (with provably positive Lyaponov exponent for the associated circle
maps), or periodic with a broad range of phase-locked periods. Throughout, we
discuss the critical underlying mechanisms, including slow-passage effects
through Hopf bifurcation, the role and origin of discontinuities, and the
impact of noiseComment: 17 figures, 40 page
Structured chaos shapes spike-response noise entropy in balanced neural networks
Large networks of sparsely coupled, excitatory and inhibitory cells occur
throughout the brain. A striking feature of these networks is that they are
chaotic. How does this chaos manifest in the neural code? Specifically, how
variable are the spike patterns that such a network produces in response to an
input signal? To answer this, we derive a bound for the entropy of multi-cell
spike pattern distributions in large recurrent networks of spiking neurons
responding to fluctuating inputs. The analysis is based on results from random
dynamical systems theory and is complimented by detailed numerical simulations.
We find that the spike pattern entropy is an order of magnitude lower than what
would be extrapolated from single cells. This holds despite the fact that
network coupling becomes vanishingly sparse as network size grows -- a
phenomenon that depends on ``extensive chaos," as previously discovered for
balanced networks without stimulus drive. Moreover, we show how spike pattern
entropy is controlled by temporal features of the inputs. Our findings provide
insight into how neural networks may encode stimuli in the presence of
inherently chaotic dynamics.Comment: 9 pages, 5 figure
Revisiting chaos in stimulus-driven spiking networks: signal encoding and discrimination
Highly connected recurrent neural networks often produce chaotic dynamics,
meaning their precise activity is sensitive to small perturbations. What are
the consequences for how such networks encode streams of temporal stimuli? On
the one hand, chaos is a strong source of randomness, suggesting that small
changes in stimuli will be obscured by intrinsically generated variability. On
the other hand, recent work shows that the type of chaos that occurs in spiking
networks can have a surprisingly low-dimensional structure, suggesting that
there may be "room" for fine stimulus features to be precisely resolved. Here
we show that strongly chaotic networks produce patterned spikes that reliably
encode time-dependent stimuli: using a decoder sensitive to spike times on
timescales of 10's of ms, one can easily distinguish responses to very similar
inputs. Moreover, recurrence serves to distribute signals throughout chaotic
networks so that small groups of cells can encode substantial information about
signals arriving elsewhere. A conclusion is that the presence of strong chaos
in recurrent networks does not prohibit precise stimulus encoding.Comment: 8 figure
Flexible Phase Dynamics for Bio-Plausible Contrastive Learning
Many learning algorithms used as normative models in neuroscience or as
candidate approaches for learning on neuromorphic chips learn by contrasting
one set of network states with another. These Contrastive Learning (CL)
algorithms are traditionally implemented with rigid, temporally non-local, and
periodic learning dynamics that could limit the range of physical systems
capable of harnessing CL. In this study, we build on recent work exploring how
CL might be implemented by biological or neurmorphic systems and show that this
form of learning can be made temporally local, and can still function even if
many of the dynamical requirements of standard training procedures are relaxed.
Thanks to a set of general theorems corroborated by numerical experiments
across several CL models, our results provide theoretical foundations for the
study and development of CL methods for biological and neuromorphic neural
networks.Comment: 23 pages, 4 figures. Paper accepted to ICML and update includes
changes made based on reviewer feedbac
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty
Among attempts at giving a theoretical account of the success of deep neural
networks, a recent line of work has identified a so-called `lazy' regime in
which the network can be well approximated by its linearization around
initialization. Here we investigate the comparative effect of the lazy (linear)
and feature learning (non-linear) regimes on subgroups of examples based on
their difficulty. Specifically, we show that easier examples are given more
weight in feature learning mode, resulting in faster training compared to more
difficult ones. In other words, the non-linear dynamics tends to sequentialize
the learning of examples of increasing difficulty. We illustrate this
phenomenon across different ways to quantify example difficulty, including
c-score, label noise, and in the presence of spurious correlations. Our results
reveal a new understanding of how deep networks prioritize resources across
example difficulty
Correlation-based model of artificially induced plasticity in motor cortex by a bidirectional brain-computer interface
Experiments show that spike-triggered stimulation performed with
Bidirectional Brain-Computer-Interfaces (BBCI) can artificially strengthen
connections between separate neural sites in motor cortex (MC). What are the
neuronal mechanisms responsible for these changes and how does targeted
stimulation by a BBCI shape population-level synaptic connectivity? The present
work describes a recurrent neural network model with probabilistic spiking
mechanisms and plastic synapses capable of capturing both neural and synaptic
activity statistics relevant to BBCI conditioning protocols. When spikes from a
neuron recorded at one MC site trigger stimuli at a second target site after a
fixed delay, the connections between sites are strengthened for spike-stimulus
delays consistent with experimentally derived spike time dependent plasticity
(STDP) rules. However, the relationship between STDP mechanisms at the level of
networks, and their modification with neural implants remains poorly
understood. Using our model, we successfully reproduces key experimental
results and use analytical derivations, along with novel experimental data. We
then derive optimal operational regimes for BBCIs, and formulate predictions
concerning the efficacy of spike-triggered stimulation in different regimes of
cortical activity.Comment: 35 pages, 9 figure
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics
A recent strategy to circumvent the exploding and vanishing gradient problem
in RNNs, and to allow the stable propagation of signals over long time scales,
is to constrain recurrent connectivity matrices to be orthogonal or unitary.
This ensures eigenvalues with unit norm and thus stable dynamics and training.
However this comes at the cost of reduced expressivity due to the limited
variety of orthogonal transformations. We propose a novel connectivity
structure based on the Schur decomposition and a splitting of the Schur form
into normal and non-normal parts. This allows to parametrize matrices with
unit-norm eigenspectra without orthogonality constraints on eigenbases. The
resulting architecture ensures access to a larger space of spectrally
constrained matrices, of which orthogonal matrices are a subset. This crucial
difference retains the stability advantages and training speed of orthogonal
RNNs while enhancing expressivity, especially on tasks that require
computations over ongoing input sequences
- …