4,496 research outputs found
Sampling-based probabilistic inference emerges from learning in neural circuits with a cost on reliability
Neural responses in the cortex change over time both systematically, due to
ongoing plasticity and learning, and seemingly randomly, due to various sources
of noise and variability. Most previous work considered each of these
processes, learning and variability, in isolation -- here we study neural
networks exhibiting both and show that their interaction leads to the emergence
of powerful computational properties. We trained neural networks on classical
unsupervised learning tasks, in which the objective was to represent their
inputs in an efficient, easily decodable form, with an additional cost for
neural reliability which we derived from basic biophysical considerations. This
cost on reliability introduced a tradeoff between energetically cheap but
inaccurate representations and energetically costly but accurate ones. Despite
the learning tasks being non-probabilistic, the networks solved this tradeoff
by developing a probabilistic representation: neural variability represented
samples from statistically appropriate posterior distributions that would
result from performing probabilistic inference over their inputs. We provide an
analytical understanding of this result by revealing a connection between the
cost of reliability, and the objective for a state-of-the-art Bayesian
inference strategy: variational autoencoders. We show that the same cost leads
to the emergence of increasingly accurate probabilistic representations as
networks become more complex, from single-layer feed-forward, through
multi-layer feed-forward, to recurrent architectures. Our results provide
insights into why neural responses in sensory areas show signatures of
sampling-based probabilistic representations, and may inform future deep
learning algorithms and their implementation in stochastic low-precision
computing systems
When do correlations increase with firing rates in recurrent networks?
A central question in neuroscience is to understand how noisy firing patterns are used to transmit information. Because neural spiking is noisy, spiking patterns are often quantified via pairwise correlations, or the probability that two cells will spike coincidentally, above and beyond their baseline firing rate. One observation frequently made in experiments, is that correlations can increase systematically with firing rate. Theoretical studies have determined that stimulus-dependent correlations that increase with firing rate can have beneficial effects on information coding; however, we still have an incomplete understanding of what circuit mechanisms do, or do not, produce this correlation-firing rate relationship. Here, we studied the relationship between pairwise correlations and firing rates in recurrently coupled excitatory-inhibitory spiking networks with conductance-based synapses. We found that with stronger excitatory coupling, a positive relationship emerged between pairwise correlations and firing rates. To explain these findings, we used linear response theory to predict the full correlation matrix and to decompose correlations in terms of graph motifs. We then used this decomposition to explain why covariation of correlations with firing rate—a relationship previously explained in feedforward networks driven by correlated input—emerges in some recurrent networks but not in others. Furthermore, when correlations covary with firing rate, this relationship is reflected in low-rank structure in the correlation matrix
Locking of correlated neural activity to ongoing oscillations
Population-wide oscillations are ubiquitously observed in mesoscopic signals
of cortical activity. In these network states a global oscillatory cycle
modulates the propensity of neurons to fire. Synchronous activation of neurons
has been hypothesized to be a separate channel of signal processing information
in the brain. A salient question is therefore if and how oscillations interact
with spike synchrony and in how far these channels can be considered separate.
Experiments indeed showed that correlated spiking co-modulates with the static
firing rate and is also tightly locked to the phase of beta-oscillations. While
the dependence of correlations on the mean rate is well understood in
feed-forward networks, it remains unclear why and by which mechanisms
correlations tightly lock to an oscillatory cycle. We here demonstrate that
such correlated activation of pairs of neurons is qualitatively explained by
periodically-driven random networks. We identify the mechanisms by which
covariances depend on a driving periodic stimulus. Mean-field theory combined
with linear response theory yields closed-form expressions for the
cyclostationary mean activities and pairwise zero-time-lag covariances of
binary recurrent random networks. Two distinct mechanisms cause time-dependent
covariances: the modulation of the susceptibility of single neurons (via the
external input and network feedback) and the time-varying variances of single
unit activities. For some parameters, the effectively inhibitory recurrent
feedback leads to resonant covariances even if mean activities show
non-resonant behavior. Our analytical results open the question of
time-modulated synchronous activity to a quantitative analysis.Comment: 57 pages, 12 figures, published versio
Learning probabilistic neural representations with randomly connected circuits
The brain represents and reasons probabilistically about complex stimuli and motor actions using a noisy, spike-based neural code. A key building block for such neural computations, as well as the basis for supervised and unsupervised learning, is the ability to estimate the surprise or likelihood of incoming high-dimensional neural activity patterns. Despite progress in statistical modeling of neural responses and deep learning, current approaches either do not scale to large neural populations or cannot be implemented using biologically realistic mechanisms. Inspired by the sparse and random connectivity of real neuronal circuits, we present a model for neural codes that accurately estimates the likelihood of individual spiking patterns and has a straightforward, scalable, efficient, learnable, and realistic neural implementation. This model’s performance on simultaneously recorded spiking activity of >100 neurons in the monkey visual and prefrontal cortices is comparable with or better than that of state-of-the-art models. Importantly, the model can be learned using a small number of samples and using a local learning rule that utilizes noise intrinsic to neural circuits. Slower, structural changes in random connectivity, consistent with rewiring and pruning processes, further improve the efficiency and sparseness of the resulting neural representations. Our results merge insights from neuroanatomy, machine learning, and theoretical neuroscience to suggest random sparse connectivity as a key design principle for neuronal computation
Functional consequences of inhibitory plasticity: homeostasis, the excitation-inhibition balance and beyond
Computational neuroscience has a long-standing tradition of investigating the consequences of excitatory synaptic plasticity. In contrast, the functions of inhibitory plasticity are still largely nebulous, particularly given the bewildering diversity of interneurons in the brain. Here, we review recent computational advances that provide first suggestions for the functional roles of inhibitory plasticity, such as a maintenance of the excitation-inhibition balance, a stabilization of recurrent network dynamics and a decorrelation of sensory responses. The field is still in its infancy, but given the existing body of theory for excitatory plasticity, it is likely to mature quickly and deliver important insights into the self-organization of inhibitory circuits in the brain
Temporal Dynamics of Decision-Making during Motion Perception in the Visual Cortex
How does the brain make decisions? Speed and accuracy of perceptual decisions covary with certainty in the input, and correlate with the rate of evidence accumulation in parietal and frontal cortical "decision neurons." A biophysically realistic model of interactions within and between Retina/LGN and cortical areas V1, MT, MST, and LIP, gated by basal ganglia, simulates dynamic properties of decision-making in response to ambiguous visual motion stimuli used by Newsome, Shadlen, and colleagues in their neurophysiological experiments. The model clarifies how brain circuits that solve the aperture problem interact with a recurrent competitive network with self-normalizing choice properties to carry out probablistic decisions in real time. Some scientists claim that perception and decision-making can be described using Bayesian inference or related general statistical ideas, that estimate the optimal interpretation of the stimulus given priors and likelihoods. However, such concepts do not propose the neocortical mechanisms that enable perception, and make decisions. The present model explains behavioral and neurophysiological decision-making data without an appeal to Bayesian concepts and, unlike other existing models of these data, generates perceptual representations and choice dynamics in response to the experimental visual stimuli. Quantitative model simulations include the time course of LIP neuronal dynamics, as well as behavioral accuracy and reaction time properties, during both correct and error trials at different levels of input ambiguity in both fixed duration and reaction time tasks. Model MT/MST interactions compute the global direction of random dot motion stimuli, while model LIP computes the stochastic perceptual decision that leads to a saccadic eye movement.National Science Foundation (SBE-0354378, IIS-02-05271); Office of Naval Research (N00014-01-1-0624); National Institutes of Health (R01-DC-02852
Neural population coding: combining insights from microscopic and mass signals
Behavior relies on the distributed and coordinated activity of neural populations. Population activity can be measured using multi-neuron recordings and neuroimaging. Neural recordings reveal how the heterogeneity, sparseness, timing, and correlation of population activity shape information processing in local networks, whereas neuroimaging shows how long-range coupling and brain states impact on local activity and perception. To obtain an integrated perspective on neural information processing we need to combine knowledge from both levels of investigation. We review recent progress of how neural recordings, neuroimaging, and computational approaches begin to elucidate how interactions between local neural population activity and large-scale dynamics shape the structure and coding capacity of local information representations, make them state-dependent, and control distributed populations that collectively shape behavior
- …