Here we analyze synaptic transmission from an information-theoretic
perspective. We derive closed-form expressions for the lower-bounds on the
capacity of a simple model of a cortical synapse under two explicit coding
paradigms. Under the ``signal estimation'' paradigm, we assume the signal to be
encoded in the mean firing rate of a Poisson neuron. The performance of an
optimal linear estimator of the signal then provides a lower bound on the
capacity for signal estimation. Under the ``signal detection'' paradigm, the
presence or absence of the signal has to be detected. Performance of the
optimal spike detector allows us to compute a lower bound on the capacity for
signal detection. We find that single synapses (for empirically measured
parameter values) transmit information poorly but significant improvement can
be achieved with a small amount of redundancy.Comment: 7 pages, 4 figures, NIPS97 proceedings: neuroscience. Originally
  submitted to the neuro-sys archive which was never publicly announced (was
  9809002

Koch, C.

Manwani, A.

English

arXiv

Here we analyze synaptic transmission from an information-theoretic perspective. We derive close-form expressions for the lower-bounds on the capacity of a simple model of a cortical synapse under two explicit coding paradigms.  Under the "signal estimation" paradigm, we assume the signal to be encoded in the mean firing rate of a Poisson neuron.  The performance of an optimal linear estimator of the signal then provides a lower bound on the capacity for signal estimation.  Under the "signal detection" paradigm, the presence or absence of the signal has to be detected.  Performance of the optimal spike detector allows us to compute a lower bound on the capacity for signal detection.  WE find that single synapses (for empirically measured parameter values) transmit information poorly but significant improvement can be achieved with a small amount of redundancy

Manwani, Amit

Koch, Christof

Caltech Authors - Main

in Advances in Neural Information Processing Systems 10,
M.I. Jordan, M.S. Kearns and S.A. Solla (eds.), pp. 201-207, MIT Press (1998).
Synaptic Transmission: An
Information-Theoretic Perspective
Amit Manwani and Christof Koch
Computation and Neural Systems Program
California Institute of Technology
Pasadena, CA 91125
email: quixote@klab.caltech.edu
koch@klab.caltech.edu
Abstract
Here we analyze synaptic transmission from an information-theoretic
perspective. We derive closed-form expressions for the lower-bounds on
the capacity of a simple model of a cortical synapse under two explicit
coding paradigms. Under the “signal estimation” paradigm, we assume
the signal to be encoded in the mean firing rate of a Poisson neuron. The
performance of an optimal linear estimator of the signal then provides
a lower bound on the capacity for signal estimation. Under the “signal
detection” paradigm, the presence or absence of the signal has to be de-
tected. Performance of the optimal spike detector allows us to compute
a lower bound on the capacity for signal detection. We find that single
synapses (for empirically measured parameter values) transmit informa-
tion poorly but significant improvement can be achieved with a small
amount of redundancy.
1 Introduction
Tools from estimation and information theory have recently been applied by researchers
(Bialek et. al, 1991) to quantify how well neurons transmit information about their random
inputs in their spike outputs. In these approaches, the neuron is treated like a black-box,
characterized empirically by a set of input-output records. This ignores the specific nature
of neuronal processing in terms of its known biophysical properties. However, a systematic
study of processing at various stages in a biophysically faithful model of a single neuron
should be able to identify the role of each stage in information transfer in terms of the pa-
rameters relating to the neuron’s dendritic structure, its spikingmechanism, etc. Employing
this reductionist approach, we focus on a important component of neural processing, the
synapse, and analyze a simple model of a cortical synapse under two different representa-
tional paradigms. Under the “signal estimation” paradigm, we assume that the input signal
is linearly encoded in the mean firing rate of a Poisson neuron and the mean-square error
in the reconstruction of the signal from the post-synaptic voltage quantifies system per-
formance. From the performance of the optimal linear estimator of the signal, a lower
bound on the capacity for signal estimation can be computed. Under the “signal detection”
paradigm, we assume that information is encoded in an all-or-none format and the error in
decidingwhether or not a presynaptic spike occurred by observing the post-synaptic voltage
quantifies system performance. This is similar to the conventional absent/present(Yes-No)
decision paradigm used in psychophysics. Performance of the optimal spike detector in
this case allows us to compute a lower bound on the capacity for signal detection.
Spike 
 Encoding
m(t)
k(t)
 Poisson 
Encoding
Stimulus
    Stochastic 
Vesicle Release
Variable EPSC
    Amplitude
 EPSP 
Shape 
P(a)
h(t)s(t)
Train
+
n(t)
v(t)
Membrane 
  Voltage
Noise
Synaptic Channel  Decoding
m(t)^
Wiener
Filter
 Optimal 
Estimator
Estimate
Decision
  Spike /
No Spike
Optimal 
Detector
Matched
  Filter
Y
!0 
!  1 
!  1 1 - 
!0 1 - 0 
1 
0 
1 
Vesicle Release 
Model 
Spike
No Spike
Release
No Release
Input 
  Spike /
No Spike
  Binary 
Encoding
X
Figure 1: Schematic block diagram for the signal detection and estimation tasks. The
synapse is modeled as a binary channel followed by a filter . where
is a random variable with probability density, .
The binary channel, (inset, = Pr[spontaneous release], = Pr [release failure]) models
probabilistic vesicle release and models the variable epsp size observed for cortical
synapses. denotes additive post-synaptic voltage noise and is assumed to be Gaussian
and white over a bandwidth . Performance of the optimal linear estimator (Wiener
Filter) and the optimal spike detector (Matched Filter) quantify synaptic efficacy for signal
estimation and detection respectively.
2 The Synaptic Channel
Synaptic transmission in cortical neurons is known to be highly random though the role of
this variability in neural computation and coding is still unclear. In central synapses, each
synaptic bouton contains only a single active release zone, as opposed to the hundreds or
thousands found at the much more reliable neuromuscular junction. Thus, in response to an
action potential in the presynaptic terminal at most one vesicle is released (Korn and Faber,
1991). Moreover, the probability of vesicle release is known to be generally low (0.1
to 0.4) from in vitro studies in some vertebrate and invertebrate systems (Stevens, 1994).
This unreliability is further compounded by the trial-to-trial variability in the amplitude of
the post-synaptic response to a vesicular release (Bekkers et. al, 1990). In some cases, the
variance in the size of EPSP is as large as the mean. The empirically measured distribution
of amplitudes is usually skewed to the right (possibly biased due the inability of measuring
very small events) and can be modeled by a Gamma distribution.
In light of the above, we model the synapse as a binary channel cascaded by a random am-
plitude filter (Fig. 1). The binary channel accounts for the probabilistic vesicle release.
and denote the probabilities of spontaneous vesicle release and failure respectively. We
follow the binary channel convention used in digital communications ( ), where-
as, is more commonly used in neurobiology. The filter is chosen to correspond to
the epsp profile of a fast AMPA-like synapse. The amplitude of the filter is modeled as
random variable with density , mean and standard deviation . The CV (standard
deviation/mean) of the distribution is denoted by . We also assume that additive Gaus-
sian voltage noise at the post-synaptic site further corrupts the epsp response. is
assumed to white with variance and a bandwidth corresponding to the membrane
time constant . One can define an effective signal-to-noise ratio, , given
by the ratio of the energy in the epsp pulse, to the noise power spectral
density, . The performance of the synapse depends on the and not on
the absolute values of or . In the above model, by regarding synaptic parameters as
constants, we have tacitly ignored history dependent effects like paired-pulse facilitation,
vesicle depletion, calcium buffering, etc, which endow the synapse with the nature of a
sophisticated nonlinear filter (Markram and Tsodyks, 1997).
X = 0
X = 1
Y = 0
Y = 1
Pm
Pf
1 - Pm
1 - Pf
No Spike
Spike
No Spike
Spike
Effective Binary
Detection Channel
b)
+
m(t) m(t)^
n(t)^
a)
Effective Continuous
Estimation Channel
Figure 2: (a) Effective chan-
nel model for signal estima-
tion. , , denote
the stimulus, the best linear es-
timate, and the reconstruction
noise respectively. (b) Effective
channel model for signal detec-
tion. and denote the bi-
nary variables corresponding to
the input and the decision re-
spectively. and are the
effective error probabilities.
3 Signal Estimation
Let us assume that the spike train of the presynaptic neuron can be modeled as a doubly
stochastic Poisson process with a rate given as a convolution between
the stimulus and a filter . The stimulus is drawn from a probability distribution
which we assume to be Gaussian. is a low-pass filter which models
the phenomenological relationship between a neuron’s firing rate and its input current.
is chosen to correspond to the membrane time constant. The exact form of is not
crucial and the above form is assumed primarily for analytical tractability. The objective is
to find the optimal estimator of from the post-synaptic voltage , where optimality
is in a least-mean square sense. The optimal mean-square estimator is, in general, non-
linear and reduces to a linear filter only when all the signals and noises are Gaussian.
However, instead of making this assumption, we restrict ourselves to the analysis of the
optimal linear estimator, , i.e. the filter which minimizes the
mean-square error where denotes an ensemble average. The
overall estimation system shown in Fig. 1 can be characterized by an effective continuous
channel (Fig. 2a) where denotes the effective reconstruction noise.
System performance can be quantified by E, the lower E, the better the synapse at signal
transmission. The expression for the optimal filter (Wiener filter) in the frequency domain is
where is the cross-spectral density (Fourier transform
of the cross-correlation ) of and and is the power spectral density of
. Theminimummean-square error is given by , .
The set is called the support of .
Another measure of system performance is the mutual information rate between
and , defined as the rate of information transmitted by about . By the
Data Processing inequality (Cover 1991), . A lower bound of
and thus of is given by the simple expression (units
of bits/sec). The lower bound is achieved when is Gaussian and is independent of
. Since the spike train is a Poisson process with rate ,
its power spectrum is given by the expression, where
is the mean firing rate. We assume that the mean ( ) and variance ( ) of are
chosen such that the probability that is negligible1 The vesicle release process
is the spike train gated by the binary channel and so it is also a Poisson process with rate
. Since is a filtered Poisson process, its power
spectral density is given by
. The cross-spectral density is given by the expression
. This allows us to write the mean-square error as,
Thus, the power spectral density of is given by . Notice
that if i.e. perfect reconstruction takes place in the limit of high
firing rates. For the parameter values chosen, , and can be ignored.
Consequently, signal estimation is shot noise limited and synaptic variability increases shot
noise by a factor . For and ,
and for and . If is chosen to be white, band-limited to
Hz, closed-form expressions for and can be obtained. The expression for is
tedious and provides little insight and so we present only the expression for below.
is a monotonic function of (decreasing) and (increasing). can be considered as
the effective number of spikes available per unit signal bandwidth and is the ratio of
the signal bandwidth and the neuron bandwidth. Plots of normalized reconstruction error
and versus mean firing rate ( ) for different values of signal bandwidth
are shown in Fig. 3a and Fig. 3b respectively. Observe that (bits/sec) is insensitive to
for firing rates upto 200Hz because the decrease in quality of estimation ( increases with
) is compensated by an increase in the number of independent samples ( ) available
per second. This phenomenon is characteristic of systems operating in the low SNR regime.
has the generic form, , where , and denote signal
bandwidth, signal power and noise power respectively. For low SNR,
, is independent of B. So one can argue that, for our choice of parameters, a single
synapse is a low SNR system. The analysis generalizes very easily to the case of multiple
synapses where all are driven by the same signal . (Manwani and Koch, in preparation).
However, instead of presenting the rigorous analysis, we appeal to the intuition gained from
the single synapse case. Since a single synapse can be regarded as a shot noise source,
parallel synapses can be treated as parallel noise sources. Let us make the plausible
1We choose and so that (std of ) so that Prob[ .
assumption that these noises are uncorrelated. If optimal estimation is carried out separately
for each synapse and the estimates are combined optimally, the effective noise variance
is given by the harmonic mean of the individual variances i.e. .
However, if the noises are added first and optimal estimation is carried out with respect
to the sum, the effective noise variance is given by the arithmetic mean of the individual
variances, i.e. . If we assume that all synapses are similar so that
, . Plots of and for the case of 5 identical synapses are
shown in Fig. 3c and Fig. 3d respectively. Notice that increases with suggesting
that the system is no longer in the low SNR regime. Thus, though a single synapse has very
low capacity, a small amount of redundancy causes a considerable increase in performance.
This is consistent with the fact the in the low regime, increases linearly with ,
consequently, linearly with , the number of synapses.
0 20 40 60 80 100 120 140 160 180 200
0.4
0.5
0.6
0.7
0.8
0.9
1
No
rm
ali
ze
d E
rro
r
0 20 40 60 80 100 120 140 160 180 200
0.4
0.5
0.6
0.7
0.8
0.9
1
Firing Rate (Hz)
No
rm
ali
ze
d E
rro
r
0 20 40 60 80 100 120 140 160 180 200
0
2
4
6
8
10
12
14
16
Inf
o. 
Ra
te 
(bi
ts/
se
c)
0 20 40 60 80 100 120 140 160 180 200
0
2
4
6
8
10
12
14
16
Firing Rate (Hz)
In
fo
. R
at
e 
(b
its
/se
c)
a)
m
B  = 10Hz  
B  = 25 Hz 
B  = 50 Hz 
B  = 75 Hz 
B  = 100 Hz
m
m
m
m
m
B  = 10Hz  
B  = 25 Hz 
B  = 50 Hz 
B  = 75 Hz 
B  = 100 Hz
m
m
m
m
m
b)
c) d)
Figure 3: and vs. mean firing rate ( ) for n = 1 [(a) and (b)] and n = 5 [(c) and (d)] identical
synapses respectively (different values of ) for signal estimation. Parameter values are ,
, = 0.6, = 0.5 msec, = 10msec, = 0.1 mV, = 100 Hz.
4 Signal Detection
The goal in signal detection is to decide which member from a finite set of signals was
generated by a source, on the basis of measurements related to the output only in a statistical
sense. Our example corresponds to its simplest case, that of binary detection. The objective
is to derive an optimal spike detector based on the post-synaptic voltage in a given time
interval. The criterion of optimality is minimum probability of error ( ). A false alarm
(FA) error occurs when a spike is falsely detected even when no presynaptic spike occurs
and a miss error (M) occurs when a spike fails to be detected. The probabilities of the errors
are denoted by and respectively. Thus, where denotes
the a priori probability of a spike occurrence. Let and be binary variables denoting
spike occurrence and the decision respectively. Thus, if a spike occurred else
. Similarly, expresses the decision that a spike occurred. The posterior likelihood
ratio is defined as and the prior likelihood as
. The optimal spike detector employs the well-known likelihood ratio
test , “If Y = 1 elseY = 0”. When , else .
Since is a random variable, . If
the noise is Gaussian and white, it can be shown that the optimal decision rule reduces
to a matched filter2, i.e. if the correlation, between and exceeds a particular
threshold (denoted by ), else . The overall decision system shown in
Fig. 1 can be treated as effective binary channel (Fig. 2b). The system performance can
be quantified either by or , the mutual information between the binary random
variables, and . Note that even when , due to the
unreliability of vesicular release. Let denote the probability of error when .
If , is the minimum possible detection error. Let and denote FA
and M errors when the release is ideal ( ). It can be shown that
Both and depend on . The optimal value of is chosen such that is minimized.
In general, and can not be expressed in closed-form and the optimal is found using
the graphical ROC analysis procedure. If we normalize such that , and can
be parametrically expressed in terms of a normalized threshold , ,
. can be computed using the
formula for the mutual information for a binary channel,
where is
the binary entropy function. The analysis can be generalized to the case of synapses but
the expressions involve -dimensional integrals which need to be evaluated numerically.
The Central Limit Theorem can be used to simplify the case of very large . Plots of
and versus for different values of (1,10, ) for the case of identical
synapses are shown in Fig. 4a and Fig. 4b respectively. Yet again, we observe the poor
performance of a single synapse and the substantial improvement due to redundancy. The
linear increase of with is similar to the result obtained for signal estimation.
5 Conclusions
We find that a single synapse is rather ineffective as a communication device but with
a little redundancy neuronal communication can be made much more robust. Infact, a
single synapse can be considered as a low SNR device, while 5 independent synapses
in parallel approach a high SNR system. This is consistently echoed in the results for
signal estimation and signal detection. The values of information rates we obtain are very
small compared to numbers obtained from some peripheral sensory neurons (Rieke et. al,
1996). This could be due to an over-conservative choice of parameter values on our part
or could argue for the preponderance of redundancy in neural systems. What we have
presented above are preliminary results of work in progress and so the path ahead is much
2For deterministic , the result is well-known, but even if is a one-sided random variable, the
matched filter can be shown to be optimal.
a)
1 2 3 4 5
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Pr
ob
ab
ilit
y o
f E
rro
r 
Number of Synapses (n)
SNR = Inf.
SNR = 10  
SNR = 1   
1 2 3 4 5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Inf
or
ma
tio
n R
ate
 (b
its
/us
e)
Number of Synapses (n)
SNR = Inf.
SNR = 10  
SNR = 1   
b)
(a) and (b) vs. the number of synapses, , (different values of ) for signal detection.
= Inf. corresponds to no post-synaptic voltage noise. All the synapses are assumed to be
identical. Parameter values are = 0.5, , , = 0.6, = 0.5 msec, = 10 msec,
= 0.1 mV, = 100 Hz.
longer than the distance we have covered so far. To the best of our knowledge, analysis
of distinct individual components of a neuron from an communications standpoint has not
been carried out before.
Acknowledgements
This research was supported by NSF, NIMH and the Sloan Center for Theoretical Neuro-
science. We thank Fabrizio Gabbiani for illuminating discussions.
References
Bekkers, J.M., Richerson, G.B. and Stevens, C.F. (1990) “Origin of variability in quantal
size in cultured hippocampal neurons and hippocampal slices,” Proc. Natl. Acad. Sci. USA
87: 5359-5362.
Bialek, W. Rieke, F. van Steveninck, R.D.R. and Warland, D. (1991) “Reading a neural
code,” Science 252: 1854-1857.
Cover, T.M., and Thomas, J.A. (1991) Elements of Information Theory. New York: Wiley.
Korn, H. and Faber, D.S. (1991) “Quantal analysis and synaptic efficacy in the CNS,”
Trends Neurosci. 14: 439-445.
Markram, H. and Tsodyks, T. (1996) “Redistibution of synaptic efficacy between neocorti-
cal pyramidal neurons,” Nature 382: 807-810.
Rieke, F. Warland, D. van Steveninck, R.D.R. and Bialek, W. (1996) Spikes: Exploring the
Neural Code. Cambridge: MIT Press.
Stevens, C.F. (1994) “What form should a cortical theory take,” In: Large-Scale Neuronal
Theories of the Brain,Koch, C. and Davis, J.L., eds., pp. 239-256. Cambridge: MIT Press.


Synaptic Transmission: An Information-Theoretic Perspective

Here we analyze synaptic transmission from an information-theoretic perspective. We derive closed-form expressions for the lower-bounds on the capacity of a simple model of a cortical synapse under two explicit coding paradigms. Under the &quot;signal estimation&quot; paradigm, we assume the signal to be encoded in the mean firing rate of a Poisson neuron. The performance of an optimal linear estimator of the signal then provides a lower bound on the capacity for signal estimation. Under the &quot;signal detection&quot; paradigm, the presence or absence of the signal has to be detected. Performance of the optimal spike detector allows us to compute a lower bound on the capacity for signal detection. We find that single synapses (for empirically measured parameter values) transmit information poorly but significant improvement can be achieved with a small amount of redundancy. 1 Introduction  Tools from estimation and information theory have recently been applied by researchers (Bialek et. al, 1991) to qua..

Amit Manwani

Christof Koch

CiteSeerX

Here we analyze synaptic transmission from an information-theoretic perspective. We derive closed-form expressions for the lower-bounds on the capacity of a simple model of a cortical synapse under two explicit coding paradigms. Under the “signal estimation ” paradigm, we assume the signal to be encoded in the mean firing rate of a Poisson neuron. The performance of an optimal linear estimator of the signal then provides a lower bound on the capacity for signal estimation. Under the “signal detection ” paradigm, the presence or absence of the signal has to be de-tected. Performance of the optimal spike detector allows us to compute a lower bound on the capacity for signal detection. We find that single synapses (for empirically measured parameter values) transmit informa-tion poorly but significant improvement can be achieved with a small amount of redundancy. 

Synaptic Transmission: An Information-Theoretic Perspective

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main

CiteSeerX

CiteSeerX