Network connectivity.

<p><b>A:</b> Schematic of an assembly <i>i</i> consisting of an excitatory (<i>E</i>_<i>i</i>) and an inhibitory (<i>I</i>_<i>i</i>) population. Red and blue lines indicate excitatory and inhibitory connections, respectively. The symbols <i>w</i> and −<i>kw</i> denote total synaptic couplings between populations. <b>B:</b> Sketch of network connectivity. The inhomogeneous network is randomly connected with connection probability <i>p</i>_rand. A feedforward structure consisting of 10 assemblies (only <i>i</i> − 1 and <i>i</i> shown) is embedded into the network. Each assembly is formed by recurrently connecting its neurons with probability <i>p</i>_rc. Subsequent assemblies are connected with feedforward probability <i>p</i>_ff between their excitatory neurons. <b>C:</b> Embedded structure as a function of connectivities.</p

Feedforward conductance versus feedforward connectivity.

<p><b>A:</b> Quality of replay as a function of connectivity and synaptic strength. <b>B:</b> The replay as a function of connectivity and total feedforward conductance input shows that the propagation is independent of connectivity as long as the total feed-forward input is kept constant. <b>C:</b> Spontaneous network dynamics described by the rate of spontaneous replay, synchrony, CV, and firing rate.</p

Assembly-sequence activation for various group sizes and connectivities.

<p><b>A:</b> Simulation results for the quality of replay. <b>B:</b> Rate of spontaneous replay. <b>C:</b> Synchrony. <b>D:</b> Coefficient of variation <b>E:</b> Firing rate. <i>ρ</i>₀ = 5 spikes/sec is the target firing rate. In C, D, and E quantities are averaged over the neurons in the last group of the sequence. The black line is an analytical estimate for the evoked replay as in Figs <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005359#pcbi.1005359.g003" target="_blank">3</a> and <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005359#pcbi.1005359.g005" target="_blank">5</a>.</p

Acrobot task.

<p>A: The acrobot swing-up task figures a double pendulum, weakly actuated by a torque at the joint. The state of the pendulum is represented by the two angles and and the corresponding angular velocities and . The goal is to lift the tip above a certain height above the fixed axis of the pendulum, corresponding to the length of the segments. B: Goal reaching latency of TD-LTP agents. The solid line shows the median of the latencies for each trial number and the shaded area represents the 25th to 75th percentiles of the agents performance. The red line represents a near-optimal strategy, obtained by the direct search method (see <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#s4" target="_blank">Models</a>). The blue line show the trajectory of one of the best amongst the 100 agents. The dotted line shows the limit after which a trial was interrupted if the agent did not reach the goal. C: Example trajectory of an agent successfully reaching the goal height (green line).</p

Alternative learning rule and nuisance term.

<p>A: Schematic comparison of the squared TD gradient learning rule of <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e533" target="_blank">Eq. 46</a> and TD-LTP, similar to <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002" target="_blank">Figure 2A</a>. B: Linear track task using the squared TD gradient rule. Same conventions as in <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002" target="_blank">Figure 2C</a>. C: linear track task using the TD-LTP rule (reprint of <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002" target="_blank">Figure 2C</a> for comparison). D: Integrands of the disturbance term for Poisson spike train statistics. Top: squared TD gradient rule. Bottom: TD-LTP rule. In each plot the numerical value under the curve is given. This corresponds to the contribution of each presynaptic spike to the nuisance term. E: Disturbance term dependence on for the squared TD gradient rule. The mean weight change under initial conditions on an unrewarded linear track task with frozen weights, using the squared TD gradient learning rule, is plotted versus , the number of neurons composing the critic. Each cross corresponds to the mean over a 200s simulation, the plot shows crosses for each condition. The line shows a fit of the data with , the dependence form suggested by <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e574" target="_blank">Eq. 50</a>. F: Same as E, for critic neurons using the TD-LTP learning rule. G, H: Same experiment as E and F, but using a rate neuron model with Gaussian noise of mean 0 and variance . The line shows a fit with , the dependence form suggested by <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e574" target="_blank">Eq. 50</a>.</p

Comparison of the Learning Window with Experimental Data

<p>The plot compares the theoretically predicted learning window with experimental data from hippocampal pyramidal cells as published by Bi and Poo [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.0030112#pcbi-0030112-b016" target="_blank">16</a>] (larger plot in the middle). Instead of the ideal power spectrum with the abrupt cutoff at <i>ν</i>_max as stated in <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.0030112#pcbi-0030112-e020" target="_blank">Equation 20</a>, a Cauchy function with <i>γ</i> = 1 / 15 ms was used (top left, the dashed line is for <i>ν</i>_max = 1 / (40 ms)). Again, the EPSP decay time was <i>τ</i> = 40 ms. This learning window corresponds to an implementation of the “trace rule” [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.0030112#pcbi-0030112-b001" target="_blank">1</a>,<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.0030112#pcbi-0030112-b004" target="_blank">4</a>,<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.0030112#pcbi-0030112-b006" target="_blank">6</a>] for a decay time of the exponential filter of 15 ms. </p

Pattern completion.

<p><b>A:</b> Quality of replay after partial activation of the first group for cue size 60% (left panel) and 20% (middle) as a function of feedforward and recurrent connectivity. The right-most panel shows the quality replay after a cue activation (20% and 60%) as a function of the recurrent connectivity (<i>p</i>_rc) while the feedforward connectivity is constant (<i>p</i>_ff = 0.05). <b>B:</b> Examples of network activity during 60% (left) and 20% (right) cue activation. The top and bottom raster plots correspond to assembly sequences with higher (<i>p</i>_rc = 0.10, top) and lower (<i>p</i>_rc = 0.06, bottom) recurrent connectivity, highlighted in A with white and black rectangles, respectively. <b>C:</b> State-space portraits representing the pulse-packet propagation. The activity in each group is quantified by the fraction of firing excitatory neurons (<i>α</i>) and the standard deviation of their spike times (<i>σ</i>). The initial stimulations are denoted with small black dots while the colored dots denote the response of the first group to the stimulations; red dot if the whole sequence is activated, and blue otherwise. Stimulations in the region with white background result in replays, while stimulating in the gray region results in no replay. The black arrows illustrate the evolution of pulse packets during the replays in B. Top: <i>p</i>_rc = 0.10; bottom: <i>p</i>_rc = 0.06.</p

Evoked replay.

<p>Assembly-sequence activation as a function of the feedforward <i>p</i>_ff and the recurrent <i>p</i>_rc connectivities. The color code denotes the quality of replay, that is, the number of subsequent groups firing without bursting (see <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005359#sec015" target="_blank">Materials and Methods</a>). The black curve corresponds to the critical connectivity required for a replay where the slope <i>c</i> of the transfer function (See <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005359#sec015" target="_blank">Materials and Methods</a> and <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005359#pcbi.1005359.e006" target="_blank">Eq 1</a>) is matched manually to fit the simulation results for connectivities <i>p</i>_rc = 0.08 and <i>p</i>_ff = 0.04. The slope <i>c</i> is also estimated analytically (dashed white line). The raster plots (<b>a-f</b>) illustrate the dynamic regimes observed for different connectivity values; neurons above the gray line belong to the background neurons.</p

Symmetric assembly sequence.

<p><b>A:</b> Schematic of an assembly sequence with symmetric connections between groups. <b>B:</b> Virtual rat position on a linear track (top) and the corresponding neuronal activity (bottom) as a function of time for 2 seconds. The rat rests at position “b” for half a second, then moves from “b” to “e” with constant speed for one second, where it rests for another 500 ms. While the rat is immobile at both ends of the track, a positive current input <i>I</i>^<i>e</i> = 2 pA is applied to the excitatory population of the first and last assembly as shown by the red background in the raster plot. Spontaneous replays start from the cued assemblies. During exploration, however, the network activity is decreased by a current <i>I</i>^<i>e</i> = −10 pA injected to the whole excitatory population, denoted with a blue horizontal bar. Strong sensory input during traversal activates the location-specific assemblies but does not result in any replay. The timing and location of the stimulations is denoted with red vertical bars in the raster plot. Recurrent and feedforward connectivities are <i>p</i>_rc = 0.15 and <i>p</i>_ff = 0.03, respectively.</p

Relation between the EPSP and the Learning Window

<p>The power spectrum is the Fourier transform of the effective learning window <i>W</i>₀, which in turn is the convolution of the learning window <i>W</i> and the EPSP <i>ε</i>. The figure shows the learning windows required for SFA for three different EPSP durations (<i>τ</i> = 4, 40, 400 ms). The maximal input frequency <i>ν</i>_max was 1 / (40 ms) in all plots. </p