Linear decoding of a complex movie.

<p><b>A</b>: An example stimulus frame. At each site (red dots = partially shown 20×20 grid) the stimulus was convolved with a spatial gaussian filter (red circle = 1<i>σ</i>). Typical RGC receptive field center size shown in gray. <b>B</b>: Responses of 91 RGCs with 750 <i>ms</i> decoding window overlaid in blue. <b>C</b>: Three example luminance traces (red) and the linear decoders’ predictions (blue). <b>D</b>: Decoded frame (same as in <b>A</b>) reconstructed from 20×20 separately decoded traces. Disc contours of the original frame shown for reference in green. <b>E</b>: RF centers of the 91 cells (black dots = centers of fitted ellipses). RF centers overlapping a chosen site (red dot) are highlighted in blue. <b>F</b>: Performance of the linear decoders across space, as Fraction of Variance Explained (FVE). Black dots as in <b>E</b>; black contour is the boundary <i>FVE</i> = 0.4. <b>G</b>: Performance of the linear decoders (FVE) across sites as a function of cell coverage (grayscale = conditional histograms, red dots = means, error bars = ± SD). <b>H</b>: Average decoding error across sites (MSE ± SD) of 10-disc-trained decoders, tested on withheld stimuli with different numbers of discs. <b>I</b>: Cells (black dots = RF center positions) contributing to the decoding at two example sites (red circles); decoding filters shown below. For each site, contributing cells (highlighted in red and joined to the site) account for at least half of the total L1 norm. <b>J</b>: Decoding field of a single cell (here, evaluated over a denser 50×50 grid and normalized to unit maximal variance); the cell’s RF center shown in black.</p

Spike-history dependencies affect decoding performance.

<p><b>A</b>: Shuffles of responses to repeated stimulus presentations remove different types of correlations, but preserve average locking to the stimulus (PSTH), and thus stimulus-induced correlations. <b>B</b>: A repeated stimulus fragment (red trace), nonlinear kernelized decoder predictions using real responses (green), and using responses without different types of correlations (gray); shown is the prediction mean ± SD over repeats. <b>C</b>: Increase in decoding error (MSE) when spike-history dependencies or noise correlations are removed (average ± SEM across sites); percentages report fractional differences relative to the original performance. <b>D</b>: Spike count distributions for a single example cell. Removing spike-history dependencies broadens the distributions, in particular in constant epochs. Dashed line = expectation for a fully randomized spike train with a matched firing rate. <b>E</b>: Variance-to-mean ratio <i>F</i> of spike count distributions for spike trains with and without spike-history dependencies. Each point is a cell that contributes most to decoding at a particular site (when the same cell contributes to multiple sites, average ± SD across sites is shown).</p

Nonlinear decoding outperforms linear decoding.

<p><b>A</b>: Luminance trace (red) with linear (blue) and nonlinear KRR (green) and neural network (grey) predictions. <b>B</b>: Average decoder performance (± SD across sites), achievable using increasing numbers of cells with highest L1 filter norm. For nonlinear decoding, “All” is the optimal subset that maximizes performance (<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1006057#pcbi.1006057.s007" target="_blank">S7 Fig</a>). Since the neural network (grey point with an error bar) simultaneously decodes the movie at all sites, it only makes sense to train it using “All” cells. <b>C</b>: Average ROC across all testing movie frames. <b>D</b>: Fractional improvement (average ± SEM across sites) of nonlinear KRR versus linear decoders for test stimuli with different numbers of discs. All decoders were trained only on the 10-disc stimulus. <b>E</b>: Decoding error (MSE; average ± SEM across sites) in fluctuating and constant epochs is significantly larger for linear decoders (p<0.001) relative to nonlinear KRR and the neural network.</p

Spike-history dependencies of intermediate strength facilitate nonlinear decoding in simple models of neural processing.

<p><b>A</b>: Schematic of a single-cell Generalized Linear Model (see <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1006057#sec002" target="_blank">Methods</a>). The neuron’s sensitivity to the stimulus is determined by a radially symmetric difference-of-Gaussians spatial filter that has a monophasic timecourse (), and combines additively with the neuron’s sensitivity to its own past spiking, given by filter (with strong refractoriness followed by weak facilitation). Importantly, shapes spike-history dependencies in the resulting spike trains. A nonlinear function <i>f</i>(⋅) (here, threshold-linear) of the combined sensitivities gives the neuron’s instantaneous firing rate that can be used to generate individual spike train instances. Shapes, as well as the temporal and spatial scales of the filters, were realistic for our data. <b>B</b>: Example rasters (50 repeats) generated with the encoding model for a given intensity trace and different magnitudes (<i>α</i>) of spiking history filter . The rasters are matched in PSTH (bottom) but differ in temporal noise correlations. <b>C</b>: Average spike count variance-to-mean ratio, <i>F</i>, (± SD) of the model as a function of <i>α</i> in fluctuating and constant epochs. <b>D</b>: Decoding error as a function of <i>α</i>. Decoders are trained for each separate <i>α</i> and tested on withheld stimuli; shade = SD over 10 spike train realizations.</p

Entropy and multi-information from the K-pairwise model.

<p>(<b>A</b>) Independent entropy per neuron, , in black, and the entropy of the K-pairwise models per neuron, , in red, as a function of <i>N</i>. Dashed lines are fits from (B). (<b>B</b>) Independent entropy scales linearly with <i>N</i> (black dashed line). Multi-information of the K-pairwise models is shown in dark red. Dashed red line is a best quadratic fit for dependence of on ; this can be rewritten as , where <i>γ</i>(<i>N</i>) (shown in inset) is the effective scaling of multi-information with system size <i>N</i>. In both panels, error bars are s.d. over 30 subnetworks at each size <i>N</i>.</p

The number of identified metastable patterns.

<p>Every recorded pattern is assigned to its basin of attraction by descending on the energy landscape. The number of distinct basins is shown as a function of the network size, <i>N</i>, for K-pairwise models (black line). Gray lines show the subsets of those basins that are encountered multiple times in the recording (more than 10 times, dark gray; more than 100 times, light gray). Error bars are s.d. over 30 subnetworks at every <i>N</i>. Note the logarithmic scale for the number of MS states.</p

K-pairwise model for a the same group of <i>N</i> = 100 cells shown in Figure 1.

<p>The neurons are again sorted in the order of decreasing firing rates. (<b>A</b>) Pairwise interactions, , and the comparison with the interactions of the pairwise model, (<b>B</b>). (<b>C</b>) Single-neuron fields, , and the comparison with the fields of the pairwise model, (<b>D</b>). (<b>E</b>) The global potential, <i>V</i>(<i>K</i>), where <i>K</i> is the number of synchronous spikes. See <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003408#s4" target="_blank"><i>Methods</i></a><i>: Parametrization of the K-pairwise model</i> for details.</p

Predicted vs real connected three–point correlations, from Eq (21).

<p>(<b>A</b>) Measured (x-axis) vs predicted by the model (y-axis), shown for an example 100 neuron subnetwork. The ∼1.6×10⁵ triplets are binned into 1000 equally populated bins; error bars in x are s.d. across the bin. The corresponding values for the predictions are grouped together, yielding the mean and the s.d. of the prediction (y-axis). Inset shows a zoom-in of the central region, for the K-pairwise model. (<b>B</b>) Error in predicted three-point correlation functions as a function of subnetwork size <i>N</i>. Shown are mean absolute deviations of the model prediction from the data, for pairwise (black) and K-pairwise (red) models; error bars are s.d. across 30 subnetworks at each <i>N</i>, and the dashed line shows the mean absolute difference between two halves of the experiment. Inset shows the distribution of three–point correlations (grey filled region) and the distribution of differences between two halves of the experiment (dashed line); note the logarithmic scale.</p

Predicting the firing probability of a neuron from the rest of the network.

<p>(<b>A</b>) Probability per unit time (spike rate) of a single neuron. Top, in red, experimental data. Lower traces, in black, predictions based on states of other neurons in an <i>N</i>–cell group, as described in the text. Solid lines are the mean prediction across all trials, and thin lines are the envelope ± one standard deviation. (<b>B</b>) Cross–correlation (CC) between predicted and observed spike rates vs. time, for each neuron in the <i>N</i> = 120 group. Green empty circles are averages of CC computed from every trial, whereas blue solid circles are the CC computed from average predictions. (<b>C</b>) Dependence of CC on the population size <i>N</i>. Thin blue lines follow single neurons as predictions are based on increasing population sizes; red line is the cell illustrated in (A), and the line with error bars shows mean ± s.d. across all cells. Green line shows the equivalent mean behavior computed for the green empty circles in (B).</p

Predicted vs real distributions of energy, <i>E</i>.

<p>(<b>A</b>) The cumulative distribution of energies, from <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003408#pcbi.1003408.e128" target="_blank">Eq (22)</a>, for the K-pairwise models (red) and the data (black), in a population of 120 neurons. Inset shows the high energy tails of the distribution, from <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003408#pcbi.1003408.e131" target="_blank">Eq (24)</a>; dashed line denotes the energy that corresponds to the probability of seeing the pattern once in an experiment. See <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003408#pcbi.1003408.s005" target="_blank">Figure S5</a> for an analogous plot for the pairwise model. (<b>B</b>) Relative difference in the first two moments (mean, , dashed; standard deviation, , solid) of the distribution of energies evaluated over real data and a sample from the corresponding model (black = pairwise; red = K-pairwise). Error bars are s.d. over 30 subnetworks at a given size <i>N</i>.</p