Semiparametric pairwise model outperforms other models.

<p><b>A)</b> Out-of-sample log-likelihood improvement relative to the pairwise model per sample per neuron averaged over subnetworks. Error bars denote variation over subnetworks (1 SD, no errorbars for <i>N</i> = 160 since there is only one subpopulation of that size in the entire dataset). The error in likelihood estimation is much smaller than the displayed error bars. <b>B)</b> The same as in A) but for single populations from two different experiments–one in which the population is stimulated with a random checkerboard stimulus, and the other where the population responds to a full-field flickering. <b>C)</b> The test set error rate averaged over neurons for predicting the response of a neuron from the activities of other neurons in 5 different subpopulations of 100 neurons. <b>D)</b> Average (across neurons) error rate decrease achieved by using a semiparametric pairwise model instead of a K-pairwise model for subpopulations of various sizes. Error bars denote 1 SD variation over subnetworks.</p

Semiparametric independent model reproduces the empirical Zipf plot.

<p>Each curve shows the probabilities of population activity patterns, <i>P</i>(<b>s</b>), sorted in decreasing order on a log-log plot. To construct the empirical Zipf plot, we directly sampled the frequencies of different patterns from data. To construct model predictions, we used the same procedure but replaced real data with artificial datasets of the same size, generated by drawing the samples from the corresponding model. Error bars are 3 SD (bootstrapped).</p

Overview of models which contain mechanisms for capturing global coupling.

<p>At any given time, the population activity pattern is defined by neurons which either spike (<i>s</i>_<i>i</i> = 1, dark discs) or are silent (<i>s</i>_<i>i</i> = 0, white discs). The probability of spiking is partially determined by an intrinsic firing bias (<i>α</i>_<i>i</i> for models without local interactions, or the diagonal terms of the coupling matrix <i>J</i> for models with local pairwise interactions). When local interactions between neurons are important, they can be parametrized by assigning each pair of neurons a coupling weight. Positive weight (orange) increases the likelihood of the paired neurons spiking together, while negative weight (blue) decreases the likelihood. The negative sum of the intrinsic firing biases of active neurons and the coupling weights of pairs which fire synchronously is referred to as the energy of the population activity pattern. The probability of a given pattern is simply proportional to the exponential of its negative energy. To capture correlations due to global coupling, previous studies considered models which bias the response probabilities with a function of the total network activity (here denoted as <i>K</i>, i.e., the sum of the activities of individual neurons). We introduce a different approach (shaded models in the figure) where global coupling is induced by mapping the energy of the activity pattern to its probability with an arbitrary (smooth and increasing) function exp(−<i>V</i> (<i>E</i>)).</p

Properties of the inferred nonlinearity for neural networks of increasing size.

<p><b>A)</b> Comparison between the inferred nonlinearity in the range of energies observed in the dataset and the log of the density of states at the same energies, showing the increasing match between the two quantities as the population size, <i>N</i>, increases. Both axes are normalized by the population size so that all curves have a similar scale. Nonlinearity can be shifted by an arbitrary constant without changing the model; to remove this redundancy, we set <i>V</i> (0) = 0 for all nonlinearities. <b>B)</b> The population size dependence of the average squared distance between the density of states and the inferred nonlinearity. Since the nonlinearity can be shifted by an arbitrary constant, we chose this offset so as to minimize the average squared distance. Error bars (1 SD) denote variation over different subnetworks. <b>C)</b> Inferred nonlinearities map to latent variables whose probability distributions can be computed and plotted for one sequence of subnetworks increasing in size (colors). As the network size increases, the dynamic range of the latent variable distribution does as well, which is quantified by the entropy of the distributions (inset).</p

The most likely value of the latent variable naturally defines two global population states.

<p><b>A)</b> For every repeat of the stimulus and for every time bin we estimate the most likely value of the latent (<i>h</i>*) given the population response at that time, as well as the total number of spiking neurons in that response (<i>K</i>). The plot shows the trajectories of <i>h</i>* and <i>K</i> averaged across repeats. Error bars correspond to 1 SD. <b>B)</b> Probability density of <i>h</i>*, i.e. the most likely value of the latent given the population response. <b>C)</b> A scatter plot of the total network activity vs. the most likely value of the latent. <i>h</i>* naturally divides the population responses into two clusters. <b>D)</b> Probability distribution of the total network activity given this global population state. While the most likely value of <i>K</i> for low <i>h</i>* is zero, the distribution has a tail that extends to <i>K</i> ≈ 5.</p

Comparison of the semiparametric independent and the independent model.

<p><b>A)</b> Probability distributions of the total activity of the population, <i>K</i>(<b>s</b>) = ∑_<i>i</i> <i>s</i>_<i>i</i>, estimated from data and from model samples. Error bars are 3 SD (bootstrapped), with the model-generated sample size equal to that of the data. <b>B)</b> Comparison of the firing rates estimated from the data and from the model samples. The firing rates predicted by the independent model should exactly match the true firing rates. Error bars are 3 SD (bootstrapped). <b>C)</b> Comparison of the predicted pairwise covariance matrix elements estimated from the model and from data, for the semiparametric independent and the independent models. The scatter of independent model covariance elements around 0 illustrates the magnitude of the sampling noise.</p

Long-range interactions can stabilize Counter patterns against variations in system size and morphogen signal.

<p>(A) Morphogen signal for three different system sizes, <i>N</i> = 50, 60, 70. Posterior boundary of the system is depicted as a dashed line. We consider how a two-gene Counter network, optimized for <i>N</i> = 60, changes upon variations in system size. (B) Resulting patterns for a Counter network for three system sizes depicted in (A). The pattern does not scale with the system size; instead, boundaries form at the same absolute location. (C) Resulting patterns for the same network as in (B) with additional negative long range interactions. Pattern shifts with system size are largely suppressed. (D) Spatial interaction strength as a function of distance. Nearest neighbors are interacting positively, while sites further away are coupled negatively with exponentially decaying strength of maximal amplitude . (E) We perturb the morphogen signal with a uniform perturbation <i>ϵ</i>, . Example patterns generated by the optimal Counter two-gene network, at different strengths of long range interactions, , and different morphogen signal perturbation magnitudes, <i>ϵ</i>. At <i>ϵ</i> = 0, irrespective of , the system generates the optimal pattern. When |<i>ϵ</i>| increases, the patterns shift (providing less PI) with weak , but when is strong, the pattern is robust to such perturbations. (F) To quantify the robustness to <i>ϵ</i> perturbations, we compute the overlap, , of the resulting pattern with the optimal Counter pattern. The overlap is shown as a function of <i>ϵ</i> and ; for strong negative , the overlap is high irrespective of the perturbation strength, <i>ϵ</i>. (G) Susceptibility to small perturbations <i>ϵ</i> as a function of shows transition into a robust regime, <i>χ</i>_<i>m</i> → 0, as increases in magnitude.</p

Comparison of PI in input gradient and output pattern.

<p>PI in <i>m</i> (solid) and <b><i>σ</i></b> (dashed) are shown for three different input noise levels ((A)-(C)) as a function of <i>χ</i>, parametrizing the gradient shape (cf. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0163628#pone.0163628.g002" target="_blank">Fig 2A</a>). The computation of is described in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0163628#pone.0163628.s003" target="_blank">S3 Appendix</a>.</p

Positional information carried by two and three patterning genes with a linear morphogen signal.

<p>(A) PI as a function of intrinsic noise level. For each noise level <i>η</i>, all parameters of the network have been optimized. The depicted regions (i)-(iii) indicate different numbers of states encoded by the resulting patterns. At each boundary (non-optimal) continuations of the coding strategy in the neighboring region are depicted as dotted curves. (B) Characteristic patterns of the different regions in (A). (C) Schematics of the network parameters for the different coding strategies. Pointed arrows denote positive interaction, blunted arrows denote negative interaction. The thickness of the arrows indicates their strength. (D) Comparison of PI carried by systems without spatial interactions and systems without local gene-gene interactions as a function of intrinsic noise. For each noise level the parameters of each system have been optimized separately. (E) PI carried by three patterning genes as a function of intrinsic noise. (F) Characteristic three-gene patterns for the different regions in (E).</p

Effect of spatial interactions on one patterning gene.

<p>(A) PI as a function of spatial interaction strength <i>J</i> with fixed intrinsic noise <i>η</i> = 2 and two levels of extrinsic noise (legend). The arrow indicates the excess PI available to an optimally spatially coupled system, relative to an uncoupled (<i>J</i> = 0) system. At nonzero extrinsic noise (<i>ν</i> = 0.1) PI is strongly suppressed at <i>J</i> < 0. (B) The average spatial pattern for three regions denoted in (A): (i) no spatial interaction; (ii) positive <i>J</i> stabilizes a pattern with a boundary against noise; (iii) negative <i>J</i> results in an alternating pattern. In region (iv) indicated in (A) the strength of spatial interactions forces the pattern in a uniform all ON or all OFF state that carries 0 bits of positional information. (C) Comparison between PI with (dotted line) and without (solid line) spatial interaction as a function of intrinsic noise, <i>η</i>. For the spatially coupled system an optimal <i>J</i>_opt has been found separately for each value of intrinsic noise <i>η</i>. The corresponding values of <i>J</i>_opt, scaled by the respective noise levels <i>η</i>, are plotted in the inset.</p