Soft-bound synaptic plasticity increases storage capacity

Accurate models of synaptic plasticity are essential to understand the adaptive properties of the nervous system and for realistic models of learning and memory. Experiments have shown that synaptic plasticity depends not only on pre- and post-synaptic activity patterns, but also on the strength of the connection itself. Namely, weaker synapses are more easily strengthened than already strong ones. This so called soft-bound plasticity automatically constrains the synaptic strengths. It is known that this has important consequences for the dynamics of plasticity and the synaptic weight distribution, but its impact on information storage is unknown. In this modeling study we introduce an information theoretic framework to analyse memory storage in an online learning setting. We show that soft-bound plasticity increases a variety of performance criteria by about 18% over hard-bound plasticity, and likely maximizes the storage capacity of synapses

The effect of imbalance between potentiation and depression on the capacity measures for hard-bound plasticity.

<p>A. The Information capacity showing the theory (black) as well as simulations for 10, 100, 1000, and 10000 synapses for (blue, violet and magenta). In contrast to the balanced case () the capacity depends on the number of synapses. A larger synaptic update always decreases capacity in the balanced case, but can improve capacity in the imbalanced case (dashed curve, , ). B. Memory lifetime decreases when potentiation and depression are imbalanced. The memory lifetime was optimized w.r.t for every setting of and .</p

Illustration of decay of the weight distributions after a potentiation event.

<p>The distribution right after the potentiation is shown by the magenta curve; as time progresses (indicated by the arrow) it decays back to the equilibrium distribution (thick black curve). A) With soft-bound plasticity the distribution is displaced but maintains its shape. During the overwriting it shifts back to the equilibrium distribution. B) With hard-bound plasticity, the distribution distorts after the potentiation due to the presence of the bounds. As it decays back to the equilibrium this distortion flattens out.</p

Relation between the information and the SNR.

<p>Top: The SNR decay curves versus pattern age for soft-bound plasticity with a large synaptic update (thin curve), and soft-bound plasticity with a small update (thick curve). Although the rules trade off between slow decay and the high initial SNR differently, the area under the curve is identical. Middle: The relation between SNR and Information, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002836#pcbi.1002836.e067" target="_blank">Eq.(5)</a>. Bottom: The Information versus pattern age calculated from the top and middle graph. The total information stored, equal to the area under the curve, is clearly larger when using small updates (thick curve) than when using large updates.</p

Effect of coding density and inhibition on performance.

<p>A. The synaptic information capacity versus the coding density for soft-bound (solid line) and hard-bound (dashed line) plasticity, when taking 0s and 1s as inputs. When coding density is 1/2, the capacity is approximately half of what it is when using as inputs. However, low coding density improves the synaptic information capacity and in the limit of very sparse codes (utmost left in the graph) the capacity reaches that of <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002836#pcbi-1002836-g003" target="_blank">Figure 3A</a>. B. Effect of feed-forward inhibition on capacity under soft-bound plasticity. Using excitatory synapses () without inhibition, capacity is strongly reduced (‘No inhibition’). Adding feed-forward inhibition maximizes information capacity (‘Feedforward inhibition’). Equivalently, high capacity is achieved when the plasticity rules are defined such as to allow for negative weights (‘Unrestricted’). C. Possible circuit to implement feed-forward inhibition. D+E. Effects of coding density and inhibition on the information (panel A+B) on the information are mirrored by the effects on the memory lifetime.</p