Spatio-Temporal Credit Assignment in Neuronal Population Learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain

Postnatal development of A-type and Kv1- and Kv2-mediated potassium channel currents in neocortical pyramidal neurons

Potassium channels regulate numerous aspects of neuronal excitability, and several voltage-gated K+ channel subunits have been identified in pyramidal neurons of rat neocortex. Previous studies have either considered the development of outward current as a whole or divided currents into transient, A-type and persistent, delayed rectifier components but did not differentiate between current components defined by α-subunit type. To facilitate comparisons of studies reporting K+ currents from animals of different ages and to understand the functional roles of specific current components, we characterized the postnatal development of identified Kv channel-mediated currents in pyramidal neurons from layers II/III from rat somatosensory cortex. Both the persistent/slowly inactivating and transient components of the total K+ current increased in density with postnatal age. We used specific pharmacological agents to test the relative contributions of putative Kv1- and Kv2-mediated currents (100 nM α-dendrotoxin and 600 nM stromatoxin, respectively). A combination of voltage protocol, pharmacology, and curve fitting was used to isolate the rapidly inactivating A-type current. We found that the density of all identified current components increased with postnatal age, approaching a plateau at 3–5 wk. We found no significant changes in the relative proportions or kinetics of any component between postnatal weeks 1 and 5, except that the activation time constant for A-type current was longer at 1 wk. The putative Kv2-mediated component was the largest at all ages. Immunocytochemistry indicated that protein expression for Kv4.2, Kv4.3, Kv1.4, and Kv2.1 increased between 1 wk and 4–5 wk of age