An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards

‘‘Beet-ing’’ the Mountain: A Review of the Physiological and Performance Effects of Dietary Nitrate Supplementation at Simulated and Terrestrial Altitude

Exposure to altitude results in multiple physiological consequences. These include, but are not limited to, a reduced maximal oxygen consumption, drop in arterial oxygen saturation, and increase in muscle metabolic perturbations at a fixed sub-maximal work rate. Exercise capacity during fixed work rate or incremental exercise and time-trial performance are also impaired at altitude relative to sea-level. Recently, dietary nitrate (NO3-) supplementation has attracted considerable interest as a nutritional aid during altitude exposure. In this review, we summarise and critically evaluate the physiological and performance effects of dietary NO3- supplementation during exposure to simulated and terrestrial altitude. Previous investigations at simulated altitude indicate that NO3- supplementation may reduce the oxygen cost of exercise, elevate arterial and tissue oxygen saturation, improve muscle metabolic function, and enhance exercise capacity/ performance. Conversely, current evidence suggests that NO3- supplementation does not augment the training response at simulated altitude. Few studies have evaluated the effects of NO3- at terrestrial altitude. Current evidence indicates potential improvements in endothelial function at terrestrial altitude following NO3- supplementation. No effects of NO3- supplementation have been observed on oxygen consumption or arterial oxygen saturation at terrestrial altitude, although further research is warranted. Limitations of the present body of literature are discussed, and directions for future research are provided

Centrality dependence of the pseudorapidity density distribution for charged particles in Pb\u2013Pb collisions at 1asNN = 2.76 TeV

We present the first wide-range measurement of the charged-particle pseudorapidity density distribution, for different centralities (the 0\u20135%, 5\u201310%, 10\u201320%, and 20\u201330% most central events) in Pb\u2013Pb collisions at 1asNN = 2.76 TeV at the LHC. The measurement is performed using the full coverage of the ALICE detectors, 125.0 < \u3b7 < 5.5, and employing a special analysis technique based on collisions arising from LHC \u2018satellite\u2019 bunches. We present the pseudorapidity density as a function of the number of participating nucleons as well as an extrapolation to the total number of produced charged particles (Nch = 17 165 \ub1 772 for the 0\u20135% most central collisions). From the measured dNch/d\u3b7 distribution we derive the rapidity density distribution, dNch/dy, under simple assumptions. The rapidity density distribution is found to be significantly wider than the predictions of the Landau model. We assess the validity of longitudinal scaling by comparing to lower energy results from RHIC. Finally the mechanisms of the underlying particle production are discussed based on a comparison with various theoretical models

Production of charged pions, kaons and protons at large transverse momenta in pp and Pb–Pb collisions at sNN=2.76\sqrt{s_{NN}}=2.76 TeV

Transverse momentum spectra of pi(+/-), K-+/- and p((p) over bar) up to p(T) = 20 GeV/c at mid-rapidity in pp, peripheral (60-80%) and central (0-5%) Pb-Pb collisions at v root s(NN) = 2.76 TeV have been measured using the ALICE detector at the Large Hadron Collider. The proton-to-pion and the kaon-to-pionratios both show a distinct peak at p(T) approximate to 3 GeV/c in central Pb-Pb collisions. Below the peak, p(T) 10 GeV/c particle ratios in pp and Pb-Pb collisions are in agreement and the nuclear modification factors for pi(+/-), K-+/- and p((p) over bar) indicate that, within the systematic and statistical uncertainties, the suppression is the same. This suggests that the chemical composition of leading particles from jets in the medium is similar to that of vacuum jets

Production of charged pions, kaons and protons at large transverse momenta in pp and Pb–Pb collisions at sNN=2.76\sqrt{s_{NN}}=2.76 TeV

Transverse momentum spectra of pi(+/-), K-+/- and p((p) over bar) up to p(T) = 20 GeV/c at mid-rapidity in pp, peripheral (60-80%) and central (0-5%) Pb-Pb collisions at v root s(NN) = 2.76 TeV have been measured using the ALICE detector at the Large Hadron Collider. The proton-to-pion and the kaon-to-pionratios both show a distinct peak at p(T) approximate to 3 GeV/c in central Pb-Pb collisions. Below the peak, p(T) 10 GeV/c particle ratios in pp and Pb-Pb collisions are in agreement and the nuclear modification factors for pi(+/-), K-+/- and p((p) over bar) indicate that, within the systematic and statistical uncertainties, the suppression is the same. This suggests that the chemical composition of leading particles from jets in the medium is similar to that of vacuum jets

Centrality, rapidity and transverse momentum dependence of J/\u3c8 suppression in Pb-Pb collisions at 1asNN= 2.76TeV

The inclusive J/.nuclear modification factor (R-AA) in Pb-Pb collisions at root(NN)-N-S = 2.76TeVhas been measured by ALICE as a function of centrality in the e+ e-decay channel at mid-rapidity (| y| < 0.8) and as a function of centrality, transverse momentum and rapidity in the + -decay channel at forward-rapidity (2.5 < y < 4). The J/.yields measured in Pb-Pb are suppressed compared to those in ppcollisions scaled by the number of binary collisions. The RAAintegrated over a centrality range corresponding to 90% of the inelastic Pb-Pb cross section is 0.72 - 0.06(stat.) - 0.10(syst.) at mid-rapidity and 0.58 - 0.01(stat.) - 0.09(syst.) at forward-rapidity. At low transverse momentum, significantly larger values of RAAare measured at forward-rapidity compared to measurements at lower energy. These features suggest that a contribution to the J/.yield originates from charm quark (re) combination in the deconfined partonic medium