Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

During sleep and awake rest, the hippocampus replays sequences of place cells that have been activated during prior experiences. These have been interpreted as a memory consolidation process, but recent results suggest a possible interpretation in terms of reinforcement learning. The Dyna reinforcement learning algorithms use off-line replays to improve learning. Under limited replay budget, a prioritized sweeping approach, which requires a model of the transitions to the predecessors, can be used to improve performance. We investigate whether such algorithms can explain the experimentally observed replays. We propose a neural network version of prioritized sweeping Q-learning, for which we developed a growing multiple expert algorithm, able to cope with multiple predecessors. The resulting architecture is able to improve the learning of simulated agents confronted to a navigation task. We predict that, in animals, learning the world model should occur during rest periods, and that the corresponding replays should be shuffled.Comment: Living Machines 2018 (Paris, France

Why are Prices Sticky? Evidence from Business Survey Data

This paper offers new insights on the price setting behaviour of German retail firms using a novel dataset that consists of a large panel of monthly business surveys from 1991-2006. The firm-level data allows matching changes in firms' prices to several other firm-characteristics. Moreover, information on price expectations allow analyzing the determinants of price updating. Using univariate and bivariate ordered probit specifications, empirical menu cost models are estimated relating the probability of price adjustment and price updating, respectively, to both time- and state- dependent variables. First, results suggest an important role for state-dependence; changes in the macroeconomic and institutional environment as well as firm-specific factors are significantly related to the timing of price adjustment. These findings imply that price setting models should endogenize the timing of price adjustment in order to generate realistic predictions concerning the transmission of monetary policy. Second, an analysis of price expectations yields similar results providing evidence in favour of state-dependent sticky plan models. Third, intermediate input cost changes are among the most important determinants of price adjustment suggesting that pricing models should explicitly incorporate price setting at different production stages. However, the results show that adjustment to input cost changes takes time indicating "additional stickiness" at the last stage of processing

Coincident Activity of Converging Pathways Enables Simultaneous Long-Term Potentiation and Long-Term Depression in Hippocampal CA1 Network In Vivo

Memory is believed to depend on activity-dependent changes in the strength of synapses, e.g. long-term potentiation (LTP) and long-term depression (LTD), which can be determined by the sequence of coincident pre- and postsynaptic activity, respectively. It remains unclear, however, whether and how coincident activity of converging efferent pathways can enable LTP and LTD in the pathways simultaneously. Here, we report that, in pentobarbital-anesthetized rats, stimulation (600 pulses, 5 Hz) to Schaffer preceding to commissural pathway within a 40-ms timing window induced similar magnitudes of LTP in both pathways onto synapses of CA1 neurons, with varied LTP magnitudes after reversal of the stimulation sequence. In contrast, in urethane-anesthetized or freely-moving rats, the stimulation to Schaffer preceding to commissural pathway induced Schaffer LTP and commissural LTD simultaneously within a 40-ms timing window, without affecting synaptic efficacy in the reversed stimulation sequence. Coincident activity of Schaffer pathways confirmed the above findings under pentobarbital and urethane anesthesia. Thus, coincident activity of converging afferent pathways tends to switch the pathways to be LTP only or LTP/LTD depending on the activity states of the hippocampus. This network rule strengthens the view that activity-dependent synaptic plasticity may well contribute to memory process of the hippocampal network with flexibility or stability from one state to another

A Study of B0 -> J/psi K(*)0 pi+ pi- Decays with the Collider Detector at Fermilab

We report a study of the decays B0 -> J/psi K(*)0 pi+ pi-, which involve the creation of a u u-bar or d d-bar quark pair in addition to a b-bar -> c-bar(c s-bar) decay. The data sample consists of 110 1/pb of p p-bar collisions at sqrt{s} = 1.8 TeV collected by the CDF detector at the Fermilab Tevatron collider during 1992-1995. We measure the branching ratios to be BR(B0 -> J/psi K*0 pi+ pi-) = (8.0 +- 2.2 +- 1.5) * 10^{-4} and BR(B0 -> J/psi K0 pi+ pi-) = (1.1 +- 0.4 +- 0.2) * 10^{-3}. Contributions to these decays are seen from psi(2S) K(*)0, J/psi K0 rho0, J/psi K*+ pi-, and J/psi K1(1270)

Direct Heme Transfer Reactions in the Group A Streptococcus Heme Acquisition Pathway

The heme acquisition machinery in Group A Streptococcus (GAS) consists of the surface proteins Shr and Shp and ATP-binding cassette transporter HtsABC. Shp cannot directly acquire heme from methemoglobin (metHb) but directly transfers its heme to HtsA. It has not been previously determined whether Shr directly relays heme from metHb to Shp. Thus, the complete pathway for heme acquisition from metHb by the GAS heme acquisition machinery has remained unclear. In this study, the metHb-to-Shr and Shr-to-Shp heme transfer reactions were characterized by spectroscopy, kinetics and protein-protein interaction analyses. Heme is efficiently transferred from the β and α subunits of metHb to Shr with rates that are 7 and 60 times greater than those of the passive heme release from metHb, indicating that Shr directly acquires heme from metHb. The rapid heme transfer from Shr to Shp involves an initial heme donor/acceptor complex and a spectrally and kinetically detectable transfer intermediate, implying that heme is directly channeled from Shr to Shp. The present results show that Shr speeds up heme transfer from metHb to Shp, whereas Shp speeds up heme transfer from Shr to HtsA. Furthermore, the findings demonstrate that Shr can interact with metHb and Shp but not HtsA. Taken together with our published results on the Shp/HtsA reaction, these findings establish a model of the heme acquisition pathway in GAS in which Shr directly extracts heme from metHb and Shp relays it from Shr to HtsA

Recall Performance Improvement in a Bio-Inspired Model of the Mammalian Hippocampus

Mammalian hippocampus is involved in short-term formation of declarative memories. We employed a bio-inspired neural model of hippocampal CA1 region consisting of a zoo of excitatory and inhibitory cells. Cells’ firing was timed to a theta oscillation paced by two distinct neuronal populations exhibiting highly regular bursting activity, one tightly coupled to the trough and the other to the peak of theta. To systematically evaluate the model’s recall performance against number of stored patterns, overlaps and ‘active cells per pattern’, its cells were driven by a non-specific excitatory input to their dendrites. This excitatory input to model excitatory cells provided context and timing information for retrieval of previously stored memory patterns. Inhibition to excitatory cells’ dendrites acted as a non-specific global threshold machine that removed spurious activity during recall. Out of the three models tested, ‘model 1’ recall quality was excellent across all conditions. ‘Model 2’ recall was the worst. The number of ‘active cells per pattern’ had a massive effect on network recall quality regardless of how many patterns were stored in it. As ‘active cells per pattern’ decreased, network’s memory capacity increased, interference effects between stored patterns decreased, and recall quality improved. Key finding was that increased firing rate of an inhibitory cell inhibiting a network of excitatory cells has a better success at removing spurious activity at the network level and improving recall quality than increasing the synaptic strength of the same inhibitory cell inhibiting the same network of excitatory cells, while keeping its firing rate fixed

Frequency-specific hippocampal-prefrontal interactions during associative learning

Much of our knowledge of the world depends on learning associations (for example, face-name), for which the hippocampus (HPC) and prefrontal cortex (PFC) are critical. HPC-PFC interactions have rarely been studied in monkeys, whose cognitive and mnemonic abilities are akin to those of humans. We found functional differences and frequency-specific interactions between HPC and PFC of monkeys learning object pair associations, an animal model of human explicit memory. PFC spiking activity reflected learning in parallel with behavioral performance, whereas HPC neurons reflected feedback about whether trial-and-error guesses were correct or incorrect. Theta-band HPC-PFC synchrony was stronger after errors, was driven primarily by PFC to HPC directional influences and decreased with learning. In contrast, alpha/beta-band synchrony was stronger after correct trials, was driven more by HPC and increased with learning. Rapid object associative learning may occur in PFC, whereas HPC may guide neocortical plasticity by signaling success or failure via oscillatory synchrony in different frequency bands.National Institute of Mental Health (U.S.) (Conte Center Grant P50-MH094263-03)National Institute of Mental Health (U.S.) (Fellowship F32-MH081507)Picower Foundatio

Diffractive Dijet Production at sqrt(s)=630 and 1800 GeV at the Fermilab Tevatron

We report a measurement of the diffractive structure function $F_{jj}^D$ of the antiproton obtained from a study of dijet events produced in association with a leading antiproton in $\bar pp$ collisions at $\sqrt s=630$ GeV at the Fermilab Tevatron. The ratio of $F_{jj}^D$ at $\sqrt s=630$ GeV to $F_{jj}^D$ obtained from a similar measurement at $\sqrt s=1800$ GeV is compared with expectations from QCD factorization and with theoretical predictions. We also report a measurement of the $\xi$ ($x$-Pomeron) and $\beta$ ($x$ of parton in Pomeron) dependence of $F_{jj}^D$ at $\sqrt s=1800$ GeV. In the region $0.035<\xi<0.095$, $|t|<1$ GeV$^2$ and $\beta<0.5$, $F_{jj}^D(\beta,\xi)$ is found to be of the form $\beta^{-1.0\pm 0.1} \xi^{-0.9\pm 0.1}$, which obeys $\beta$-$\xi$ factorization.Comment: LaTeX, 9 pages, Submitted to Phys. Rev. Letter

Measurement of the branching fraction and CP content for the decay B(0) -> D(*+)D(*-)

This is the pre-print version of the Article. The official published version can be accessed from the links below. Copyright @ 2002 APS.We report a measurement of the branching fraction of the decay B0→D*+D*- and of the CP-odd component of its final state using the BABAR detector. With data corresponding to an integrated luminosity of 20.4  fb-1 collected at the Υ(4S) resonance during 1999–2000, we have reconstructed 38 candidate signal events in the mode B0→D*+D*- with an estimated background of 6.2±0.5 events. From these events, we determine the branching fraction to be B(B0→D*+D*-)=[8.3±1.6(stat)±1.2(syst)]×10-4. The measured CP-odd fraction of the final state is 0.22±0.18(stat)±0.03(syst).This work is supported by DOE and NSF (USA), NSERC (Canada), IHEP (China), CEA and CNRS-IN2P3 (France), BMBF (Germany), INFN (Italy), NFR (Norway), MIST (Russia), and PPARC (United Kingdom). Individuals have received support from the A.P. Sloan Foundation, Research Corporation, and Alexander von Humboldt Foundation