Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

A new framework for cortico-striatal plasticity: behavioural theory meets In vitro data at the reinforcement-action interface

Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem—action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface

Planning activity for internally generated reward goals in monkey amygdala neurons

neuron

Amazon River carbon dioxide outgassing fuelled by wetlands

River systems connect the terrestrial biosphere, the atmosphere and the ocean in the global carbon cycle(1). A recent estimate suggests that up to 3 petagrams of carbon per year could be emitted as carbon dioxide (CO2) from global inland waters, offsetting the carbon uptake by terrestrial ecosystems(2). It is generally assumed that inland waters emit carbon that has been previously fixed upstream by land plant photosynthesis, then transferred to soils, and subsequently transported downstream in run-off. But at the scale of entire drainage basins, the lateral carbon fluxes carried by small rivers upstream do not account for all of the CO2 emitted from inundated areas downstream(3,4). Three-quarters of the world's flooded land consists of temporary wetlands(5), but the contribution of these productive ecosystems(6) to the inland water carbon budget has been largely overlooked. Here we show that wetlands pump large amounts of atmospheric CO2 into river waters in the floodplains of the central Amazon. Flooded forests and floating vegetation export large amounts of carbon to river waters and the dissolved CO2 can be transported dozens to hundreds of kilometres downstream before being emitted. We estimate that Amazonian wetlands export half of their gross primary production to river waters as dissolved CO2 and organic carbon, compared with only a few per cent of gross primary production exported in upland (not flooded) ecosystems(1,7). Moreover, we suggest that wetland carbon export is potentially large enough to account for at least the 0.21 petagrams of carbon emitted per year as CO2 from the central Amazon River and its floodplains(8). Global carbon budgets should explicitly address temporary or vegetated flooded areas, because these ecosystems combine high aerial primary production with large, fast carbon export, potentially supporting a substantial fraction of CO2 evasion from inland waters

The Gothenburg H70 Birth cohort study 2014-16 : design, methods and study population.

To improve health care for older persons, we need to learn more about ageing, e.g. identify protective factors and early markers for diseases. The Gothenburg H70 Birth Cohort Studies (the H70 studies) are multidisciplinary epidemiological studies examining representative birth cohorts of older populations in Gothenburg, Sweden. So far, six birth cohorts of 70-year-olds have been examined over time, and examinations have been virtually identical between studies. This paper describes the study procedures for the baseline examination of the Birth cohort 1944, conducted in 2014-16. In this study, all men and women born 1944 on specific dates, and registered as residents in Gothenburg, were eligible for participation (n = 1839). A total of 1203 (response rate 72.2%; 559 men and 644 women; mean age 70.5 years) agreed to participate in the study. The study comprised sampling of blood and cerebrospinal fluid, psychiatric, cognitive, and physical health examinations, examinations of genetics and family history, use of medications, social factors, functional ability and disability, physical fitness and activity, body composition, lung function, audiological and ophthalmological examinations, diet, brain imaging, as well as a close informant interview, and qualitative studies. As in previous examinations, data collection serves as a basis for future longitudinal follow-up examinations. The research gained from the H70 studies has clinical relevance in relation to prevention, early diagnosis, clinical course, experience of illness, understanding pathogenesis and prognosis. Results will increase our understanding of ageing and inform service development, which may lead to enhanced quality of care for older persons