Partial Adaptation of Obtained and Observed Value Signals Preserves Information about Gains and Losses

Given that the range of rewarding and punishing outcomes of actions is large but neural coding capacity is limited, efficient processing of outcomes by the brain is necessary. One mechanism to increase efficiency is to rescale neural output to the range of outcomes expected in the current context, and process only experienced deviations from this expectation. However, this mechanism comes at the cost of not being able to discriminate between unexpectedly low losses when times are bad versus unexpectedly high gains when times are good. Thus, too much adaptation would result in disregarding information about the nature and absolute magnitude of outcomes, preventing learning about the longer-term value structure of the environment. Here we investigate the degree of adaptation in outcome coding brain regions in humans, for directly experienced outcomes and observed outcomes. We scanned participants while they performed a social learning task in gain and loss blocks. Multivariate pattern analysis showed two distinct networks of brain regions adapt to the most likely outcomes within a block. Frontostriatal areas adapted to directly experienced outcomes, whereas lateral frontal and temporoparietal regions adapted to observed social outcomes. Critically, in both cases, adaptation was incomplete and information about whether the outcomes arose in a gain block or a loss block was retained. Univariate analysis confirmed incomplete adaptive coding in these regions but also detected nonadapting outcome signals. Thus, although neural areas rescale their responses to outcomes for efficient coding, they adapt incompletely and keep track of the longer-term incentives available in the environment

Partial Adaptation of Obtained and Observed Value Signals Preserves Information about Gains and Losses.

UNLABELLED: Given that the range of rewarding and punishing outcomes of actions is large but neural coding capacity is limited, efficient processing of outcomes by the brain is necessary. One mechanism to increase efficiency is to rescale neural output to the range of outcomes expected in the current context, and process only experienced deviations from this expectation. However, this mechanism comes at the cost of not being able to discriminate between unexpectedly low losses when times are bad versus unexpectedly high gains when times are good. Thus, too much adaptation would result in disregarding information about the nature and absolute magnitude of outcomes, preventing learning about the longer-term value structure of the environment. Here we investigate the degree of adaptation in outcome coding brain regions in humans, for directly experienced outcomes and observed outcomes. We scanned participants while they performed a social learning task in gain and loss blocks. Multivariate pattern analysis showed two distinct networks of brain regions adapt to the most likely outcomes within a block. Frontostriatal areas adapted to directly experienced outcomes, whereas lateral frontal and temporoparietal regions adapted to observed social outcomes. Critically, in both cases, adaptation was incomplete and information about whether the outcomes arose in a gain block or a loss block was retained. Univariate analysis confirmed incomplete adaptive coding in these regions but also detected nonadapting outcome signals. Thus, although neural areas rescale their responses to outcomes for efficient coding, they adapt incompletely and keep track of the longer-term incentives available in the environment. SIGNIFICANCE STATEMENT: Optimal value-based choice requires that the brain precisely and efficiently represents positive and negative outcomes. One way to increase efficiency is to adapt responding to the most likely outcomes in a given context. However, too strong adaptation would result in loss of precise representation (e.g., when the avoidance of a loss in a loss-context is coded the same as receipt of a gain in a gain-context). We investigated an intermediate form of adaptation that is efficient while maintaining information about received gains and avoided losses. We found that frontostriatal areas adapted to directly experienced outcomes, whereas lateral frontal and temporoparietal regions adapted to observed social outcomes. Importantly, adaptation was intermediate, in line with influential models of reference dependence in behavioral economics

Pseudorehearsal in value function approximation

Catastrophic forgetting is of special importance in reinforcement learning, as the data distribution is generally non-stationary over time. We study and compare several pseudorehearsal approaches for Q-learning with function approximation in a pole balancing task. We have found that pseudorehearsal seems to assist learning even in such very simple problems, given proper initialization of the rehearsal parameters

Flexible attention allocation to visual and auditory working memory tasks: manipulating reward induces a trade-off

Prominent roles for general attention resources are posited in many models of working memory, but the manner in which these can be allocated differs between models or is not sufficiently specified. We varied the payoffs for correct responses in two temporally-overlapping recognition tasks, a visual array comparison task and a tone sequence comparison task. In the critical conditions, an increase in reward for one task corresponded to a decrease in reward for the concurrent task, but memory load remained constant. Our results show patterns of interference consistent with a trade-off between the tasks, suggesting that a shared resource can be flexibly divided, rather than only fully allotted to either of the tasks. Our findings support a role for a domain-general resource in models of working memory, and furthermore suggest that this resource is flexibly divisible

Visual Working Memory Load-Related Changes in Neural Activity and Functional Connectivity

BACKGROUND: Visual working memory (VWM) helps us store visual information to prepare for subsequent behavior. The neuronal mechanisms for sustaining coherent visual information and the mechanisms for limited VWM capacity have remained uncharacterized. Although numerous studies have utilized behavioral accuracy, neural activity, and connectivity to explore the mechanism of VWM retention, little is known about the load-related changes in functional connectivity for hemi-field VWM retention. METHODOLOGY/PRINCIPAL FINDINGS: In this study, we recorded electroencephalography (EEG) from 14 normal young adults while they performed a bilateral visual field memory task. Subjects had more rapid and accurate responses to the left visual field (LVF) memory condition. The difference in mean amplitude between the ipsilateral and contralateral event-related potential (ERP) at parietal-occipital electrodes in retention interval period was obtained with six different memory loads. Functional connectivity between 128 scalp regions was measured by EEG phase synchronization in the theta- (4-8 Hz), alpha- (8-12 Hz), beta- (12-32 Hz), and gamma- (32-40 Hz) frequency bands. The resulting matrices were converted to graphs, and mean degree, clustering coefficient and shortest path length was computed as a function of memory load. The results showed that brain networks of theta-, alpha-, beta-, and gamma- frequency bands were load-dependent and visual-field dependent. The networks of theta- and alpha- bands phase synchrony were most predominant in retention period for right visual field (RVF) WM than for LVF WM. Furthermore, only for RVF memory condition, brain network density of theta-band during the retention interval were linked to the delay of behavior reaction time, and the topological property of alpha-band network was negative correlation with behavior accuracy. CONCLUSIONS/SIGNIFICANCE: We suggest that the differences in theta- and alpha- bands between LVF and RVF conditions in functional connectivity and topological properties during retention period may result in the decline of behavioral performance in RVF task

Population Distribution, Settlement Patterns and Accessibility across Africa in 2010

The spatial distribution of populations and settlements across a country and their interconnectivity and accessibility from urban areas are important for delivering healthcare, distributing resources and economic development. However, existing spatially explicit population data across Africa are generally based on outdated, low resolution input demographic data, and provide insufficient detail to quantify rural settlement patterns and, thus, accurately measure population concentration and accessibility. Here we outline approaches to developing a new high resolution population distribution dataset for Africa and analyse rural accessibility to population centers. Contemporary population count data were combined with detailed satellite-derived settlement extents to map population distributions across Africa at a finer spatial resolution than ever before. Substantial heterogeneity in settlement patterns, population concentration and spatial accessibility to major population centres is exhibited across the continent. In Africa, 90% of the population is concentrated in less than 21% of the land surface and the average per-person travel time to settlements of more than 50,000 inhabitants is around 3.5 hours, with Central and East Africa displaying the longest average travel times. The analyses highlight large inequities in access, the isolation of many rural populations and the challenges that exist between countries and regions in providing access to services. The datasets presented are freely available as part of the AfriPop project, providing an evidence base for guiding strategic decisions

A statistical approach to estimating the strength of cell-cell interactions under the differential adhesion hypothesis

International audienceBACKGROUND: The Differential Adhesion Hypothesis (DAH) is a theory of the organization of cells within a tissue which has been validated by several biological experiments and tested against several alternative computational models. RESULTS: In this study, a statistical approach was developed for the estimation of the strength of adhesion, incorporating earlier discrete lattice models into a continuous marked point process framework. This framework allows to describe an ergodic Markov Chain Monte Carlo algorithm that can simulate the model and reproduce empirical biological patterns. The estimation procedure, based on a pseudo-likelihood approximation, is validated with simulations, and a brief application to medulloblastoma stained by beta-catenin markers is given. CONCLUSION: Our model includes the strength of cell-cell adhesion as a statistical parameter. The estimation procedure for this parameter is consistent with experimental data and would be useful for high-throughput cancer studies

Gene Expression Changes in GABAA Receptors and Cognition Following Chronic Ketamine Administration in Mice

Ketamine is a well-known anesthetic agent and a drug of abuse. Despite its widespread use and abuse, little is known about its long-term effects on the central nervous system. The present study was designed to evaluate the effect of long-term (1- and 3-month) ketamine administration on learning and memory and associated gene expression levels in the brain. The Morris water maze was used to assess spatial memory and gene expression changes were assayed using Affymetrix Genechips; a focus on the expression of GABAA receptors that mediate a tonic inhibition in the brain, was confirmed by quantitative real-time PCR and western blot. Compared with saline controls, there was a decline in learning and memory performance in the ketamine-treated mice. Genechip results showed that 110 genes were up-regulated and 136 genes were down-regulated. An ontology analysis revealed the most significant effects of ketamine were on GABAA receptors. In particular, there was a significant up-regulation of both mRNA and protein levels of the alpha 5 subunit (Gabra5) of the GABAA receptors in the prefrontal cortex. In conclusion, chronic exposure to ketamine impairs working memory in mice, which may be explained at least partly by up-regulation of Gabra5 subunits in the prefrontal cortex

Vulnerability of the British swine industry to classical swine fever

Classical swine fever (CSF) is a notifiable, highly contagious viral disease of swine which results in severe welfare and economic consequences in affected countries. To improve preparedness, it is critical to have some understanding of how CSF would spread should it be introduced. Based on the data recorded during the 2000 epidemic of CSF in Great Britain (GB), a spatially explicit, premises-based model was developed to explore the risk of CSF spread in GB. We found that large outbreaks of CSF would be rare and generated from a limited number of areas in GB. Despite the consistently low vulnerability of the British swine industry to large CSF outbreaks, we identified concerns with respect to the role played by the non-commercial sector of the industry. The model further revealed how various epidemiological features may influence the spread of CSF in GB, highlighting the importance of between-farm biosecurity in preventing widespread dissemination of the virus. Knowledge of factors affecting the risk of spread are key components for surveillance planning and resource allocation, and this work provides a valuable stepping stone in guiding policy on CSF surveillance and control in GB