Dopamine enhances model-based over model-free choice behavior

Decision making is often considered to arise out of contributions from a model-free habitual system and a model-based goal-directed system. Here, we investigated the effect of a dopamine manipulation on the degree to which either system contributes to instrumental behavior in a two-stage Markov decision task, which has been shown to discriminate model-free from model-based control. We found increased dopamine levels promote model-based over model-free choice

Adaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum

Behavioral and neural evidence reveal a prospective goal-directed decision process that relies on mental simulation of the environment, and a retrospective habitual process that caches returns previously garnered from available choices. Artificial systems combine the two by simulating the environment up to some depth and then exploiting habitual values as proxies for consequences that may arise in the further future. Using a three-step task, we provide evidence that human subjects use such a normative plan-until-habit strategy, implying a spectrum of approaches that interpolates between habitual and goal-directed responding. We found that increasing time pressure led to shallower goal-directed planning, suggesting that a speed-accuracy tradeoff controls the depth of planning with deeper search leading to more accurate evaluation, at the cost of slower decision-making. We conclude that subjects integrate habit-based cached values directly into goal-directed evaluations in a normative manner

Modelled mortality benefits of multi-cancer early detection screening in England

Background Screening programmes utilising blood-based multi-cancer early detection (MCED) tests, which can detect a shared cancer signal from any site in the body with a single, low false-positive rate, could reduce cancer burden through early diagnosis. Methods A natural history (‘interception’) model of cancer was previously used to characterise potential benefits of MCED screening (based on published performance of an MCED test). We built upon this using a two-population survival model to account for an increased risk of death from cfDNA-detectable cancers relative to cfDNA-non-detectable cancers. We developed another model allowing some cancers to metastasise directly from stage I, bypassing intermediate tumour stages. We used incidence and survival-by-stage data from the National Cancer Registration and Analysis Service in England to estimate longer-term benefits to a cohort screened between ages 50–79 years. Results Estimated late-stage and mortality reductions were robust to a range of assumptions. With the least favourable dwell (sojourn) time and cfDNA status hazard ratio assumptions, we estimated, among 100,000 screened individuals, 67 (17%) fewer cancer deaths per year corresponding to 2029 fewer deaths in those screened between ages 50–79 years. Conclusion Realising the potential benefits of MCED tests could substantially reduce late-stage cancer diagnoses and mortality

Impact of screening participation on modelled mortality benefits of a multi-cancer early detection test by socioeconomic group in England.

BACKGROUND: Cancer burden is higher and cancer screening participation is lower among individuals living in more socioeconomically deprived areas of England, contributing to worse health outcomes and shorter life expectancy. Owing to higher multi-cancer early detection (MCED) test sensitivity for poor-prognosis cancers and greater cancer burden in groups experiencing greater deprivation, MCED screening programmes may have greater relative benefits in these groups. We modelled potential differential benefits of MCED screening between deprivation groups in England at different levels of screening participation. METHODS: We applied the interception multi-cancer screening model to cancer incidence and survival data made available by the National Cancer Registration and Analysis Service in England to estimate reductions in late-stage diagnoses and cancer mortality from an MCED screening programme by deprivation group across 24 cancer types. We assessed the impact of varying the proportion of people who participated in annual screening in each deprivation group on these estimates. RESULTS: The modelled benefits of an MCED screening programme were substantial: reductions in late-stage diagnoses were 160 and 274 per 100 000 persons in the least and most deprived groups, respectively. Reductions in cancer mortality were 60 and 99 per 100 000 persons in the least and most deprived groups, respectively. Benefits were greatest in the most deprived group at every participation level and were attenuated with lower screening participation. CONCLUSIONS: For the greatest possible population benefit and to decrease health inequalities, an MCED implementation strategy should focus on enhancing equitable, informed participation, enabling equal participation across all socioeconomic deprivation groups. TRIAL REGISTRATION NUMBER: NCT05611632

Modelled mortality benefits of multi-cancer early detection screening in England

Background: Screening programmes utilising blood-based multi-cancer early detection (MCED) tests, which can detect a shared cancer signal from any site in the body with a single, low false-positive rate, could reduce cancer burden through early diagnosis. Methods: A natural history (‘interception’) model of cancer was previously used to characterise potential benefits of MCED screening (based on published performance of an MCED test). We built upon this using a two-population survival model to account for an increased risk of death from cfDNA-detectable cancers relative to cfDNA-non-detectable cancers. We developed another model allowing some cancers to metastasise directly from stage I, bypassing intermediate tumour stages. We used incidence and survival-by-stage data from the National Cancer Registration and Analysis Service in England to estimate longer-term benefits to a cohort screened between ages 50–79 years. Results: Estimated late-stage and mortality reductions were robust to a range of assumptions. With the least favourable dwell (sojourn) time and cfDNA status hazard ratio assumptions, we estimated, among 100,000 screened individuals, 67 (17%) fewer cancer deaths per year corresponding to 2029 fewer deaths in those screened between ages 50–79 years. Conclusion: Realising the potential benefits of MCED tests could substantially reduce late-stage cancer diagnoses and mortality

Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

Valence-dependent influence of serotonin depletion on model-based choice strategy.

Human decision-making arises from both reflective and reflexive mechanisms, which underpin goal-directed and habitual behavioural control. Computationally, these two systems of behavioural control have been described by different learning algorithms, model-based and model-free learning, respectively. Here, we investigated the effect of diminished serotonin (5-hydroxytryptamine) neurotransmission using dietary tryptophan depletion (TD) in healthy volunteers on the performance of a two-stage decision-making task, which allows discrimination between model-free and model-based behavioural strategies. A novel version of the task was used, which not only examined choice balance for monetary reward but also for punishment (monetary loss). TD impaired goal-directed (model-based) behaviour in the reward condition, but promoted it under punishment. This effect on appetitive and aversive goal-directed behaviour is likely mediated by alteration of the average reward representation produced by TD, which is consistent with previous studies. Overall, the major implication of this study is that serotonin differentially affects goal-directed learning as a function of affective valence. These findings are relevant for a further understanding of psychiatric disorders associated with breakdown of goal-directed behavioural control such as obsessive-compulsive disorders or addictions.This research was funded by Wellcome Trust Grants awarded to VV (Intermediate WT Fellowship) and Programme Grant (089589/Z/09/Z) awarded to TWR, BJE, ACR, JWD and BJS. It was conducted at the Behavioural and Clinical Neuroscience Institute, which is supported by a joint award from the Medical Research Council and Wellcome Trust (G00001354). YW was supported by the Fyssen Foundation. SP is supported by Marie Curie Intra-European Fellowship (FP7-People-2012-IEF).This is the final version of the article. It first appeared from NPG via http://dx.doi.org/10.1038/mp.2015.4

Role of Dopamine D2 Receptors in Human Reinforcement Learning

Influential neurocomputational models emphasize dopamine (DA) as an electrophysiological and neurochemical correlate of reinforcement learning. However, evidence of a specific causal role of DA receptors in learning has been less forthcoming, especially in humans. Here we combine, in a between-subjects design, administration of a high dose of the selective DA D2/3-receptor antagonist sulpiride with genetic analysis of the DA D2 receptor in a behavioral study of reinforcement learning in a sample of 78 healthy male volunteers. In contrast to predictions of prevailing models emphasizing DA's pivotal role in learning via prediction errors, we found that sulpiride did not disrupt learning, but rather induced profound impairments in choice performance. The disruption was selective for stimuli indicating reward, while loss avoidance performance was unaffected. Effects were driven by volunteers with higher serum levels of the drug, and in those with genetically-determined lower density of striatal DA D2 receptors. This is the clearest demonstration to date for a causal modulatory role of the DA D2 receptor in choice performance that might be distinct from learning. Our findings challenge current reward prediction error models of reinforcement learning, and suggest that classical animal models emphasizing a role of postsynaptic DA D2 receptors in motivational aspects of reinforcement learning may apply to humans as well.Neuropsychopharmacology accepted article peview online, 09 April 2014; doi:10.1038/npp.2014.84

Harnessing citizen science through mobile phone technology to screen for immunohistochemical biomarkers in bladder cancer

Background: Immunohistochemistry (IHC) is often used in personalisation of cancer treatments. Analysis of large data sets to uncover predictive biomarkers by specialists can be enormously time-consuming. Here we investigated crowdsourcing as a means of reliably analysing immunostained cancer samples to discover biomarkers predictive of cancer survival. Methods: We crowdsourced the analysis of bladder cancer TMA core samples through the smartphone app ‘Reverse the Odds’. Scores from members of the public were pooled and compared to a gold standard set scored by appropriate specialists. We also used crowdsourced scores to assess associations with disease-specific survival. Results: Data were collected over 721 days, with 4,744,339 classifications performed. The average time per classification was approximately 15 s, with approximately 20,000 h total non-gaming time contributed. The correlation between crowdsourced and expert H-scores (staining intensity × proportion) varied from 0.65 to 0.92 across the markers tested, with six of 10 correlation coefficients at least 0.80. At least two markers (MRE11 and CK20) were significantly associated with survival in patients with bladder cancer, and a further three markers showed results warranting expert follow-up. Conclusions: Crowdsourcing through a smartphone app has the potential to accurately screen IHC data and greatly increase the speed of biomarker discovery