238 research outputs found

    Combined model-free and model-sensitive reinforcement learning in non-human primates

    Get PDF
    Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, modelsensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials. We trained rhesus monkeys to perform a two-stage decision task designed to elicit and discriminate the use of MF and MS methods. A descriptive analysis of choice behaviour revealed directly that the structure of the task (of MS importance) and the reward history (of MF and MS importance) significantly influenced both choice and response vigour. A detailed, trial-by-trial computational analysis confirmed that choices were made according to a combination of strategies, with a dominant influence of a particular form of model sensitivity that persisted over weeks of testing. The residuals from this model necessitated development of a new combined RL model which incorporates a particular credit assignment weighting procedure. Finally, response vigor exhibited a subtly different collection ofMFand MS influences. These results provide new illumination onto RL behavioural processes in non-human primates

    Comparison of stimulus-evoked cerebral hemodynamics in the awake mouse and under a novel anesthetic regime

    Get PDF
    Neural activity is closely followed by a localised change in cerebral blood flow, a process termed neurovascular coupling. These hemodynamic changes form the basis of contrast in functional magnetic resonance imaging (fMRI) and are used as a correlate for neural activity. Anesthesia is widely employed in animal fMRI and neurovascular studies, however anesthetics are known to profoundly affect neural and vascular physiology, particularly in mice. Therefore, we investigated the efficacy of a novel ‘modular’ anesthesia that combined injectable (fentanyl-fluanisone/midazolam) and volatile (isoflurane) anesthetics in mice. To characterize sensory-evoked cortical hemodynamic responses, we used optical imaging spectroscopy to produce functional maps of changes in tissue oxygenation and blood volume in response to mechanical whisker stimulation. Following fine-tuning of the anesthetic regime, stimulation elicited large and robust hemodynamic responses in the somatosensory cortex, characterized by fast arterial activation, increases in total and oxygenated hemoglobin, and decreases in deoxygenated hemoglobin. Overall, the magnitude and speed of evoked hemodynamic responses under anesthesia resembled those in the awake state, indicating that the novel anesthetic combination significantly minimizes the impact of anesthesia. Our findings have broad implications for both neurovascular research and longitudinal fMRI studies that increasingly require the use of genetically engineered mice

    Solar ultraviolet radiation exposure, and incidence of childhood acute lymphocytic leukaemia and non-Hodgkin lymphoma in a US population-based dataset

    Get PDF
    Background: Acute lymphocytic leukaemia (ALL) and non-Hodgkin lymphoma (NHL) are among the commonest types of childhood cancer. Some previous studies suggested that elevated ultraviolet radiation (UVR) exposures increase ALL risk; many more indicate NHL risk is reduced. Methods: We assessed age&lt;20 ALL/NHL incidence in Surveillance, Epidemiology and End Results data using AVGLO-derived UVR irradiance/cumulative radiant exposure measures, using quasi-likelihood models accounting for underdispersion, adjusted for age, sex, racial/ethnic group and other county-level socioeconomic variables. Results: There were 30,349 cases of ALL and 8062 of NHL, with significant increasing trends of ALL with UVR irradiance (relative risk (RR) = 1.200/mW/cm2 (95% CI 1.060, 1.359, p = 0.0040)), but significant decreasing trends for NHL (RR = 0.646/mW/cm2 (95% CI 0.512, 0.816, p = 0.0002)). There was a borderline-significant increasing trend of ALL with UVR cumulative radiant exposure (RR = 1.444/MJ/cm2 (95% CI 0.949, 2.197, p = 0.0865)), and significant decreasing trends for NHL (RR = 0.284/MJ/cm2 (95% CI 0.166, 0.485, p &lt; 0.0001)). ALL and NHL trend RR is substantially increased among those aged 0–3. All-age trend RRs are most extreme (increasing for ALL, decreasing for NHL) for Hispanics for both UVR measures. Conclusions:Our more novel finding, of excess UVR-related ALL risk, is consistent with some previous studies, but is not clear-cut, and in need of replication.</p

    Neural computations underlying action-based decision making in the human brain

    Get PDF
    Action-based decision making involves choices between different physical actions to obtain rewards. To make such decisions the brain needs to assign a value to each action and then compare them to make a choice. Using fMRI in human subjects, we found evidence for action-value signals in supplementary motor cortex. Separate brain regions, most prominently ventromedial prefrontal cortex, were involved in encoding the expected value of the action that was ultimately taken. These findings differentiate two main forms of value signals in the human brain: those relating to the value of each available action, likely reflecting signals that are a precursor of choice, and those corresponding to the expected value of the action that is subsequently chosen, and therefore reflecting the consequence of the decision process. Furthermore, we also found signals in the dorsomedial frontal cortex that resemble the output of a decision comparator, which implicates this region in the computation of the decision itself

    Solar ultraviolet radiation exposure, and incidence of childhood acute lymphocytic leukaemia and non-Hodgkin lymphoma in a US population-based dataset

    Get PDF
    Background: Acute lymphocytic leukaemia (ALL) and non-Hodgkin lymphoma (NHL) are among the commonest types of childhood cancer. Some previous studies suggested that elevated ultraviolet radiation (UVR) exposures increase ALL risk; many more indicate NHL risk is reduced. Methods: We assessed age<20 ALL/NHL incidence in Surveillance, Epidemiology and End Results data using AVGLO-derived UVR irradiance/cumulative radiant exposure measures, using quasi-likelihood models accounting for underdispersion, adjusted for age, sex, racial/ethnic group and other county-level socioeconomic variables. Results: There were 30,349 cases of ALL and 8062 of NHL, with significant increasing trends of ALL with UVR irradiance (relative risk (RR) = 1.200/mW/cm2 (95% CI 1.060, 1.359, p = 0.0040)), but significant decreasing trends for NHL (RR = 0.646/mW/cm2 (95% CI 0.512, 0.816, p = 0.0002)). There was a borderline-significant increasing trend of ALL with UVR cumulative radiant exposure (RR = 1.444/MJ/cm2 (95% CI 0.949, 2.197, p = 0.0865)), and significant decreasing trends for NHL (RR = 0.284/MJ/cm2 (95% CI 0.166, 0.485, p  Conclusions: Our more novel finding, of excess UVR-related ALL risk, is consistent with some previous studies, but is not clear-cut, and in need of replication

    Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making

    Get PDF
    Considerable evidence has emerged to implicate ventromedial prefrontal cortex in encoding expectations of future reward during value-based decision making. However, the nature of the learned associations upon which such representations depend is much less clear. Here, we aimed to determine whether expected reward representations in this region could be driven by action–outcome associations, rather than being dependent on the associative value assigned to particular discriminative stimuli. Subjects were scanned with functional magnetic resonance imaging while performing 2 variants of a simple reward-related decision task. In one version, subjects made choices between 2 different physical motor responses in the absence of discriminative stimuli, whereas in the other version, subjects chose between 2 different stimuli that were randomly assigned to different responses on a trial-by-trial basis. Using an extension of a reinforcement learning algorithm, we found activity in ventromedial prefrontal cortex tracked expected future reward during the action-based task as well as during the stimulus-based task, indicating that value representations in this region can be driven by action–outcome associations. These findings suggest that ventromedial prefrontal cortex may play a role in encoding the value of chosen actions irrespective of whether those actions denote physical motor responses or more abstract decision options

    Seizure epicenter depth and translaminar field potential synchrony underlie complex variations in tissue oxygenation during ictal initiation

    Get PDF
    Whether functional hyperemia during epileptic activity is adequate to meet the heightened metabolic demand of such events is controversial. Whereas some studies have demonstrated hyperoxia during ictal onsets, other work has reported transient hypoxic episodes that are spatially dependent on local surface microvasculature. Crucially, how laminar differences in ictal evolution can affect subsequent cerebrovascular responses has not been thus far investigated, and is likely significant in view of possible laminar-dependent neurovascular mechanisms and angioarchitecture. We addressed this open question using a novel multi-modal methodology enabling concurrent measurement of cortical tissue oxygenation, blood flow and hemoglobin concentration, alongside laminar recordings of neural activity, in a urethane anesthetized rat model of recurrent seizures induced by 4-aminopyridine. We reveal there to be a close relationship between seizure epicenter depth, translaminar LFP synchrony and tissue oxygenation during the early stages of recurrent seizures, whereby deep layer seizures are associated with decreased cross laminar synchrony and prolonged periods of hypoxia, and middle layer seizures are accompanied by increased cross-laminar synchrony and hyperoxia. Through comparison with functional activation by somatosensory stimulation and graded hypercapnia, we show that these seizure-related cerebrovascular responses occur in the presence of conserved neural-hemodynamic and blood flow-volume coupling. Our data provide new insights into the laminar dependency of seizure-related neurovascular responses, which may reconcile inconsistent observations of seizure-related hypoxia in the literature, and highlight a potential layer-dependent vulnerability that may contribute to the harmful effects of clinical recurrent seizures. The relevance of our findings to perfusion-related functional neuroimaging techniques in epilepsy are also discussed

    Direct effects of climate change on productivity of European aquaculture

    Get PDF
    Aquaculture managers and industry must take into account the impact of climate change on production and environmental quality to ensure that sector growth is sustainable over the coming decades, a key requirement for food security. The potential effects of climate change on aquaculture range from changes to production capacity in existing cultivation areas to changes in the areas themselves, which may become unsuitable for particular species, but also suitable for new species. The prediction of where and how such changes may occur is challenging, not least because the cultivated species may themselves exhibit plasticity, which makes it difficult to forecast the degree to which different locations and culture types may be affected. This work presents a modelling approach used to predict the potential effects of climate change on aquaculture, considering six key finfish and shellfish species of economic importance in Europe: Atlantic salmon (Salmo salar), gilthead seabream (Sparus aurata), sea bass (Dicentrarchus labrax), Pacific oyster (Crassostrea gigas), blue mussel (Mytilus edulis) and Mediterranean mussel (Mytilus galloprovincialis). The focus is on effects on physiology, growth performance and environmental footprint, and the resultant economic impact at the farm scale. Climate projections for present-day conditions; mid-century (2040–2060) and end-of-century (2080–2100) were extracted from regionally downscaled global climate models and used to force bioenergetic models. For each of those time periods, two different carbon concentration scenarios were considered: a moderate situation (IPCC RCP 4.5) and an extreme situation (IPCC RCP 8.5). Projected temperature changes will have variable effects on growth depending on the species and geographic region. From the case studies analysed, gilthead bream farmed in sea cages in the western Mediterranean was the most vulnerable, whereas offshore-suspended mussel culture in SW Portugal was least affected. Most of the marine finfish simulated were projected to have decreased feeding efficiency in both mid-century and end-of-century climate scenarios. Bivalve shellfish showed a decreasing trend with respect to most productivity parameters as climate change progresses, under both emission scenarios. As a general trend across species and regions, economic uncertainty is expected to increase under all future projections

    A novel method for classifying cortical state to identify the accompanying changes in cerebral hemodynamics

    Get PDF
    Background: Many brain imaging techniques interpret the haemodynamic response as an indirect indicator of underlying neural activity. However, a challenge when interpreting this blood based signal is how changes in brain state may affect both baseline and stimulus evoked haemodynamics. New method: We developed an Automatic Brain State Classifier (ABSC), validated on data from anaesthetised rodents. It uses vectorised information obtained from the windowed spectral frequency power of the Local Field Potential. Current state is then classified by comparing this vectorised information against that calculated from state specific training datasets. Results: The ABSC identified two user defined brain states (synchronised and desynchronised), with high accuracy (~90%). Baseline haemodynamics were found to be significantly different in the two identified states. During state defined periods of elevated baseline haemodynamics we found significant decreases in evoked haemodynamic responses to somatosensory stimuli. Comparison to existing methods: State classification - The ABSC (~90%) demonstrated greater accuracy than clustering (~66%) or 'power threshold' (~64%) methods of comparison.Haemodynamic averaging - Our novel approach of selectively averaging stimulus evoked haemodynamic trials by brain state yields higher quality data than creating a single average from all trials. Conclusions: The ABSC can account for some of the commonly observed trial-to-trial variability in haemodynamic responses which arises from changes in cortical state. This variability might otherwise be incorrectly attributed to alternative interpretations. A greater understanding of the effects of cortical state on haemodynamic changes could be used to inform techniques such as general linear modelling (GLM), commonly used in fMRI
    • …
    corecore