Combined model-free and model-sensitive reinforcement learning in non-human primates

Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, modelsensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials. We trained rhesus monkeys to perform a two-stage decision task designed to elicit and discriminate the use of MF and MS methods. A descriptive analysis of choice behaviour revealed directly that the structure of the task (of MS importance) and the reward history (of MF and MS importance) significantly influenced both choice and response vigour. A detailed, trial-by-trial computational analysis confirmed that choices were made according to a combination of strategies, with a dominant influence of a particular form of model sensitivity that persisted over weeks of testing. The residuals from this model necessitated development of a new combined RL model which incorporates a particular credit assignment weighting procedure. Finally, response vigor exhibited a subtly different collection ofMFand MS influences. These results provide new illumination onto RL behavioural processes in non-human primates

Comparison of stimulus-evoked cerebral hemodynamics in the awake mouse and under a novel anesthetic regime

Neural activity is closely followed by a localised change in cerebral blood flow, a process termed neurovascular coupling. These hemodynamic changes form the basis of contrast in functional magnetic resonance imaging (fMRI) and are used as a correlate for neural activity. Anesthesia is widely employed in animal fMRI and neurovascular studies, however anesthetics are known to profoundly affect neural and vascular physiology, particularly in mice. Therefore, we investigated the efficacy of a novel ‘modular’ anesthesia that combined injectable (fentanyl-fluanisone/midazolam) and volatile (isoflurane) anesthetics in mice. To characterize sensory-evoked cortical hemodynamic responses, we used optical imaging spectroscopy to produce functional maps of changes in tissue oxygenation and blood volume in response to mechanical whisker stimulation. Following fine-tuning of the anesthetic regime, stimulation elicited large and robust hemodynamic responses in the somatosensory cortex, characterized by fast arterial activation, increases in total and oxygenated hemoglobin, and decreases in deoxygenated hemoglobin. Overall, the magnitude and speed of evoked hemodynamic responses under anesthesia resembled those in the awake state, indicating that the novel anesthetic combination significantly minimizes the impact of anesthesia. Our findings have broad implications for both neurovascular research and longitudinal fMRI studies that increasingly require the use of genetically engineered mice

The impact of habitat quality inside protected areas on distribution of the Dominican Republic’s last endemic non-volant land mammals

The Hispaniolan solenodon, Solenodon paradoxus, and Hispaniolan hutia, Plagiodontia aedium, are the Dominican Republic’s only surviving endemic non-volant land mammals, and are high priorities for conservation. The country has an extensive protected area (PA) network designed to maintain habitats and benefit biodiversity, but which faces significant anthropogenic threats likely to detrimentally impact both species. We examined how differences in habitats, forest structure, topography, and human activity influence presence of solenodons and hutias across the Dominican Republic. Systematic surveys of seven PAs were undertaken to record indirect signs, with presence-absence data analyzed using a multi-model inference approach incorporating ecological variables from both field and GIS data. Solenodons were detected relatively frequently, whereas detections of hutias were uncommon. Lower elevations, increased surrounding tree cover, canopy closure, and reduced levels of low vegetation are all associated with increased probability of detecting solenodons, whereas agriculture and mangrove represent poor-quality habitat. Increased canopy closure, tree basal area (indicating older-growth forest), and increased rock substrate (providing more den sites) are associated with increased probability of detecting hutias. Our findings indicated that human activities within PAs are likely to negatively affect both species, and conservation activities should focus on preventing encroachment and conversion of forest to agriculture to maintain high-quality forest habitats. El solenodonte de la Hispaniola, Solenodon paradoxus, y la hutia de la Hispaniola, Plagiodontia aedium, son los únicos mamíferos endémicos terrestres no voladores que sobreviven en la República Dominicana, su conservación es de alta prioridad. El país tiene una extensa red de áreas protegidas (AP) diseñada para mantener hábitats y beneficiar la biodiversidad, pero se enfrenta a amenazas antropogénicas. Sin embargo, no existen datos cuantitativos para evaluar las presiones antropogénicas que amenazan a los solenodontes y las hutias. Examinamos cómo las diferencias en los hábitats, la estructura del bosque, la topografía y la actividad humana influyen la presencia de solenodontes y hutias en toda la República Dominicana. Se realizaron encuestas sistemáticas de siete AP para registrar los signos indirectos de ambas especies, los datos de presencia/ausencia fueron analizados mediante inferencia multimodelo que incorpora variables ecológicas de los datos de campo y Sistema de Información Geográfica. Los Solenodontes se detectaron relativamente frecuentemente, mientras que las detecciones de hutias fueron menos comunes. Las elevaciones más bajas, el aumento de la cubierta arbórea circundante, el cierre del dosel y los niveles reducidos de vegetación baja se asocian con una mayor probabilidad de detectar solenodones. Mientras que la agricultura y los manglares representan un hábitat de mala calidad para el solenodonte. Aumento del cierre del dosel, área basal del árbol (que indica un bosque más antiguo) y un sustrato con mayor proporcion de roca (que proporciona más sitios para madrigueras) se asocian con una mayor probabilidad de detectar hutias. Nuestros hallazgos indican que las actividades humanas dentro de las AP pueden afectar negativamente a ambas especies. Las actividades de conservación deberían enfocarse en mantener hábitats forestales de alta calidad por medio de prevenir la invasión y la conversión de los bosques a agricultura

Solar ultraviolet radiation exposure, and incidence of childhood acute lymphocytic leukaemia and non-Hodgkin lymphoma in a US population-based dataset

Background: Acute lymphocytic leukaemia (ALL) and non-Hodgkin lymphoma (NHL) are among the commonest types of childhood cancer. Some previous studies suggested that elevated ultraviolet radiation (UVR) exposures increase ALL risk; many more indicate NHL risk is reduced. Methods: We assessed age&lt;20 ALL/NHL incidence in Surveillance, Epidemiology and End Results data using AVGLO-derived UVR irradiance/cumulative radiant exposure measures, using quasi-likelihood models accounting for underdispersion, adjusted for age, sex, racial/ethnic group and other county-level socioeconomic variables. Results: There were 30,349 cases of ALL and 8062 of NHL, with significant increasing trends of ALL with UVR irradiance (relative risk (RR) = 1.200/mW/cm2 (95% CI 1.060, 1.359, p = 0.0040)), but significant decreasing trends for NHL (RR = 0.646/mW/cm2 (95% CI 0.512, 0.816, p = 0.0002)). There was a borderline-significant increasing trend of ALL with UVR cumulative radiant exposure (RR = 1.444/MJ/cm2 (95% CI 0.949, 2.197, p = 0.0865)), and significant decreasing trends for NHL (RR = 0.284/MJ/cm2 (95% CI 0.166, 0.485, p &lt; 0.0001)). ALL and NHL trend RR is substantially increased among those aged 0–3. All-age trend RRs are most extreme (increasing for ALL, decreasing for NHL) for Hispanics for both UVR measures. Conclusions:Our more novel finding, of excess UVR-related ALL risk, is consistent with some previous studies, but is not clear-cut, and in need of replication.</p

Neural computations underlying action-based decision making in the human brain

Action-based decision making involves choices between different physical actions to obtain rewards. To make such decisions the brain needs to assign a value to each action and then compare them to make a choice. Using fMRI in human subjects, we found evidence for action-value signals in supplementary motor cortex. Separate brain regions, most prominently ventromedial prefrontal cortex, were involved in encoding the expected value of the action that was ultimately taken. These findings differentiate two main forms of value signals in the human brain: those relating to the value of each available action, likely reflecting signals that are a precursor of choice, and those corresponding to the expected value of the action that is subsequently chosen, and therefore reflecting the consequence of the decision process. Furthermore, we also found signals in the dorsomedial frontal cortex that resemble the output of a decision comparator, which implicates this region in the computation of the decision itself

Solar ultraviolet radiation exposure, and incidence of childhood acute lymphocytic leukaemia and non-Hodgkin lymphoma in a US population-based dataset

Background: Acute lymphocytic leukaemia (ALL) and non-Hodgkin lymphoma (NHL) are among the commonest types of childhood cancer. Some previous studies suggested that elevated ultraviolet radiation (UVR) exposures increase ALL risk; many more indicate NHL risk is reduced. Methods: We assessed age<20 ALL/NHL incidence in Surveillance, Epidemiology and End Results data using AVGLO-derived UVR irradiance/cumulative radiant exposure measures, using quasi-likelihood models accounting for underdispersion, adjusted for age, sex, racial/ethnic group and other county-level socioeconomic variables. Results: There were 30,349 cases of ALL and 8062 of NHL, with significant increasing trends of ALL with UVR irradiance (relative risk (RR) = 1.200/mW/cm2 (95% CI 1.060, 1.359, p = 0.0040)), but significant decreasing trends for NHL (RR = 0.646/mW/cm2 (95% CI 0.512, 0.816, p = 0.0002)). There was a borderline-significant increasing trend of ALL with UVR cumulative radiant exposure (RR = 1.444/MJ/cm2 (95% CI 0.949, 2.197, p = 0.0865)), and significant decreasing trends for NHL (RR = 0.284/MJ/cm2 (95% CI 0.166, 0.485, p  Conclusions: Our more novel finding, of excess UVR-related ALL risk, is consistent with some previous studies, but is not clear-cut, and in need of replication

Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making

Considerable evidence has emerged to implicate ventromedial prefrontal cortex in encoding expectations of future reward during value-based decision making. However, the nature of the learned associations upon which such representations depend is much less clear. Here, we aimed to determine whether expected reward representations in this region could be driven by action–outcome associations, rather than being dependent on the associative value assigned to particular discriminative stimuli. Subjects were scanned with functional magnetic resonance imaging while performing 2 variants of a simple reward-related decision task. In one version, subjects made choices between 2 different physical motor responses in the absence of discriminative stimuli, whereas in the other version, subjects chose between 2 different stimuli that were randomly assigned to different responses on a trial-by-trial basis. Using an extension of a reinforcement learning algorithm, we found activity in ventromedial prefrontal cortex tracked expected future reward during the action-based task as well as during the stimulus-based task, indicating that value representations in this region can be driven by action–outcome associations. These findings suggest that ventromedial prefrontal cortex may play a role in encoding the value of chosen actions irrespective of whether those actions denote physical motor responses or more abstract decision options

Seizure epicenter depth and translaminar field potential synchrony underlie complex variations in tissue oxygenation during ictal initiation

Whether functional hyperemia during epileptic activity is adequate to meet the heightened metabolic demand of such events is controversial. Whereas some studies have demonstrated hyperoxia during ictal onsets, other work has reported transient hypoxic episodes that are spatially dependent on local surface microvasculature. Crucially, how laminar differences in ictal evolution can affect subsequent cerebrovascular responses has not been thus far investigated, and is likely significant in view of possible laminar-dependent neurovascular mechanisms and angioarchitecture. We addressed this open question using a novel multi-modal methodology enabling concurrent measurement of cortical tissue oxygenation, blood flow and hemoglobin concentration, alongside laminar recordings of neural activity, in a urethane anesthetized rat model of recurrent seizures induced by 4-aminopyridine. We reveal there to be a close relationship between seizure epicenter depth, translaminar LFP synchrony and tissue oxygenation during the early stages of recurrent seizures, whereby deep layer seizures are associated with decreased cross laminar synchrony and prolonged periods of hypoxia, and middle layer seizures are accompanied by increased cross-laminar synchrony and hyperoxia. Through comparison with functional activation by somatosensory stimulation and graded hypercapnia, we show that these seizure-related cerebrovascular responses occur in the presence of conserved neural-hemodynamic and blood flow-volume coupling. Our data provide new insights into the laminar dependency of seizure-related neurovascular responses, which may reconcile inconsistent observations of seizure-related hypoxia in the literature, and highlight a potential layer-dependent vulnerability that may contribute to the harmful effects of clinical recurrent seizures. The relevance of our findings to perfusion-related functional neuroimaging techniques in epilepsy are also discussed

Direct effects of climate change on productivity of European aquaculture

Aquaculture managers and industry must take into account the impact of climate change on production and environmental quality to ensure that sector growth is sustainable over the coming decades, a key requirement for food security. The potential effects of climate change on aquaculture range from changes to production capacity in existing cultivation areas to changes in the areas themselves, which may become unsuitable for particular species, but also suitable for new species. The prediction of where and how such changes may occur is challenging, not least because the cultivated species may themselves exhibit plasticity, which makes it difficult to forecast the degree to which different locations and culture types may be affected. This work presents a modelling approach used to predict the potential effects of climate change on aquaculture, considering six key finfish and shellfish species of economic importance in Europe: Atlantic salmon (Salmo salar), gilthead seabream (Sparus aurata), sea bass (Dicentrarchus labrax), Pacific oyster (Crassostrea gigas), blue mussel (Mytilus edulis) and Mediterranean mussel (Mytilus galloprovincialis). The focus is on effects on physiology, growth performance and environmental footprint, and the resultant economic impact at the farm scale. Climate projections for present-day conditions; mid-century (2040–2060) and end-of-century (2080–2100) were extracted from regionally downscaled global climate models and used to force bioenergetic models. For each of those time periods, two different carbon concentration scenarios were considered: a moderate situation (IPCC RCP 4.5) and an extreme situation (IPCC RCP 8.5). Projected temperature changes will have variable effects on growth depending on the species and geographic region. From the case studies analysed, gilthead bream farmed in sea cages in the western Mediterranean was the most vulnerable, whereas offshore-suspended mussel culture in SW Portugal was least affected. Most of the marine finfish simulated were projected to have decreased feeding efficiency in both mid-century and end-of-century climate scenarios. Bivalve shellfish showed a decreasing trend with respect to most productivity parameters as climate change progresses, under both emission scenarios. As a general trend across species and regions, economic uncertainty is expected to increase under all future projections

A novel method for classifying cortical state to identify the accompanying changes in cerebral hemodynamics

Background: Many brain imaging techniques interpret the haemodynamic response as an indirect indicator of underlying neural activity. However, a challenge when interpreting this blood based signal is how changes in brain state may affect both baseline and stimulus evoked haemodynamics. New method: We developed an Automatic Brain State Classifier (ABSC), validated on data from anaesthetised rodents. It uses vectorised information obtained from the windowed spectral frequency power of the Local Field Potential. Current state is then classified by comparing this vectorised information against that calculated from state specific training datasets. Results: The ABSC identified two user defined brain states (synchronised and desynchronised), with high accuracy (~90%). Baseline haemodynamics were found to be significantly different in the two identified states. During state defined periods of elevated baseline haemodynamics we found significant decreases in evoked haemodynamic responses to somatosensory stimuli. Comparison to existing methods: State classification - The ABSC (~90%) demonstrated greater accuracy than clustering (~66%) or 'power threshold' (~64%) methods of comparison.Haemodynamic averaging - Our novel approach of selectively averaging stimulus evoked haemodynamic trials by brain state yields higher quality data than creating a single average from all trials. Conclusions: The ABSC can account for some of the commonly observed trial-to-trial variability in haemodynamic responses which arises from changes in cortical state. This variability might otherwise be incorrectly attributed to alternative interpretations. A greater understanding of the effects of cortical state on haemodynamic changes could be used to inform techniques such as general linear modelling (GLM), commonly used in fMRI