Model-free and model-based reward prediction errors in EEG

Learning theorists posit two reinforcement learning systems: model-free and model-based. Model-based learning incorporates knowledge about structure and contingencies in the world to assign candidate actions with an expected value. Model-free learning is ignorant of the world’s structure; instead, actions hold a value based on prior reinforcement, with this value updated by expectancy violation in the form of a reward prediction error. Because they use such different learning mechanisms, it has been previously assumed that model-based and model-free learning are computationally dissociated in the brain. However, recent fMRI evidence suggests that the brain may compute reward prediction errors to both model-free and model-based estimates of value, signalling the possibility that these systems interact. Because of its poor temporal resolution, fMRI risks confounding reward prediction errors with other feedback-related neural activity. In the present study, EEG was used to show the presence of both model-based and model-free reward prediction errors and their place in a temporal sequence of events including state prediction errors and action value updates. This demonstration of model-based prediction errors questions a long-held assumption that model-free and model-based learning are dissociated in the brain

Genotoxic capacity of Cd/Se semiconductor quantum dots with differing surface chemistries.

Quantum dots (QD) have unique electronic and optical properties promoting biotechnological advances. However, our understanding of the toxicological structure-activity relationships remains limited. This study aimed to determine the biological impact of varying nanomaterial surface chemistry by assessing the interaction of QD with either a negative (carboxyl), neutral (hexadecylamine; HDA) or positive (amine) polymer coating with human lymphoblastoid TK6 cells. Following QD physico-chemical characterisation, cellular uptake was quantified by optical and electron microscopy. Cytotoxicity was evaluated and genotoxicity was characterised using the micronucleus assay (gross chromosomal damage) and the HPRT forward mutation assay (point mutagenicity). Cellular damage mechanisms were also explored, focusing on oxidative stress and mitochondrial damage. Cell uptake, cytotoxicity and genotoxicity were found to be dependent on QD surface chemistry. Carboxyl-QD demonstrated the smallest agglomerate size and greatest cellular uptake, which correlated with a dose dependent increase in cytotoxicity and genotoxicity. Amine-QD induced minimal cellular damage, while HDA-QD promoted substantial induction of cell death and genotoxicity. However, HDA-QD were not internalised by the cells and the damage they caused was most likely due to free cadmium release caused by QD dissolution. Oxidative stress and induced mitochondrial reactive oxygen species were only partially associated with cytotoxicity and genotoxicity induced by the QD, hence were not the only mechanisms of importance. Colloidal stability, nanoparticle (NP) surface chemistry, cellular uptake levels and the intrinsic characteristics of the NPs are therefore critical parameters impacting genotoxicity induced by QD

Feature- versus rule-based generalization in rats, pigeons and humans

Humans can spontaneously create rules that allow them to efficiently generalize what they have learned to novel situations. An enduring question is whether rule-based generalization is uniquely human or whether other animals can also abstract rules and apply them to novel situations. In recent years, there have been a number of high-profile claims that animals such as rats can learn rules. Most of those claims are quite weak because it is possible to demonstrate that simple associative systems (which do not learn rules) can account for the behavior in those tasks. Using a procedure that allows us to clearly distinguish feature-based from rule-based generalization (the Shanks-Darby procedure), we demonstrate that adult humans show rule-based generalization in this task, while generalization in rats and pigeons was based on featural overlap between stimuli. In brief, when learning that a stimulus made of two components ("AB") predicts a different outcome than its elements ("A" and "B"), people spontaneously abstract an opposites rule and apply it to new stimuli (e.g., knowing that "C" and "D" predict one outcome, they will predict that "CD" predicts the opposite outcome). Rats and pigeons show the reverse behavior-they generalize what they have learned, but on the basis of similarity (e.g., "CD" is similar to "C" and "D", so the same outcome is predicted for the compound stimulus as for the components). Genuinely rule-based behavior is observed in humans, but not in rats and pigeons, in the current procedure

Pre-testing effects are target-specific and are not driven by a generalised state of curiosity

Guessing an answer to an unfamiliar question prior to seeing the answer leads to better memory than studying alone (the pre-testing effect), which some theories attribute to increased curiosity. A similar effect occurs in general knowledge learning: people are more likely to recall information that they were initially curious to learn. Gruber and Ranganath [(2019). How curiosity enhances hippocampus-dependent memory: The prediction, appraisal, curiosity, and exploration (PACE) framework. Trends in Cognitive Sciences, 23(12), 1014–1025] argued that unanswered questions can cause a state of curiosity during which encoding is enhanced for the missing answer, but also for incidental information presented at the time. If pre-testing similarly induces curiosity, then it too should produce better memory for incidental information. We tested this idea in three experiments that varied the order, nature and timing of the incidental material presented within a pre-testing context. All three experiments demonstrated a reliable pre-testing effect for the targets, but no benefit for the incidental material presented before the target. This pattern suggests that the pre-testing effect is highly specific and is not consistent with a generalised state of curiosity

Cell type-dependent changes in CdSe/ZnS quantum dot uptake and toxic endpoints.

Toxicity of nanoparticles (NPs) is often correlated with the physicochemical characteristics of the materials. However, some discrepancies are noted in in-vitro studies on quantum dots (QDs) with similar physicochemical properties. This is partly related to variations in cell type. In this study, we show that epithelial (BEAS-2B), fibroblast (HFF-1), and lymphoblastoid (TK6) cells show different biological responses following exposure to QDs. These cells represented the 3 main portals of NP exposure: bronchial, skin, and circulatory. The uptake and toxicity of negatively and positively charged CdSe:ZnS QDs of the same core size but with different surface chemistries (carboxyl or amine polymer coatings) were investigated in full and reduced serum containing media following 1 and 3 cell cycles. Following thorough physicochemical characterization, cellular uptake, cytotoxicity, and gross chromosomal damage were measured. Cellular damage mechanisms in the form of reactive oxygen species and the expression of inflammatory cytokines IL-8 and TNF-α were assessed. QDs uptake and toxicity significantly varied in the different cell lines. BEAS-2B cells demonstrated the highest level of QDs uptake yet displayed a strong resilience with minimal genotoxicity following exposure to these NPs. In contrast, HFF-1 and TK6 cells were more susceptible to toxicity and genotoxicity, respectively, as a result of exposure to QDs. Thus, this study demonstrates that in addition to nanomaterial physicochemical characterization, a clear understanding of cell type-dependent variation in uptake coupled to the inherently different capacities of the cell types to cope with exposure to these exogenous materials are all required to predict genotoxicity

Negative mood reverses devaluation of goal-directed drug-seeking favouring an incentive learning account of drug dependence.

BACKGROUND: Two theories explain how negative mood primes smoking behaviour. The stimulus–response (S-R) account argues that in the negative mood state, smoking is experienced as more reinforcing, establishing a direct (automatic) association between the negative mood state and smoking behaviour. By contrast, the incentive learning account argues that in the negative mood state smoking is expected to be more reinforcing, which integrates with instrumental knowledge of the response required to produce that outcome. OBJECTIVES: One differential prediction is that whereas the incentive learning account anticipates that negative mood induction could augment a novel tobacco-seeking response in an extinction test, the S-R account could not explain this effect because the extinction test prevents S-R learning by omitting experience of the reinforcer. METHODS: To test this, overnight-deprived daily smokers (n = 44) acquired two instrumental responses for tobacco and chocolate points, respectively, before smoking to satiety. Half then received negative mood induction to raise the expected value of tobacco, opposing satiety, whilst the remainder received positive mood induction. Finally, a choice between tobacco and chocolate was measured in extinction to test whether negative mood could augment tobacco choice, opposing satiety, in the absence of direct experience of tobacco reinforcement. RESULTS: Negative mood induction not only abolished the devaluation of tobacco choice, but participants with a significant increase in negative mood increased their tobacco choice in extinction, despite satiety. CONCLUSIONS: These findings suggest that negative mood augments drug-seeking by raising the expected value of the drug through incentive learning, rather than through automatic S-R control

A dimensional summation account of polymorphous category learning

This is the author accepted manuscript. The final version is available from Springer via the DOI in this record.Data and code availaibility: The data and code for all analyses for all experiments are available at the OSF addresses given in each Results section. The stimuli are available at the same locations.Polymorphous concepts are hard to learn, and this is perhaps surprising because they, like many natural concepts, have an overall similarity structure. However, the dimensional summation hypothesis (Milton & Wills, 2004) predicts this difficulty. It also makes a number of other predictions about polymorphous concept formation, which are tested here. In Experiment 1 we confirm the theory’s prediction that polymorphous concept formation should be facilitated by deterministic pretraining on the constituent features of the stimulus. This facilitation is relative to an equivalent amount of training on the polymorphous concept itself. In Experiments 2–4, the dimensional summation account of this single feature pretraining effect is contrasted with some other accounts, including a more general strategic account (Experiment 2), seriality of training and stimulus decomposition accounts (Experiment 3), and the role of errors (Experiment 4). The dimensional summation hypothesis provides the best account of these data. In Experiment 5, a further prediction is confirmed — the single feature pretraining effect is eliminated by a concurrent counting task. The current experiments suggest the hypothesis that natural concepts might be acquired by the deliberate serial summation of evidence. This idea has testable implications for classroom learning.Biotechnology and Biological Sciences Research Council (BBSRC

Free classification of large sets of everyday objects is more thematic than taxonomic

Traditionally it has been thought that the overall organisation of categories in the brain is taxonomic. To examine this assumption, we had adults sort 140–150 diverse, familiar objects from different basic-level categories. Almost all the participants (80/81) sorted the objects more thematically than taxonomically. Sorting was only weakly modulated by taxonomic priming, and people still produced many thematically structured clusters when explicitly instructed to sort taxonomically. The first clusters that people produced were rated as having equal taxonomic and thematic structure. However, later clusters were rated as being increasingly thematically organised. A minority of items were consistently clustered taxonomically, but the overall dominance of thematically structured clusters suggests that people know more thematic than taxonomic relations among everyday objects. A final study showed that the semantic relations used to sort a given item in the initial studies predicted the proportion of thematic to taxonomic word associates generated to that item. However, unlike the results of the sorting task, most of these single word associates were related taxonomically. This latter difference between the results of large-scale, free sorting tasks versus single word association tasks suggests that thematic relations may be more numerous, but weaker, than taxonomic associations in our stored conceptual network. Novel statistical and numerical methods for objectively measuring sorting consistency were developed during the course of this investigation, and have been made publicly available

Characterizing Nanoparticles in Biological Matrices: Tipping Points in Agglomeration State and Cellular Delivery In Vitro.

Understanding the delivered cellular dose of nanoparticles is imperative in nanomedicine and nanosafety, yet is known to be extremely complex because of multiple interactions between nanoparticles, their environment, and the cells. Here, we use 3-D reconstruction of agglomerates preserved by cryogenic snapshot sampling and imaged by electron microscopy to quantify the "bioavailable dose" that is presented at the cell surface and formed by the process of individual nanoparticle sequestration into agglomerates in the exposure media. Critically, using 20 and 40 nm carboxylated polystyrene-latex and 16 and 85 nm silicon dioxide nanoparticles, we show that abrupt, dose-dependent "tipping points" in agglomeration state can arise, subsequently affecting cellular delivery and increasing toxicity. These changes are triggered by shifts in the ratio of the total nanoparticle surface area to biomolecule abundance, with the switch to a highly agglomerated state effectively changing the test article midassay, challenging the dose-response paradigm for nanosafety experiments. By characterizing nanoparticle numbers per agglomerate, we show these tipping points can lead to the formation of extreme agglomeration states whereby 90% of an administered dose is contained and delivered to the cells by just the top 2% of the largest agglomerates. We thus demonstrate precise definition, description, and comparison of the nanoparticle dose formed in different experimental environments and show that this description is critical to understanding cellular delivery and toxicity. We further empirically "stress-test" the commonly used dynamic light scattering approach, establishing its limitations to present an analysis strategy that significantly improves the usefulness of this popular nanoparticle characterization technique