Scaling prediction errors to reward variability benefits error-driven learning in humans.

Effective error-driven learning requires individuals to adapt learning to environmental reward variability. The adaptive mechanism may involve decays in learning rate across subsequent trials, as shown previously, and rescaling of reward prediction errors. The present study investigated the influence of prediction error scaling and, in particular, the consequences for learning performance. Participants explicitly predicted reward magnitudes that were drawn from different probability distributions with specific standard deviations. By fitting the data with reinforcement learning models, we found scaling of prediction errors, in addition to the learning rate decay shown previously. Importantly, the prediction error scaling was closely related to learning performance, defined as accuracy in predicting the mean of reward distributions, across individual participants. In addition, participants who scaled prediction errors relative to standard deviation also presented with more similar performance for different standard deviations, indicating that increases in standard deviation did not substantially decrease "adapters'" accuracy in predicting the means of reward distributions. However, exaggerated scaling beyond the standard deviation resulted in impaired performance. Thus efficient adaptation makes learning more robust to changing variability.This work was supported by the Wellcome Trust and the Niels Stensen Foundation.This is the final version of the article. It first appeared from the American Physiological Society via http://dx.doi.org/10.1152/jn.00483.201

Adaptive Prediction Error Coding in the Human Midbrain and Striatum Facilitates Behavioral Adaptation and Learning Efficiency.

Effective error-driven learning benefits from scaling of prediction errors to reward variability. Such behavioral adaptation may be facilitated by neurons coding prediction errors relative to the standard deviation (SD) of reward distributions. To investigate this hypothesis, we required participants to predict the magnitude of upcoming reward drawn from distributions with different SDs. After each prediction, participants received a reward, yielding trial-by-trial prediction errors. In line with the notion of adaptive coding, BOLD response slopes in the Substantia Nigra/Ventral Tegmental Area (SN/VTA) and ventral striatum were steeper for prediction errors occurring in distributions with smaller SDs. SN/VTA adaptation was not instantaneous but developed across trials. Adaptive prediction error coding was paralleled by behavioral adaptation, as reflected by SD-dependent changes in learning rate. Crucially, increased SN/VTA and ventral striatal adaptation was related to improved task performance. These results suggest that adaptive coding facilitates behavioral adaptation and supports efficient learning.This study was supported by the Wellcome Trust (W.S., P.C.F.), Bernard Wolfe Health Neuroscience Fund (P.C.F.) and the Niels Stensen Foundation (K.M.J.D.). We thank William Stauffer, Armin Lak and Joost Haarsma for useful discussions.This is the final version of the article. It first appeared from Cell Press via http://dx.doi.org/10.1016/j.neuron.2016.04.01

Dopamine, Prediction Error and Beyond

A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference

The Measurement of Language Lateralization with Functional Transcranial Doppler and Functional MRI: A Critical Evaluation

Cerebral language lateralization can be assessed in several ways. In healthy subjects, functional MRI (fMRI) during performance of a language task has evolved to be the most frequently applied method. Functional transcranial Doppler (fTCD) may provide a valid alternative, but has been used rarely. Both techniques have their own strengths and weaknesses and as a result may be applied in different fields of research. Until now, only one relatively small study (n = 13) investigated the correlation between lateralization indices (LIs) measured by fTCD and fMRI and showed a remarkably high correlation. To further evaluate the correlation between LIs measured with fTCD and fMRI, we compared LIs of 22 healthy subjects (12 left- and 10 right-handed) using the same word generation paradigm for the fTCD as for the fMRI experiment. LIs measured with fTCD were highly but imperfectly correlated with LIs measured with fMRI (Spearman's rho = 0.75, p < 0.001). The imperfectness of the correlation can partially be explained by methodological restrictions of fMRI as well as fTCD. Our results suggest that fTCD can be a valid alternative for fMRI to measure lateralization, particularly when costs or mobility are important factors in the study design

Applying the Higher Education Academy framework for partnership in learning and teaching in higher education to online partnership learning communities: A case study and an extended model

As internet access and use increase exponentially, pedagogical practice becomes increasingly embedded in online platforms. We report on an online initiative of engaged student learning, the peer-led, staff-assisted e-helpdesk for research methods and statistics, which we evaluated and redeveloped using the lens and guiding principles of the framework for partnership in learning and teaching of the Higher Education Academy (HEA). The aim of the redevelopment was to steer the initiative towards a more integrative and sustainable implementation, as manifest in the applied construct of an online partnership learning community. Our evolving experience of the e-helpdesk highlighted the central role of the facilitator in engineering and maintaining social presence in the online community. We propose an extended model for building an online partnership learning community, whereby partnership encapsulates all the essential elements of student and staff partnership as outlined in the HEA framework, but is also critically defined by similar parameters of partnership between users and facilitators. In this model, the facilitator’s role becomes more involved in instructional teaching as disciplinary expertise increases, but descending levels of disciplinary expertise can foster ascending levels of independent learning and shared discovery for both users and facilitators.&nbsp; &nbsp

Natural Language Processing markers in first episode psychosis and people at clinical high-risk.

Funder: MQ: Transforming Mental Health; Grant(s): MQF17_24Recent work has suggested that disorganised speech might be a powerful predictor of later psychotic illness in clinical high risk subjects. To that end, several automated measures to quantify disorganisation of transcribed speech have been proposed. However, it remains unclear which measures are most strongly associated with psychosis, how different measures are related to each other and what the best strategies are to collect speech data from participants. Here, we assessed whether twelve automated Natural Language Processing markers could differentiate transcribed speech excerpts from subjects at clinical high risk for psychosis, first episode psychosis patients and healthy control subjects (total N = 54). In-line with previous work, several measures showed significant differences between groups, including semantic coherence, speech graph connectivity and a measure of whether speech was on-topic, the latter of which outperformed the related measure of tangentiality. Most NLP measures examined were only weakly related to each other, suggesting they provide complementary information. Finally, we compared the ability of transcribed speech generated using different tasks to differentiate the groups. Speech generated from picture descriptions of the Thematic Apperception Test and a story re-telling task outperformed free speech, suggesting that choice of speech generation method may be an important consideration. Overall, quantitative speech markers represent a promising direction for future clinical applications.SEM was supported by the Accelerate Programme for Scientific Discovery, funded by Schmidt Futures, a Fellowship from The Alan Turing Institute, London, and a Henslow Fellowship at Lucy Cavendish College, University of Cambridge, funded by the Cambridge Philosophical Society. PEV is supported by a fellowship from MQ: Transforming Mental Health (MQF17_24). This work was supported by The Alan Turing Institute under the EPSRC grant EP/N510129/1, the NIHR Cambridge Biomedical Research Centre (BRC-1215-20014), the UK Medical Research Council (MRC) and the National Institute for Health Research (NIHR) Mental Health Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London

Natural Language Processing markers in first episode psychosis and people at clinical high-risk.

Funder: MQ: Transforming Mental Health; Grant(s): MQF17_24Recent work has suggested that disorganised speech might be a powerful predictor of later psychotic illness in clinical high risk subjects. To that end, several automated measures to quantify disorganisation of transcribed speech have been proposed. However, it remains unclear which measures are most strongly associated with psychosis, how different measures are related to each other and what the best strategies are to collect speech data from participants. Here, we assessed whether twelve automated Natural Language Processing markers could differentiate transcribed speech excerpts from subjects at clinical high risk for psychosis, first episode psychosis patients and healthy control subjects (total N = 54). In-line with previous work, several measures showed significant differences between groups, including semantic coherence, speech graph connectivity and a measure of whether speech was on-topic, the latter of which outperformed the related measure of tangentiality. Most NLP measures examined were only weakly related to each other, suggesting they provide complementary information. Finally, we compared the ability of transcribed speech generated using different tasks to differentiate the groups. Speech generated from picture descriptions of the Thematic Apperception Test and a story re-telling task outperformed free speech, suggesting that choice of speech generation method may be an important consideration. Overall, quantitative speech markers represent a promising direction for future clinical applications.SEM was supported by the Accelerate Programme for Scientific Discovery, funded by Schmidt Futures, a Fellowship from The Alan Turing Institute, London, and a Henslow Fellowship at Lucy Cavendish College, University of Cambridge, funded by the Cambridge Philosophical Society. PEV is supported by a fellowship from MQ: Transforming Mental Health (MQF17_24). This work was supported by The Alan Turing Institute under the EPSRC grant EP/N510129/1, the NIHR Cambridge Biomedical Research Centre (BRC-1215-20014), the UK Medical Research Council (MRC) and the National Institute for Health Research (NIHR) Mental Health Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London

Applying the Higher Education Academy Framework for Partnership in Learning and Teaching in Higher Education to Online Partnership Learning Communities: A Case Study and an Extended Model

As internet access and use increase exponentially, pedagogical practice becomes increasingly embedded in online platforms. We report on an online initiative of engaged student learning, the peer-led, staff-assisted e-helpdesk for research methods and statistics, which we evaluated and redeveloped using the lens and guiding principles of the framework for partnership in learning and teaching of the Higher Education Academy (HEA). The aim of the redevelopment was to steer the initiative towards a more integrative and sustainable implementation, as manifest in the applied construct of an online partnership learning community. Our evolving experience of the e-helpdesk highlighted the central role of the facilitator in engineering and maintaining social presence in the online community. We propose an extended model for building an online partnership learning community, whereby partnership encapsulates all the essential elements of student and staff partnership as outlined in the HEA framework, but is also critically defined by similar parameters of partnership between users and facilitators. In this model, the facilitator’s role becomes more involved in instructional teaching as disciplinary expertise increases, but descending levels of disciplinary expertise can foster ascending levels of independent learning and shared discovery for both users and facilitators

Auditory Hallucinations and the Brain’s Resting-State Networks: Findings and Methodological Observations

In recent years, there has been increasing interest in the potential for alterations to the brain’s resting-state networks (RSNs) to explain various kinds of psychopathology. RSNs provide an intriguing new explanatory framework for hallucinations, which can occur in different modalities and population groups, but which remain poorly understood. This collaboration from the International Consortium on Hallucination Research (ICHR) reports on the evidence linking resting-state alterations to auditory hallucinations (AH) and provides a critical appraisal of the methodological approaches used in this area. In the report, we describe findings from resting connectivity fMRI in AH (in schizophrenia and nonclinical individuals) and compare them with findings from neurophysiological research, structural MRI, and research on visual hallucinations (VH). In AH, various studies show resting connectivity differences in left-hemisphere auditory and language regions, as well as atypical interaction of the default mode network and RSNs linked to cognitive control and salience. As the latter are also evident in studies of VH, this points to a domain-general mechanism for hallucinations alongside modality-specific changes to RSNs in different sensory regions. However, we also observed high methodological heterogeneity in the current literature, affecting the ability to make clear comparisons between studies. To address this, we provide some methodological recommendations and options for future research on the resting state and hallucinations

Oscillatory Cortical Network Involved in Auditory Verbal Hallucinations in Schizophrenia

Auditory verbal hallucinations (AVH), a prominent symptom of schizophrenia, are often highly distressing for patients. Better understanding of the pathogenesis of hallucinations could increase therapeutic options. Magnetoencephalography (MEG) provides direct measures of neuronal activity and has an excellent temporal resolution, offering a unique opportunity to study AVH pathophysiology.Twelve patients (10 paranoid schizophrenia, 2 psychosis not otherwise specified) indicated the presence of AVH by button-press while lying in a MEG scanner. As a control condition, patients performed a self-paced button-press task. AVH-state and non-AVH state were contrasted in a region-of-interest (ROI) approach. In addition, the two seconds before AVH onset were contrasted with the two seconds after AVH onset to elucidate a possible triggering mechanism.AVH correlated with a decrease in beta-band power in the left temporal cortex. A decrease in alpha-band power was observed in the right inferior frontal gyrus. AVH onset was related to a decrease in theta-band power in the right hippocampus.These results suggest that AVH are triggered by a short aberration in the theta band in a memory-related structure, followed by activity in language areas accompanying the experience of AVH itself