The brain as a generative model: information-theoretic surprise in learning and action

Our environment is rich with statistical regularities, such as a sudden cold gust of wind indicating a potential change in weather. A combination of theoretical work and empirical evidence suggests that humans embed this information in an internal representation of the world. This generative model is used to perform probabilistic inference, which may be approximated through surprise minimization. This process rests on current beliefs enabling predictions, with expectation violation amounting to surprise. Through repeated interaction with the world, beliefs become more accurate and grow more certain over time. Perception and learning may be accounted for by minimizing surprise of current observations, while action is proposed to minimize expected surprise of future events. This framework thus shows promise as a common formulation for different brain functions. The work presented here adopts information-theoretic quantities of surprise to investigate both perceptual learning and action. We recorded electroencephalography (EEG) of participants in a somatosensory roving-stimulus paradigm and performed trial-by-trial modeling of cortical dynamics. Bayesian model selection suggests early processing in somatosensory cortices to encode confidence-corrected surprise and subsequently Bayesian surprise. This suggests the somatosensory system to signal surprise of observations and update a probabilistic model learning transition probabilities. We also extended this framework to include audition and vision in a multi-modal roving-stimulus study. Next, we studied action by investigating a sensitivity to expected Bayesian surprise. Interestingly, this quantity is also known as information gain and arises as an incentive to reduce uncertainty in the active inference framework, which can correspond to surprise minimization. In comparing active inference to a classical reinforcement learning model on the two-step decision-making task, we provided initial evidence for active inference to better account for human model-based behaviour. This appeared to relate to participants’ sensitivity to expected Bayesian surprise and contributed to explaining exploration behaviour not accounted for by the reinforcement learning model. Overall, our findings provide evidence for information-theoretic surprise as a model for perceptual learning signals while also guiding human action.Unsere Umwelt ist reich an statistischen Regelmäßigkeiten, wie z. B. ein plötzlicher kalter Windstoß, der einen möglichen Wetterumschwung ankündigt. Eine Kombination aus theoretischen Arbeiten und empirischen Erkenntnissen legt nahe, dass der Mensch diese Informationen in eine interne Darstellung der Welt einbettet. Dieses generative Modell wird verwendet, um probabilistische Inferenz durchzuführen, die durch Minimierung von Überraschungen angenähert werden kann. Der Prozess beruht auf aktuellen Annahmen, die Vorhersagen ermöglichen, wobei eine Verletzung der Erwartungen einer Überraschung gleichkommt. Durch wiederholte Interaktion mit der Welt nehmen die Annahmen mit der Zeit an Genauigkeit und Gewissheit zu. Es wird angenommen, dass Wahrnehmung und Lernen durch die Minimierung von Überraschungen bei aktuellen Beobachtungen erklärt werden können, während Handlung erwartete Überraschungen für zukünftige Beobachtungen minimiert. Dieser Rahmen ist daher als gemeinsame Bezeichnung für verschiedene Gehirnfunktionen vielversprechend. In der hier vorgestellten Arbeit werden informationstheoretische Größen der Überraschung verwendet, um sowohl Wahrnehmungslernen als auch Handeln zu untersuchen. Wir haben die Elektroenzephalographie (EEG) von Teilnehmern in einem somatosensorischen Paradigma aufgezeichnet und eine trial-by-trial Modellierung der kortikalen Dynamik durchgeführt. Die Bayes'sche Modellauswahl deutet darauf hin, dass frühe Verarbeitung in den somatosensorischen Kortizes confidence corrected surprise und Bayesian surprise kodiert. Dies legt nahe, dass das somatosensorische System die Überraschung über Beobachtungen signalisiert und ein probabilistisches Modell aktualisiert, welches wiederum Wahrscheinlichkeiten in Bezug auf Übergänge zwischen Reizen lernt. In einer weiteren multimodalen Roving-Stimulus-Studie haben wir diesen Rahmen auch auf die auditorische und visuelle Modalität ausgeweitet. Als Nächstes untersuchten wir Handlungen, indem wir die Empfindlichkeit gegenüber der erwarteten Bayesian surprise betrachteten. Interessanterweise ist diese informationstheoretische Größe auch als Informationsgewinn bekannt und stellt, im Rahmen von active inference, einen Anreiz dar, Unsicherheit zu reduzieren. Dies wiederum kann einer Minimierung der Überraschung entsprechen. Durch den Vergleich von active inference mit einem klassischen Modell des Verstärkungslernens (reinforcement learning) bei der zweistufigen Entscheidungsaufgabe konnten wir erste Belege dafür liefern, dass active inference menschliches modellbasiertes Verhalten besser abbildet. Dies scheint mit der Sensibilität der Teilnehmer gegenüber der erwarteten Bayesian surprise zusammenzuhängen und trägt zur Erklärung des Explorationsverhaltens bei, das jedoch nicht vom reinforcement learning-Modell erklärt werden kann. Insgesamt liefern unsere Ergebnisse Hinweise für Formulierungen der informationstheoretischen Überraschung als Modell für Signale wahrnehmungsbasierten Lernens, die auch menschliches Handeln steuern

Active inference and the two-step task

Sequential decision problems distill important challenges frequently faced by humans. Through repeated interactions with an uncertain world, unknown statistics need to be learned while balancing exploration and exploitation. Reinforcement learning is a prominent method for modeling such behaviour, with a prevalent application being the two-step task. However, recent studies indicate that the standard reinforcement learning model sometimes describes features of human task behaviour inaccurately and incompletely. We investigated whether active inference, a framework proposing a trade-off to the exploration-exploitation dilemma, could better describe human behaviour. Therefore, we re-analysed four publicly available datasets of the two-step task, performed Bayesian model selection, and compared behavioural model predictions. Two datasets, which revealed more model-based inference and behaviour indicative of directed exploration, were better described by active inference, while the models scored similarly for the remaining datasets. Learning using probability distributions appears to contribute to the improved model fits. Further, approximately half of all participants showed sensitivity to information gain as formulated under active inference, although behavioural exploration effects were not fully captured. These results contribute to the empirical validation of active inference as a model of human behaviour and the study of alternative models for the influential two-step task

EEG mismatch responses in a multimodal roving stimulus paradigm provide evidence for probabilistic inference across audition, somatosensation, and vision

The human brain is constantly subjected to a multimodal stream of probabilistic sensory inputs. Electroencephalography (EEG) signatures, such as the mismatch negativity (MMN) and the P3, can give valuable insight into neuronal probabilistic inference. Although reported for different modalities, mismatch responses have largely been studied in isolation, with a strong focus on the auditory MMN. To investigate the extent to which early and late mismatch responses across modalities represent comparable signatures of uni- and cross-modal probabilistic inference in the hierarchically structured cortex, we recorded EEG from 32 participants undergoing a novel tri-modal roving stimulus paradigm. The employed sequences consisted of high and low intensity stimuli in the auditory, somatosensory and visual modalities and were governed by unimodal transition probabilities and cross-modal conditional dependencies. We found modality specific signatures of MMN (~100–200 ms) in all three modalities, which were source localized to the respective sensory cortices and shared right lateralized prefrontal sources. Additionally, we identified a cross-modal signature of mismatch processing in the P3a time range (~300–350 ms), for which a common network with frontal dominance was found. Across modalities, the mismatch responses showed highly comparable parametric effects of stimulus train length, which were driven by standard and deviant response modulations in opposite directions. Strikingly, P3a responses across modalities were increased for mispredicted stimuli with low cross-modal conditional probability, suggesting sensitivity to multimodal (global) predictive sequence properties. Finally, model comparisons indicated that the observed single trial dynamics were best captured by Bayesian learning models tracking unimodal stimulus transitions as well as cross-modal conditional dependencies

Neural surprise in somatosensory Bayesian learning

Tracking statistical regularities of the environment is important for shaping human behavior and perception. Evidence suggests that the brain learns environmental dependencies using Bayesian principles. However, much remains unknown about the employed algorithms, for somesthesis in particular. Here, we describe the cortical dynamics of the somatosensory learning system to investigate both the form of the generative model as well as its neural surprise signatures. Specifically, we recorded EEG data from 40 participants subjected to a somatosensory roving-stimulus paradigm and performed single-trial modeling across peri-stimulus time in both sensor and source space. Our Bayesian model selection procedure indicates that evoked potentials are best described by a non-hierarchical learning model that tracks transitions between observations using leaky integration. From around 70ms post-stimulus onset, secondary somatosensory cortices are found to represent confidence-corrected surprise as a measure of model inadequacy. Indications of Bayesian surprise encoding, reflecting model updating, are found in primary somatosensory cortex from around 140ms. This dissociation is compatible with the idea that early surprise signals may control subsequent model update rates. In sum, our findings support the hypothesis that early somatosensory processing reflects Bayesian perceptual learning and contribute to an understanding of its underlying mechanisms

The effect of ketamine and D-cycloserine on the high frequency resting EEG spectrum in humans

Rationale Preclinical studies indicate that high-frequency oscillations, above 100 Hz (HFO:100–170 Hz), are a potential translatable biomarker for pharmacological studies, with the rapid acting antidepressant ketamine increasing both gamma (40–100 Hz) and HFO. Objectives To assess the effect of the uncompetitive NMDA antagonist ketamine, and of D-cycloserine (DCS), which acts at the glycine site on NMDA receptors on HFO in humans. Methods We carried out a partially double-blind, 4-way crossover study in 24 healthy male volunteers. Each participant received an oral tablet and an intravenous infusion on each of four study days. The oral treatment was either DCS (250 mg or 1000 mg) or placebo. The infusion contained 0.5 mg/kg ketamine or saline placebo. The four study conditions were therefore placebo-placebo, 250 mg DCS-placebo, 1000 mg DCS-placebo, or placebo-ketamine. Results Compared with placebo, frontal midline HFO magnitude was increased by ketamine (p = 0.00014) and 1000 mg DCS (p = 0.013). Frontal gamma magnitude was also increased by both these treatments. However, at a midline parietal location, only HFO were increased by DCS, and not gamma, whilst ketamine increased both gamma and HFO at this location. Ketamine induced psychomimetic effects, as measured by the PSI scale, whereas DCS did not increase the total PSI score. The perceptual distortion subscale scores correlated with the posterior low gamma to frontal high beta ratio. Conclusions Our results suggest that, at high doses, a partial NMDA agonist (DCS) has similar effects on fast neural oscillations as an NMDA antagonist (ketamine). As HFO were induced without psychomimetic effects, they may prove a useful drug development target

Neural surprise in somatosensory Bayesian learning

This dataset accompanies the paper 'Neural surprise in somatosensory Bayesian learning'

Qualitative changes in human γ-secretase underlie familial Alzheimer's disease

Presenilin (PSEN) pathogenic mutations cause familial Alzheimer's disease (AD [FAD]) in an autosomal-dominant manner. The extent to which the healthy and diseased alleles influence each other to cause neurodegeneration remains unclear. In this study, we assessed γ-secretase activity in brain samples from 15 nondemented subjects, 22 FAD patients harboring nine different mutations in PSEN1, and 11 sporadic AD (SAD) patients. FAD and control brain samples had similar overall γ-secretase activity levels, and therefore, loss of overall (endopeptidase) γ-secretase function cannot be an essential part of the pathogenic mechanism. In contrast, impaired carboxypeptidase-like activity (γ-secretase dysfunction) is a constant feature in all FAD brains. Significantly, we demonstrate that pharmacological activation of the carboxypeptidase-like γ-secretase activity with γ-secretase modulators alleviates the mutant PSEN pathogenic effects. Most SAD cases display normal endo- and carboxypeptidase-like γ-secretase activities. However and interestingly, a few SAD patient samples display γ-secretase dysfunction, suggesting that γ-secretase may play a role in some SAD cases. In conclusion, our study highlights qualitative shifts in amyloid-β (Aβ) profiles as the common denominator in FAD and supports a model in which the healthy allele contributes with normal Aβ products and the diseased allele generates longer aggregation-prone peptides that act as seeds inducing toxic amyloid conformations.status: publishe

Qualitative changes in human γ-secretase underlie familial Alzheimer’s disease

Presenilin (PSEN) pathogenic mutations cause familial Alzheimer's disease (AD [FAD]) in an autosomal-dominant manner. The extent to which the healthy and diseased alleles influence each other to cause neurodegeneration remains unclear. In this study, we assessed γ-secretase activity in brain samples from 15 nondemented subjects, 22 FAD patients harboring nine different mutations in PSEN1, and 11 sporadic AD (SAD) patients. FAD and control brain samples had similar overall γ-secretase activity levels, and therefore, loss of overall (endopeptidase) γ-secretase function cannot be an essential part of the pathogenic mechanism. In contrast, impaired carboxypeptidase-like activity (γ-secretase dysfunction) is a constant feature in all FAD brains. Significantly, we demonstrate that pharmacological activation of the carboxypeptidase-like γ-secretase activity with γ-secretase modulators alleviates the mutant PSEN pathogenic effects. Most SAD cases display normal endo- and carboxypeptidase-like γ-secretase activities. However and interestingly, a few SAD patient samples display γ-secretase dysfunction, suggesting that γ-secretase may play a role in some SAD cases. In conclusion, our study highlights qualitative shifts in amyloid-β (Aβ) profiles as the common denominator in FAD and supports a model in which the healthy allele contributes with normal Aβ products and the diseased allele generates longer aggregation-prone peptides that act as seeds inducing toxic amyloid conformations