A Bayesian foundation for individual learning under uncertainty

Computational learning models are critical for understanding mechanisms of adaptive behavior. However, the two major current frameworks, reinforcement learning (RL) and Bayesian learning, both have certain limitations. For example, many Bayesian models are agnostic of inter-individual variability and involve complicated integrals, making online learning difficult. Here, we introduce a generic hierarchical Bayesian framework for individual learning under multiple forms of uncertainty (e. g., environmental volatility and perceptual uncertainty). The model assumes Gaussian random walks of states at all but the first level, with the step size determined by the next highest level. The coupling between levels is controlled by parameters that shape the influence of uncertainty on learning in a subject-specific fashion. Using variational Bayes under a mean-field approximation and a novel approximation to the posterior energy function, we derive trial-by-trial update equations which (i) are analytical and extremely efficient, enabling real-time learning, (ii) have a natural interpretation in terms of RL, and (iii) contain parameters representing processes which play a key role in current theories of learning, e. g., precision-weighting of prediction error. These parameters allow for the expression of individual differences in learning and may relate to specific neuromodulatory mechanisms in the brain. Our model is very general: it can deal with both discrete and continuous states and equally accounts for deterministic and probabilistic relations between environmental events and perceptual states (i. e., situations with and without perceptual uncertainty). These properties are illustrated by simulations and analyses of empirical time series. Overall, our framework provides a novel foundation for understanding normal and pathological learning that contextualizes RL within a generic Bayesian scheme and thus connects it to principles of optimality from probability theory

An electrophysiological validation of stochastic DCM for fMRI

In this note, we assess the predictive validity of stochastic dynamic causal modeling (sDCM) of functional magnetic resonance imaging (fMRI) data, in terms of its ability to explain changes in the frequency spectrum of concurrently acquired electroencephalography (EEG) signal. We first revisit the heuristic model proposed in Kilner et al. (2005), which suggests that fMRI activation is associated with a frequency modulation of the EEG signal (rather than an amplitude modulation within frequency bands). We propose a quantitative derivation of the underlying idea, based upon a neural field formulation of cortical activity. In brief, dense lateral connections induce a separation of time scales, whereby fast (and high spatial frequency) modes are enslaved by slow (low spatial frequency) modes. This slaving effect is such that the frequency spectrum of fast modes (which dominate EEG signals) is controlled by the amplitude of slow modes (which dominate fMRI signals). We then use conjoint empirical EEG-fMRI data—acquired in epilepsy patients—to demonstrate the electrophysiological underpinning of neural fluctuations inferred from sDCM for fMRI