Active inference, eye movements and oculomotor delays.

This paper considers the problem of sensorimotor delays in the optimal control of (smooth) eye movements under uncertainty. Specifically, we consider delays in the visuo-oculomotor loop and their implications for active inference. Active inference uses a generalisation of Kalman filtering to provide Bayes optimal estimates of hidden states and action in generalised coordinates of motion. Representing hidden states in generalised coordinates provides a simple way of compensating for both sensory and oculomotor delays. The efficacy of this scheme is illustrated using neuronal simulations of pursuit initiation responses, with and without compensation. We then consider an extension of the generative model to simulate smooth pursuit eye movements-in which the visuo-oculomotor system believes both the target and its centre of gaze are attracted to a (hidden) point moving in the visual field. Finally, the generative model is equipped with a hierarchical structure, so that it can recognise and remember unseen (occluded) trajectories and emit anticipatory responses. These simulations speak to a straightforward and neurobiologically plausible solution to the generic problem of integrating information from different sources with different temporal delays and the particular difficulties encountered when a system-like the oculomotor system-tries to control its environment with delayed signals

Visual motion processing and human tracking behavior

The accurate visual tracking of a moving object is a human fundamental skill that allows to reduce the relative slip and instability of the object's image on the retina, thus granting a stable, high-quality vision. In order to optimize tracking performance across time, a quick estimate of the object's global motion properties needs to be fed to the oculomotor system and dynamically updated. Concurrently, performance can be greatly improved in terms of latency and accuracy by taking into account predictive cues, especially under variable conditions of visibility and in presence of ambiguous retinal information. Here, we review several recent studies focusing on the integration of retinal and extra-retinal information for the control of human smooth pursuit.By dynamically probing the tracking performance with well established paradigms in the visual perception and oculomotor literature we provide the basis to test theoretical hypotheses within the framework of dynamic probabilistic inference. We will in particular present the applications of these results in light of state-of-the-art computer vision algorithms

Active inference and oculomotor pursuit: the dynamic causal modelling of eye movements.

This paper introduces a new paradigm that allows one to quantify the Bayesian beliefs evidenced by subjects during oculomotor pursuit. Subjects' eye tracking responses to a partially occluded sinusoidal target were recorded non-invasively and averaged. These response averages were then analysed using dynamic causal modelling (DCM). In DCM, observed responses are modelled using biologically plausible generative or forward models - usually biophysical models of neuronal activity

Active inference and the anatomy of oculomotion

Given that eye movement control can be framed as an inferential process, how are the requisite forces generated to produce anticipated or desired fixation? Starting from a generative model based on simple Newtonian equations of motion, we derive a variational solution to this problem and illustrate the plausibility of its implementation in the oculomotor brainstem. We show, through simulation, that the Bayesian filtering equations that implement ‘planning as inference’ can generate both saccadic and smooth pursuit eye movements. Crucially, the associated message passing maps well onto the known connectivity and neuroanatomy of the brainstem – and the changes in these messages over time are strikingly similar to single unit recordings of neurons in the corresponding nuclei. Furthermore, we show that simulated lesions to axonal pathways reproduce eye movement patterns of neurological patients with damage to these tracts

Action and behavior: a free-energy formulation

We have previously tried to explain perceptual inference and learning under a free-energy principle that pursues Helmholtz’s agenda to understand the brain in terms of energy minimization. It is fairly easy to show that making inferences about the causes of sensory data can be cast as the minimization of a free-energy bound on the likelihood of sensory inputs, given an internal model of how they were caused. In this article, we consider what would happen if the data themselves were sampled to minimize this bound. It transpires that the ensuing active sampling or inference is mandated by ergodic arguments based on the very existence of adaptive agents. Furthermore, it accounts for many aspects of motor behavior; from retinal stabilization to goal-seeking. In particular, it suggests that motor control can be understood as fulfilling prior expectations about proprioceptive sensations. This formulation can explain why adaptive behavior emerges in biological agents and suggests a simple alternative to optimal control theory. We illustrate these points using simulations of oculomotor control and then apply to same principles to cued and goal-directed movements. In short, the free-energy formulation may provide an alternative perspective on the motor control that places it in an intimate relationship with perception

Anticipatory Semantic Processes

Why anticipatory processes correspond to cognitive abilities of living systems? To be adapted to an environment, behaviors need at least i) internal representations of events occurring in the external environment; and ii) internal anticipations of possible events to occur in the external environment. Interactions of these two opposite but complementary cognitive properties lead to various patterns of experimental data on semantic processing. How to investigate dynamic semantic processes? Experimental studies in cognitive psychology offer several interests such as: i) the control of the semantic environment such as words embedded in sentences; ii) the methodological tools allowing the observation of anticipations and adapted oculomotor behavior during reading; and iii) the analyze of different anticipatory processes within the theoretical framework of semantic processing. What are the different types of semantic anticipations? Experimental data show that semantic anticipatory processes involve i) the coding in memory of sequences of words occurring in textual environments; ii) the anticipation of possible future words from currently perceived words; and iii) the selection of anticipated words as a function of the sequences of perceived words, achieved by anticipatory activations and inhibitory selection processes. How to modelize anticipatory semantic processes? Localist or distributed neural networks models can account for some types of semantic processes, anticipatory or not. Attractor neural networks coding temporal sequences are presented as good candidate for modeling anticipatory semantic processes, according to specific properties of the human brain such as i) auto-associative memory; ii) learning and memorization of sequences of patterns; and iii) anticipation of memorized patterns from previously perceived patterns

Dynamic causal modelling of eye movements during pursuit: Confirming precision-encoding in V1 using MEG

This paper shows that it is possible to estimate the subjective precision (inverse variance) of Bayesian beliefs during oculomotor pursuit. Subjects viewed a sinusoidal target, with or without random fluctuations in its motion. Eye trajectories and magnetoencephalographic (MEG) data were recorded concurrently. The target was periodically occluded, such that its reappearance caused a visual evoked response field (ERF). Dynamic causal modelling (DCM) was used to fit models of eye trajectories and the ERFs. The DCM for pursuit was based on predictive coding and active inference, and predicts subjects' eye movements based on their (subjective) Bayesian beliefs about target (and eye) motion. The precisions of these hierarchical beliefs can be inferred from behavioural (pursuit) data. The DCM for MEG data used an established biophysical model of neuronal activity that includes parameters for the gain of superficial pyramidal cells, which is thought to encode precision at the neuronal level. Previous studies (using DCM of pursuit data) suggest that noisy target motion increases subjective precision at the sensory level: i.e., subjects attend more to the target's sensory attributes. We compared (noisy motion-induced) changes in the synaptic gain based on the modelling of MEG data to changes in subjective precision estimated using the pursuit data. We demonstrate that imprecise target motion increases the gain of superficial pyramidal cells in V1 (across subjects). Furthermore, increases in sensory precision – inferred by our behavioural DCM – correlate with the increase in gain in V1, across subjects. This is a step towards a fully integrated model of brain computations, cortical responses and behaviour that may provide a useful clinical tool in conditions like schizophrenia

Generative Models for Active Vision

The active visual system comprises the visual cortices, cerebral attention networks, and oculomotor system. While fascinating in its own right, it is also an important model for sensorimotor networks in general. A prominent approach to studying this system is active inference—which assumes the brain makes use of an internal (generative) model to predict proprioceptive and visual input. This approach treats action as ensuring sensations conform to predictions (i.e., by moving the eyes) and posits that visual percepts are the consequence of updating predictions to conform to sensations. Under active inference, the challenge is to identify the form of the generative model that makes these predictions—and thus directs behavior. In this paper, we provide an overview of the generative models that the brain must employ to engage in active vision. This means specifying the processes that explain retinal cell activity and proprioceptive information from oculomotor muscle fibers. In addition to the mechanics of the eyes and retina, these processes include our choices about where to move our eyes. These decisions rest upon beliefs about salient locations, or the potential for information gain and belief-updating. A key theme of this paper is the relationship between “looking” and “seeing” under the brain's implicit generative model of the visual world

Detecting motion trajectories: How do perception and action use visual information?

This item is only available electronically.Whenever a person moves to intercept an object, they engage in a complex set of predictions, about the object’s trajectory, and about the set of motions required to intercept it. However, the way that people use perceptual information to intercept rapidly moving objects is currently not well understood. This is because the problem is multifaceted, as there are delays in receptor transduction, neural conduction, processing and muscle activation. There is considerable as to how the two systems interact, there is some evidence that they do (Watamaniuk & Heinen, 2003). In order to assess the differences between trajectory prediction for perceptual judgments and pointing movements we examined participants using the same stimulus, a moving random dot cinematogram (Watamaniuk & Heinen, 1999; Williams & Sekuler, 1984), which was manipulated across conditions. We used a within subjects repeated measures design to compare participants’ performance on two tasks, a perceptual (two alternative forced-choice) task and a pointing task (N = 6). For both tasks we assessed participants’ precision in extrapolating the trajectory of the cinematogram, as well as their response latency. If the two systems use the same visual information, we would expect that precision for each task changes similarly across the conditions. We found similar patterns of error for both tasks, with lower durations and higher bandwidth motion signals displaying greater directional error. This provides further insight into how we use visual information to guide movement. In particular, it provides insight as to how differences in motion perception affects interceptive movements.Thesis (B.PsychSc(Hons)) -- University of Adelaide, School of Psychology, 202

Knowing when to stop: Aberrant precision and evidence accumulation in schizophrenia

Predictive coding and active inference formulations of the dysconnection hypothesis suggest that subjects with schizophrenia (SZ) hold unduly precise prior beliefs to compensate for a failure of sensory attenuation. This implies that SZ subjects should both initiate responses prematurely during evidence-accumulation tasks and fail to inhibit their responses at long stop-signal delays. SZ and healthy control subjects were asked to report the timing of billiards-ball collisions and were occasionally required to withhold their responses. SZ subjects showed larger temporal estimation errors, which were associated with premature responses and decreased response inhibition. To account for these effects, we used hierarchical (Bayesian) drift-diffusion models (HDDM) and model selection procedures to adjudicate among four hypotheses. HDDM revealed that the precision of prior beliefs (i.e., starting point) rather than increased sensory precision (i.e., drift rate) drove premature responses and impaired response inhibition in patients with SZ. From the perspective of active inference, we suggest that premature predictions in SZ are responses that, heuristically, are traded off against accuracy to ensure action execution. On the basis of previous work, we suggest that the right insular cortex might mediate this trade-off