Active inference and oculomotor pursuit: the dynamic causal modelling of eye movements.

This paper introduces a new paradigm that allows one to quantify the Bayesian beliefs evidenced by subjects during oculomotor pursuit. Subjects' eye tracking responses to a partially occluded sinusoidal target were recorded non-invasively and averaged. These response averages were then analysed using dynamic causal modelling (DCM). In DCM, observed responses are modelled using biologically plausible generative or forward models - usually biophysical models of neuronal activity

GazeStereo3D: seamless disparity manipulations

Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied

Practical Saccade Prediction for Head-Mounted Displays: Towards a Comprehensive Model

Eye-tracking technology is an integral component of new display devices suchas virtual and augmented reality headsets. Applications of gaze informationrange from new interaction techniques exploiting eye patterns togaze-contingent digital content creation. However, system latency is still asignificant issue in many of these applications because it breaks thesynchronization between the current and measured gaze positions. Consequently,it may lead to unwanted visual artifacts and degradation of user experience. Inthis work, we focus on foveated rendering applications where the quality of animage is reduced towards the periphery for computational savings. In foveatedrendering, the presence of latency leads to delayed updates to the renderedframe, making the quality degradation visible to the user. To address thisissue and to combat system latency, recent work proposes to use saccade landingposition prediction to extrapolate the gaze information from delayedeye-tracking samples. While the benefits of such a strategy have already beendemonstrated, the solutions range from simple and efficient ones, which makeseveral assumptions about the saccadic eye movements, to more complex andcostly ones, which use machine learning techniques. Yet, it is unclear to whatextent the prediction can benefit from accounting for additional factors. Thispaper presents a series of experiments investigating the importance ofdifferent factors for saccades prediction in common virtual and augmentedreality applications. In particular, we investigate the effects of saccadeorientation in 3D space and smooth pursuit eye-motion (SPEM) and how theirinfluence compares to the variability across users. We also present a simpleyet efficient correction method that adapts the existing saccade predictionmethods to handle these factors without performing extensive data collection.<br

Aerospace Medicine and Biology. A continuing bibliography with indexes (supplement 225)

This bibliography lists 140 reports, articles, and other documents introduced into the NASA scientific and technical information system in October 1981

On-line motor control in patients with Parkinson's disease

Recent models based, in part on a study of Huntington's disease, suggest that the basal ganglia are involved in on-line movement guidance. Two experiments were conducted to investigate this idea. First, we studied advanced Parkinson's disease patients performing a reaching task known to depend on on-line guidance. The task was to ‘look and point' in the dark at visual targets displayed in the peripheral visual field. In some trials, the target location was slightly modified during saccadic gaze displacement (when vision is suppressed). In both patient and control groups, the target jump induced a gradual modification of the movement which diverged smoothly from its original path to reach the new target location. No deficit was found in the patients, except for an increased latency to respond to the target jump (Parkinson's disease: 243 ms; controls: 166 ms). A computational simulation indicated that this response slowing was likely to be a by-product of bradykinesia. The unexpected inconsistency between this result and previous reports was investigated in a second experiment. We hypothesized that the relevant factor was the characteristics of the corrections to be performed. To test this prediction, we investigated a task requiring corrections of the same type as investigated in Huntington's disease, namely large, consciously detected errors induced by large target jumps at hand movement onset. In contrast with the smooth adjustments observed in the first experiment, the subjects responded to the target jump by generating a discrete corrective sub-movement. While this iterative response was relatively rapid in the control subjects (220 ms), Parkinson's disease patients exhibited either dramatically late (>730 ms) or totally absent on-line corrections. When on-line corrections were absent, the initial motor response was completed before a second corrective response was initiated (the latency of the corrective response was the same as the latency of the initial response). Considered together, these results suggest that basal ganglia dependent circuits are not critical for feedback loops involving a smooth modulation of the ongoing command. These circuits may rather contribute to the generation of discrete corrective sub-movements. This deficit is in line with the general impairment of sequential and simultaneous actions in patients with basal ganglia disorder

The computational neurology of active vision

In this thesis, we appeal to recent developments in theoretical neurobiology – namely, active inference – to understand the active visual system and its disorders. Chapter 1 reviews the neurobiology of active vision. This introduces some of the key conceptual themes around attention and inference that recur through subsequent chapters. Chapter 2 provides a technical overview of active inference, and its interpretation in terms of message passing between populations of neurons. Chapter 3 applies the material in Chapter 2 to provide a computational characterisation of the oculomotor system. This deals with two key challenges in active vision: deciding where to look, and working out how to look there. The homology between this message passing and the brain networks solving these inference problems provide a basis for in silico lesion experiments, and an account of the aberrant neural computations that give rise to clinical oculomotor signs (including internuclear ophthalmoplegia). Chapter 4 picks up on the role of uncertainty resolution in deciding where to look, and examines the role of beliefs about the quality (or precision) of data in perceptual inference. We illustrate how abnormal prior beliefs influence inferences about uncertainty and give rise to neuromodulatory changes and visual hallucinatory phenomena (of the sort associated with synucleinopathies). We then demonstrate how synthetic pharmacological perturbations that alter these neuromodulatory systems give rise to the oculomotor changes associated with drugs acting upon these systems. Chapter 5 develops a model of visual neglect, using an oculomotor version of a line cancellation task. We then test a prediction of this model using magnetoencephalography and dynamic causal modelling. Chapter 6 concludes by situating the work in this thesis in the context of computational neurology. This illustrates how the variational principles used here to characterise the active visual system may be generalised to other sensorimotor systems and their disorders

Computational Study of Multisensory Gaze-Shift Planning

In response to appearance of multimodal events in the environment, we often make a gaze-shift in order to focus the attention and gather more information. Planning such a gaze-shift involves three stages: 1) to determine the spatial location for the gaze-shift, 2) to find out the time to initiate the gaze-shift, 3) to work out a coordinated eye-head motion to execute the gaze-shift. There have been a large number of experimental investigations to inquire the nature of multisensory and oculomotor information processing in any of these three levels separately. Here in this thesis, we approach this problem as a single executive program and propose computational models for them in a unified framework. The first spatial problem is viewed as inferring the cause of cross-modal stimuli, whether or not they originate from a common source (chapter 2). We propose an evidence-accumulation decision-making framework, and introduce a spatiotemporal similarity measure as the criterion to choose to integrate the multimodal information or not. The variability of report of sameness, observed in experiments, is replicated as functions of the spatial and temporal patterns of target presentations. To solve the second temporal problem, a model is built upon the first decision-making structure (chapter 3). We introduce an accumulative measure of confidence on the chosen causal structure, as the criterion for initiation of action. We propose that gaze-shift is implemented when this confidence measure reaches a threshold. The experimentally observed variability of reaction time is simulated as functions of spatiotemporal and reliability features of the cross-modal stimuli. The third motor problem is considered to be solved downstream of the two first networks (chapter 4). We propose a kinematic strategy that coordinates eye-in-head and head-on-shoulder movements, in both spatial and temporal dimensions, in order to shift the line of sight towards the inferred position of the goal. The variabilities in contributions of eyes and head movements to gaze-shift are modeled as functions of the retinal error and the initial orientations of eyes and head. The three models should be viewed as parts of a single executive program that integrates perceptual and motor processing across time and space

3D Dynamic Modeling of the Head-Neck Complex for Fast Eye and Head Orientation Movements Research

A 3D dynamic computer model for the movement of the head-neck complex is presented. It incorporates anatomically correct information about the diverse elements forming the system. The skeleton is considered as a set of interconnected rigid 3D bodies following the Newton-Euler laws of movement. The muscles are modeled using Enderle's linear model, which shows equivalent dynamic characteristics to Loeb's virtual muscle model. The soft tissues, namely, the ligaments, intervertebral disks, and facet joints, are modeled considering their physiological roles and dynamics. In contrast with other head and neck models developed for safety research, the model is aimed to study the neural control of the complex during fast eye and head movements, such as saccades and gaze shifts. In particular, the time-optimal hypothesis and the feedback control ones are discussed

Aerospace medicine and biology. A continuing bibliography with indexes, supplement 206, May 1980

This bibliography lists 169 reports, articles, and other documents introduced into the NASA scientific and technical information system in April 1980

Ocular biomechanics modelling for visual fatigue assessment in virtual environments

The study objectively quantifies visual fatigue caused by immersion in virtual reality. Visual fatigue assessment is done through ocular biomechanics modelling and eye tracking to analyse eye movement and muscle forces into a visual fatigue index