Gaze-orientation during transient occlusion

Fast moving objects are often transiently occluded in our normal surrounds as they pass behind other surfaces and objects. The consequent loss of drive from visual feedback can be compensated by extra-retinal input. Evidence from behavioural studies indicates that gaze orientation during transient occlusion is not simply a reflexive response but can also be predictive of the upcoming motion. Indeed, while smooth pursuit eye velocity often falls below target velocity during a transient occlusion, it increases prior to object reappearance in order to reduce retinal slip. Moreover, the smooth response is combined with saccadic eye movements that match eye displacement to object displacement and thus minimize position error at object reappearance. Comparisons of conditions that require fixation or pursuit suggest that the maintenance of gaze orientation during transient occlusion is the habitual response and can facilitate both spatial and temporal estimation. Interconnected areas of the frontal and parietal cortex have been shown to be active during pursuit of occluded object motion and are thus thought to be involved in the control of gaze-orientation as well as representing object motion. Future work should determine whether expertise in sport mediates oculomotor control and thereby perception of relevant information to support expert performance

Event Prediction and Object Motion Estimation in the Development of Visual Attention

A model of gaze control is describes that includes mechanisms for predictive control using a forward model and event driven expectations of target behavior. The model roughly undergoes stages similar to those of human infants if the influence of the predictive systems is gradually increased

A Computer-Aided Training (CAT) System for Short Track Speed Skating

Short track speed skating has become popular all over the world. The demands of a computer-aided training (CAT) system are booming due to this fact. However, the existing commercial systems for sports are highly dependent on expensive equipment and complicated hardware calibration. This dissertation presents a novel CAT system for tracking multiple skaters in short track skating competitions. Aiming at the challenges, we utilize global rink information to compensate camera motion and obtain the global spatial information of skaters; apply Random Forest to fuse multiple cues and predict the blobs for each of the skaters; and finally develop a silhouette and edge-based template matching and blob growing method to allocate each blob to corresponding skaters. The proposed multiple skaters tracking algorithm organically integrates multi-cue fusion, dynamic appearance modeling, machine learning, etc. to form an efficient and robust CAT system. The effectiveness and robustness of the proposed method are presented through experiments

Occlusion-Robust MVO: Multimotion Estimation Through Occlusion Via Motion Closure

Visual motion estimation is an integral and well-studied challenge in autonomous navigation. Recent work has focused on addressing multimotion estimation, which is especially challenging in highly dynamic environments. Such environments not only comprise multiple, complex motions but also tend to exhibit significant occlusion. Previous work in object tracking focuses on maintaining the integrity of object tracks but usually relies on specific appearance-based descriptors or constrained motion models. These approaches are very effective in specific applications but do not generalize to the full multimotion estimation problem. This paper presents a pipeline for estimating multiple motions, including the camera egomotion, in the presence of occlusions. This approach uses an expressive motion prior to estimate the SE (3) trajectory of every motion in the scene, even during temporary occlusions, and identify the reappearance of motions through motion closure. The performance of this occlusion-robust multimotion visual odometry (MVO) pipeline is evaluated on real-world data and the Oxford Multimotion Dataset.Comment: To appear at the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). An earlier version of this work first appeared at the Long-term Human Motion Planning Workshop (ICRA 2019). 8 pages, 5 figures. Video available at https://www.youtube.com/watch?v=o_N71AA6FR

Visual motion processing and human tracking behavior

The accurate visual tracking of a moving object is a human fundamental skill that allows to reduce the relative slip and instability of the object's image on the retina, thus granting a stable, high-quality vision. In order to optimize tracking performance across time, a quick estimate of the object's global motion properties needs to be fed to the oculomotor system and dynamically updated. Concurrently, performance can be greatly improved in terms of latency and accuracy by taking into account predictive cues, especially under variable conditions of visibility and in presence of ambiguous retinal information. Here, we review several recent studies focusing on the integration of retinal and extra-retinal information for the control of human smooth pursuit.By dynamically probing the tracking performance with well established paradigms in the visual perception and oculomotor literature we provide the basis to test theoretical hypotheses within the framework of dynamic probabilistic inference. We will in particular present the applications of these results in light of state-of-the-art computer vision algorithms

Key-Pose Prediction in Cyclic Human Motion

In this paper we study the problem of estimating innercyclic time intervals within repetitive motion sequences of top-class swimmers in a swimming channel. Interval limits are given by temporal occurrences of key-poses, i.e. distinctive postures of the body. A key-pose is defined by means of only one or two specific features of the complete posture. It is often difficult to detect such subtle features directly. We therefore propose the following method: Given that we observe the swimmer from the side, we build a pictorial structure of poselets to robustly identify random support poses within the regular motion of a swimmer. We formulate a maximum likelihood model which predicts a key-pose given the occurrences of multiple support poses within one stroke. The maximum likelihood can be extended with prior knowledge about the temporal location of a key-pose in order to improve the prediction recall. We experimentally show that our models reliably and robustly detect key-poses with a high precision and that their performance can be improved by extending the framework with additional camera views.Comment: Accepted at WACV 2015, 8 pages, 3 figure

The extrapolation of occluded motion: basic mechanism and application

Predicting the future states of moving objects that are hidden by an occluder for a brief period is of paramount importance to our ability to interact within a dynamic environment. This phenomenon is known as motion extrapolation (ME). Numerous gaps in the literature can be found disregarding the mechanisms involved in ME of which the current thesis attempts to address. Behavioural experiments usually utilize a prediction-of-motion paradigm, which requires participants to make a direct estimation of the time-to-contact (TTC). In this task, the initial trajectory of a target stimulus is presented, which then becomes occluded, observers are then asked to respond when they believe the target has reached a marked point behind that occluder without it ever actually reappearing (Tresilian, 1999; Rosenbaum, 1972). Alternatively, other experiments have adopted a timing discrimination task in which participants are required to indicate whether a moving target, following occlusion, reappears ‘early’ or ‘late’ (Makin, Poliakoff & El-Deredy, 2009; Makin, Poliakoff, Ackerley & El-Deredy, 2012). Experiments In the first part of this thesis, I investigated whether the visual memory system is active during the extrapolation of occluded motion and whether it reflects speed misperception due to the well-known illusion such as the apparent slower speed of low contrast object or large size object (Thompson 1982; Epstein 1978). Results revealed that with a TTC task observers estimate longer time to contact with low contrast and large stimuli compared to high contrast and small stimuli respectively. Note that the stimuli in both conditions are moving at equal speed. Therefore, the illusion of the apparent slower speed with low contrast and large stimuli remains in the visual memory system and influences motion extrapolation. Chapter III aims to investigate the interaction between real motion and motion extrapolation. Gilden and colleagues (1995) showed that motion adaptation affects TTC judgment showing that real motion detectors are somehow also involved during ME. A step further that I made was to investigate the effect of brief motion priming and adaptation, occurring at the earliest levels of the cortical visual streams, on time-to-contact (TTC) estimation of a target passing behind an occluder. By using different exposure times of directional motion presented in the occluder area prior to the target’s disappearance behind it, my aim was to modulate (prime or adapt) extrapolated motion of the invisible target, thus producing different TTC estimates. Results showed that longer (yet sub-second) exposures to motion in the same direction of the target produced late TTC estimates, whereas shorter exposures produced shorter TTC estimates, indicating that rapid forms of motion adaptation and motion priming affect extrapolated motion. My findings suggest that motion extrapolation might occur at the earliest levels of cortical processing of motion, where these rapid mechanisms of priming and adaptation take place. In Chapter IV of my thesis, I explore not only the visual factors of motion extrapolation, but also the timing mechanisms involved and their electrophysiological correlates. The first question is whether the temporal processing is required for accurate ME, and whether this is indexed by neural activity of the Contingent Negative Variation (CNV). A second question is, whether there is a specific electrophysiological correlates that highlight the shifting from real motion perception to motion extrapolation. In this electroencephalographic experiment, participants were adapted with a moving texture (Gilden et al., 1995). The adaptation with the moving texture could bias and modify temporal processing. Participants made a direct estimation of Time to Contact, which showed that classic adaptations were able to bias temporal judgments and modulate the amplitude of the CNV, suggesting a complex feedforward-feedback network between low- and high level cortical mechanisms. Finally, a negative defection (N190) was found, for the first time, as a neurophysiological correlate in the temporal-occipital electrodes in the right and left hemisphere for the rightwards and leftwards ME respectively, indicating the involvement of motion mechanisms of intermediate cortical level in ME. Chapter V aims to show at distinguishing between extrapolation, and interpolation of occluded motion. Extrapolation is the ability to extract the trajectory, speed and direction of a moving target that becomes hidden by an occluder, thanks to the information extracted from the visible trajectory. Interpolation is a similar phenomenon, i.e. from the visible trajectory one can extract speed and direction as in Extrapolation. The main difference is that for interpolate visible cue are needed along the invisible trajectory. If the occluder is invisible and the occluded trajectory is symmetrical respect to a visible cue, one can connect these cues (spatial points) in order to form a spatio-temporal map and infer where and when the target will reappear. This is not possible in absence of visible cues such as in extrapolation condition. In a new task, observers were required to press a button as fast as possible (reaction time) when they saw a moving target reappearing from an invisible occluder. Results showed that observers could even anticipate the reappearance of an object moving behind the occluder. However, only in some circumstances: i) when the occluder was not positioned over the blind spot but in retinal areas that project to the visual cortex; ii) with an entirely invisible occluder the visible motion before occlusion had to be presented and iii) visual-spatial cues had to signal the center of the invisible trajectory. When these conditions are given, observers can use the spatial information given by the point of disappearance, the visible cue that represented the center of the invisible trajectory, then infer the point of reappearance by symmetry. Therefore having a set of discrete spatial positions (and its cortical representation) in which the moving occluded target will be in a certain moment of time, it is convenient to interpolate this point in order to create a spatio-temporal map to infer where and when the object will be (saliency map). I consider this process of motion interpolation as an amodal filling-in process. The last part of my thesis involved a practical application of ME. Participants cannot interpolate when the moving target passes in a zone over retinal areas that do not project to the visual cortex (blind spot). In this case, observers perform a true reaction time and do not anticipate the response. Patients with Macular Degeneration cannot see with their fovea since it is damaged. Therefore, that part of the retina does not project to the visual cortex anymore. In a task in which they have to press a response button when a moving target disappear into or reappear from their scotoma, we predict that they cannot anticipate the response to the reappearance of the target. Five patients with macular degeneration were therefore instructed to press a button when they see a moving target disappear into and reappear from their scotoma. Patients repeated this task several times with different linear trajectories of the target. Connecting the point in space in which a patient presses the button, it was possible to draw the shape and the size of the scotoma with a software. The size of the scomota found with this experiment was compared with that measured with a Nidek MP-1. A linear correlation of R2 about of 0.8 was found between the Nidek MP-1 and scotoma measured connecting the point in which patients reported to see the target reappear from their scotoma. Therefore, this software which was written by me (considering its limits) may become a useful tool to obtain a reliable perimetry in a given situation in which an expensive machine such as the MP-1 is not available

Velocity memory

It is known that primates are sensitive to the velocity of moving objects. We can also remember velocity information after moving objects disappear. This cognitive faculty has been investigated before, however, the literature on velocity memory to date has been fragmented. For example, velocity memory has been disparately described as a system that controls eye movements and delayed discrimination. Furthermore, velocity memory may have a role in motion extrapolation, i.e. the ability to judge the position of a moving target after it becomes occluded. This thesis provides a unifying account of velocity memory, and uses electroencephalography (EEG) to explore its neural basis. In Chapter 2, the relationship between oculomotor control and motion extrapolation was investigated. Two forms of motion extrapolation task were presented. In the first, participants observed a moving target disappear then reappear further along its path. Reappearance could be at the correct time, too early or too late. Participants discriminated reappearance error with a two-alternative forced choice button press. In the second task, participants saw identical targets travel behind a visible occluder, and they attempted to press a button at the exact time that it reached the other side. Tasks were completed under fixation and free viewing conditions. The accuracy of participant's judgments was reduced by fixation in both tasks. In addition, eye movements were systematically related to behavioural responses, and small eye movements during fixation were affected by occluded motion. These three results imply that common velocity memory and pre-motor systems mediate eye movements and motion extrapolation. In Chapter 3, different types of velocity representation were explored. Another motion extrapolation task was presented, and targets of a particular colour were associated with fast or slow motion. On identical-velocity probe trials, colour still influenced response times. This indicates that long-term colour-velocity associations influence motion extrapolation. In Chapter 4, interference between subsequently encoded velocities was explored. There was robust interference between motion extrapolation and delayed discrimination tasks, suggesting that common processes are involved in both. In Chapter 5, EEG was used to investigate when memory-guided tracking begins during motion extrapolation. This study compared conditions where participants covertly tracked visible and occluded targets. It was found that a specific event related potential (ERP) appeared around 200 ms post occlusion, irrespective of target location or velocity. This component could delineate the onset of memory guided tracking during occlusion. Finally, Chapter 6 presents evidence that a change in alpha band activity is associated with information processing during motion extrapolation tasks. In light of these results, it is concluded that a common velocity memory system is involved a variety of tasks. In the general discussion (Chapter 7), a new account of velocity memory is proposed. It is suggested that a velocity memory reflects persistent synchronization across several velocity sensitive neural populations after stimulus offset. This distributed network is involved in sensory-motor integration, and can remain active without visual input. Theoretical work on eye movements, delayed discrimination and motion extrapolation could benefit from this account of velocity memory.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Velocity memory

It is known that primates are sensitive to the velocity of moving objects. We can also remember velocity information after moving objects disappear. This cognitive faculty has been investigated before, however, the literature on velocity memory to date has been fragmented. For example, velocity memory has been disparately described as a system that controls eye movements and delayed discrimination. Furthermore, velocity memory may have a role in motion extrapolation, i.e. the ability to judge the position of a moving target after it becomes occluded. This thesis provides a unifying account of velocity memory, and uses electroencephalography (EEG) to explore its neural basis. In Chapter 2, the relationship between oculomotor control and motion extrapolation was investigated. Two forms of motion extrapolation task were presented. In the first, participants observed a moving target disappear then reappear further along its path. Reappearance could be at the correct time, too early or too late. Participants discriminated reappearance error with a two-alternative forced choice button press. In the second task, participants saw identical targets travel behind a visible occluder, and they attempted to press a button at the exact time that it reached the other side. Tasks were completed under fixation and free viewing conditions. The accuracy of participant's judgments was reduced by fixation in both tasks. In addition, eye movements were systematically related to behavioural responses, and small eye movements during fixation were affected by occluded motion. These three results imply that common velocity memory and pre-motor systems mediate eye movements and motion extrapolation. In Chapter 3, different types of velocity representation were explored. Another motion extrapolation task was presented, and targets of a particular colour were associated with fast or slow motion. On identical-velocity probe trials, colour still influenced response times. This indicates that long-term colour-velocity associations influence motion extrapolation. In Chapter 4, interference between subsequently encoded velocities was explored. There was robust interference between motion extrapolation and delayed discrimination tasks, suggesting that common processes are involved in both. In Chapter 5, EEG was used to investigate when memory-guided tracking begins during motion extrapolation. This study compared conditions where participants covertly tracked visible and occluded targets. It was found that a specific event related potential (ERP) appeared around 200 ms post occlusion, irrespective of target location or velocity. This component could delineate the onset of memory guided tracking during occlusion. Finally, Chapter 6 presents evidence that a change in alpha band activity is associated with information processing during motion extrapolation tasks. In light of these results, it is concluded that a common velocity memory system is involved a variety of tasks. In the general discussion (Chapter 7), a new account of velocity memory is proposed. It is suggested that a velocity memory reflects persistent synchronization across several velocity sensitive neural populations after stimulus offset. This distributed network is involved in sensory-motor integration, and can remain active without visual input. Theoretical work on eye movements, delayed discrimination and motion extrapolation could benefit from this account of velocity memory.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Active inference and oculomotor pursuit: the dynamic causal modelling of eye movements.

This paper introduces a new paradigm that allows one to quantify the Bayesian beliefs evidenced by subjects during oculomotor pursuit. Subjects' eye tracking responses to a partially occluded sinusoidal target were recorded non-invasively and averaged. These response averages were then analysed using dynamic causal modelling (DCM). In DCM, observed responses are modelled using biologically plausible generative or forward models - usually biophysical models of neuronal activity