Event Prediction and Object Motion Estimation in the Development of Visual Attention

A model of gaze control is describes that includes mechanisms for predictive control using a forward model and event driven expectations of target behavior. The model roughly undergoes stages similar to those of human infants if the influence of the predictive systems is gradually increased

Visual motion processing and human tracking behavior

The accurate visual tracking of a moving object is a human fundamental skill that allows to reduce the relative slip and instability of the object's image on the retina, thus granting a stable, high-quality vision. In order to optimize tracking performance across time, a quick estimate of the object's global motion properties needs to be fed to the oculomotor system and dynamically updated. Concurrently, performance can be greatly improved in terms of latency and accuracy by taking into account predictive cues, especially under variable conditions of visibility and in presence of ambiguous retinal information. Here, we review several recent studies focusing on the integration of retinal and extra-retinal information for the control of human smooth pursuit.By dynamically probing the tracking performance with well established paradigms in the visual perception and oculomotor literature we provide the basis to test theoretical hypotheses within the framework of dynamic probabilistic inference. We will in particular present the applications of these results in light of state-of-the-art computer vision algorithms

Using the literature to quantify the learning curve: a case study

Objective: To assess whether a literature review of a technology can allow a learning curve to be quantified. Methods: The literature for fibreoptic intubation was searched for studies reporting information relevant to the learning curve. The Cochrane Librar y, Medline, Embase and Science Citation index were searched. Studies that reported a procedure time were included. Data were abstracted on the three features of learning: initial level, rate of learning and asymptote level. Random effect meta-analysis was performed. Results: Only 21 studies gave explicit information concerning the previous experience of the operator(s). There were 32 different definitions of procedure time. From 4 studies of fibreoptic nasotracheal intubation, the mean starting level and time for the 10th procedure (95% confidence interval) was estimated to be 133s (113, 153) and 71s (62, 79) respectively. Conclusions: The review approach allowed learning to be quantified for our example technology. Poor and insufficient reporting constrained formal statistical estimation. Standardised reporting of non-drug techniques with adequate learning curve details is needed to inform trial design and costeffectiveness analysis

Gaze-orientation during transient occlusion

Fast moving objects are often transiently occluded in our normal surrounds as they pass behind other surfaces and objects. The consequent loss of drive from visual feedback can be compensated by extra-retinal input. Evidence from behavioural studies indicates that gaze orientation during transient occlusion is not simply a reflexive response but can also be predictive of the upcoming motion. Indeed, while smooth pursuit eye velocity often falls below target velocity during a transient occlusion, it increases prior to object reappearance in order to reduce retinal slip. Moreover, the smooth response is combined with saccadic eye movements that match eye displacement to object displacement and thus minimize position error at object reappearance. Comparisons of conditions that require fixation or pursuit suggest that the maintenance of gaze orientation during transient occlusion is the habitual response and can facilitate both spatial and temporal estimation. Interconnected areas of the frontal and parietal cortex have been shown to be active during pursuit of occluded object motion and are thus thought to be involved in the control of gaze-orientation as well as representing object motion. Future work should determine whether expertise in sport mediates oculomotor control and thereby perception of relevant information to support expert performance

Study of Human Hand-Eye Coordination Using Machine Learning Techniques in a Virtual Reality Setup

Theories of visually guided action are characterized as closed-loop control in the presence of reliable sources of visual information, and predictive control to compensate for visuomotor delay and temporary occlusion. However, prediction is not well understood. To investigate, a series of studies was designed to characterize the role of predictive strategies in humans as they perform visually guided actions, and to guide the development of computational models that capture these strategies. During data collection, subjects were immersed in a virtual reality (VR) system and were tasked with using a paddle to intercept a virtual ball. To force subjects into a predictive mode of control, the ball was occluded or made invisible for a portion of its 3D parabolic trajectory. The subjects gaze, hand and head movements were recorded during the performance. To improve the quality of gaze estimation, new algorithms were developed for the measurement and calibration of spatial and temporal errors of an eye tracking system. The analysis focused on the subjects gaze and hand movements reveal that, when the temporal constraints of the task did not allow the subjects to use closed-loop control, they utilized a short-term predictive strategy. Insights gained through behavioral analysis were formalized into computational models of visual prediction using machine learning techniques. In one study, LSTM recurrent neural networks were utilized to explain how information is integrated and used to guide predictive movement of the hand and eyes. In a subsequent study, subject data was used to train an inverse reinforcement learning (IRL) model that captures the full spectrum of strategies from closed-loop to predictive control of gaze and paddle placement. A comparison of recovered reward values between occlusion and no-occlusion conditions revealed a transition from online to predictive control strategies within a single course of action. This work has shed new insights on predictive strategies that guide our eye and hand movements

GHOST: experimenting countermeasures for conflicts in the pilot's activity

An approach for designing countermeasures to cure conflict in aircraft pilots’ activities is presented, both based on Artificial Intelligence and Human Factors concepts. The first step is to track the pilot’s activity, i.e. to reconstruct what he has actually done thanks to the flight parameters and reference models describing the mission and procedures. The second step is to detect conflict in the pilot’s activity, and this is linked to what really matters to the achievement of the mission. The third step is to design accu- rate countermeasures which are likely to do bet- ter than the existing onboard devices. The three steps are presented and supported by experimental results obtained from private and professional pi- lots

Brain Control of Movement Execution Onset Using Local Field Potentials in Posterior Parietal Cortex

The precise control of movement execution onset is essential for safe and autonomous cortical motor prosthetics. A recent study from the parietal reach region (PRR) suggested that the local field potentials (LFPs) in this area might be useful for decoding execution time information because of the striking difference in the LFP spectrum between the plan and execution states (Scherberger et al., 2005). More specifically, the LFP power in the 0–10 Hz band sharply rises while the power in the 20–40 Hz band falls as the state transitions from plan to execution. However, a change of visual stimulus immediately preceded reach onset, raising the possibility that the observed spectral change reflected the visual event instead of the reach onset. Here, we tested this possibility and found that the LFP spectrum change was still time locked to the movement onset in the absence of a visual event in self-paced reaches. Furthermore, we successfully trained the macaque subjects to use the LFP spectrum change as a "go" signal in a closed-loop brain-control task in which the animals only modulated the LFP and did not execute a reach. The execution onset was signaled by the change in the LFP spectrum while the target position of the cursor was controlled by the spike firing rates recorded from the same site. The results corroborate that the LFP spectrum change in PRR is a robust indicator for the movement onset and can be used for control of execution onset in a cortical prosthesis

A Computer-Aided Training (CAT) System for Short Track Speed Skating

Short track speed skating has become popular all over the world. The demands of a computer-aided training (CAT) system are booming due to this fact. However, the existing commercial systems for sports are highly dependent on expensive equipment and complicated hardware calibration. This dissertation presents a novel CAT system for tracking multiple skaters in short track skating competitions. Aiming at the challenges, we utilize global rink information to compensate camera motion and obtain the global spatial information of skaters; apply Random Forest to fuse multiple cues and predict the blobs for each of the skaters; and finally develop a silhouette and edge-based template matching and blob growing method to allocate each blob to corresponding skaters. The proposed multiple skaters tracking algorithm organically integrates multi-cue fusion, dynamic appearance modeling, machine learning, etc. to form an efficient and robust CAT system. The effectiveness and robustness of the proposed method are presented through experiments