A computer vision model for visual-object-based attention and eye movements

This is the post-print version of the final paper published in Computer Vision and Image Understanding. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2008 Elsevier B.V.This paper presents a new computational framework for modelling visual-object-based attention and attention-driven eye movements within an integrated system in a biologically inspired approach. Attention operates at multiple levels of visual selection by space, feature, object and group depending on the nature of targets and visual tasks. Attentional shifts and gaze shifts are constructed upon their common process circuits and control mechanisms but also separated from their different function roles, working together to fulfil flexible visual selection tasks in complicated visual environments. The framework integrates the important aspects of human visual attention and eye movements resulting in sophisticated performance in complicated natural scenes. The proposed approach aims at exploring a useful visual selection system for computer vision, especially for usage in cluttered natural visual environments.National Natural Science of Founda- tion of Chin

Attentive monitoring of multiple video streams driven by a Bayesian foraging strategy

In this paper we shall consider the problem of deploying attention to subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multi-stream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g. activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Dataset, a publicly available dataset, are presented to illustrate the utility of the proposed technique.Comment: Accepted to IEEE Transactions on Image Processin

The Role of Representations in Executive Function: Investigating a Developmental Link between Flexibility and Abstraction.

Young children often perseverate, engaging in previously correct, but no longer appropriate behaviors. One account posits that such perseveration results from the use of stimulus-specific representations of a situation, which are distinct from abstract, generalizable representations that support flexible behavior. Previous findings supported this account, demonstrating that only children who flexibly switch between rules could generalize their behavior to novel stimuli. However, this link between flexibility and generalization might reflect general cognitive abilities, or depend upon similarities across the measures or their temporal order. The current work examined these issues by testing the specificity and generality of this link. In two experiments with 3-year-old children, flexibility was measured in terms of switching between rules in a card-sorting task, while abstraction was measured in terms of selecting which stimulus did not belong in an odd-one-out task. The link between flexibility and abstraction was general across (1) abstraction dimensions similar to or different from those in the card-sorting task and (2) abstraction tasks that preceded or followed the switching task. Good performance on abstraction and flexibility measures did not extend to all cognitive tasks, including an IQ measure, and dissociated from children's ability to gaze at the correct stimulus in the odd-one-out task, suggesting that the link between flexibility and abstraction is specific to such measures, rather than reflecting general abilities that affect all tasks. We interpret these results in terms of the role that developing prefrontal cortical regions play in processes such as working memory, which can support both flexibility and abstraction

A Self-initializing Eyebrow Tracker for Binary Switch Emulation

We designed the Eyebrow-Clicker, a camera-based human computer interface system that implements a new form of binary switch. When the user raises his or her eyebrows, the binary switch is activated and a selection command is issued. The Eyebrow-Clicker thus replaces the "click" functionality of a mouse. The system initializes itself by detecting the user's eyes and eyebrows, tracks these features at frame rate, and recovers in the event of errors. The initialization uses the natural blinking of the human eye to select suitable templates for tracking. Once execution has begun, a user therefore never has to restart the program or even touch the computer. In our experiments with human-computer interaction software, the system successfully determined 93% of the time when a user raised his eyebrows.Office of Naval Research; National Science Foundation (IIS-0093367

Multi-Timescale Perceptual History Resolves Visual Ambiguity

When visual input is inconclusive, does previous experience aid the visual system in attaining an accurate perceptual interpretation? Prolonged viewing of a visually ambiguous stimulus causes perception to alternate between conflicting interpretations. When viewed intermittently, however, ambiguous stimuli tend to evoke the same percept on many consecutive presentations. This perceptual stabilization has been suggested to reflect persistence of the most recent percept throughout the blank that separates two presentations. Here we show that the memory trace that causes stabilization reflects not just the latest percept, but perception during a much longer period. That is, the choice between competing percepts at stimulus reappearance is determined by an elaborate history of prior perception. Specifically, we demonstrate a seconds-long influence of the latest percept, as well as a more persistent influence based on the relative proportion of dominance during a preceding period of at least one minute. In case short-term perceptual history and long-term perceptual history are opposed (because perception has recently switched after prolonged stabilization), the long-term influence recovers after the effect of the latest percept has worn off, indicating independence between time scales. We accommodate these results by adding two positive adaptation terms, one with a short time constant and one with a long time constant, to a standard model of perceptual switching

The simultaneous extraction of multiple social categories from unfamiliar faces

The research was supported by an award from the Experimental Psychology Society's Small Grant scheme.Peer reviewedPostprin