A computer vision model for visual-object-based attention and eye movements

This is the post-print version of the final paper published in Computer Vision and Image Understanding. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2008 Elsevier B.V.This paper presents a new computational framework for modelling visual-object-based attention and attention-driven eye movements within an integrated system in a biologically inspired approach. Attention operates at multiple levels of visual selection by space, feature, object and group depending on the nature of targets and visual tasks. Attentional shifts and gaze shifts are constructed upon their common process circuits and control mechanisms but also separated from their different function roles, working together to fulfil flexible visual selection tasks in complicated visual environments. The framework integrates the important aspects of human visual attention and eye movements resulting in sophisticated performance in complicated natural scenes. The proposed approach aims at exploring a useful visual selection system for computer vision, especially for usage in cluttered natural visual environments.National Natural Science of Founda- tion of Chin

Attentive processing improves object recognition

The human visual system can recognize several thousand object categories irrespective of their position and size. This combination of selectivity and invariance is built up gradually across several stages of visual processing. However, the recognition of multiple objects in cluttered visual scenes presents a difficult problem for human as well as machine vision systems. The human visual system has evolved to perform two stages of visual processing: a pre-attentive parallel processing stage, in which the entire visual field is processed at once and a slow serial attentive processing stage, in which aregion of interest in an input image is selected for "specialized" analysis by an attentional spotlight. We argue that this strategy evolved to overcome the limitation of purely feed forward processing in the presence of clutter and crowding. Using a Bayesian model of attention along with a hierarchical model of feed forward recognition on a data set of real world images, we show that this two stage attentive processing can improve recognition in cluttered and crowded conditions

Finding any Waldo: zero-shot invariant and efficient visual search

Searching for a target object in a cluttered scene constitutes a fundamental challenge in daily vision. Visual search must be selective enough to discriminate the target from distractors, invariant to changes in the appearance of the target, efficient to avoid exhaustive exploration of the image, and must generalize to locate novel target objects with zero-shot training. Previous work has focused on searching for perfect matches of a target after extensive category-specific training. Here we show for the first time that humans can efficiently and invariantly search for natural objects in complex scenes. To gain insight into the mechanisms that guide visual search, we propose a biologically inspired computational model that can locate targets without exhaustive sampling and generalize to novel objects. The model provides an approximation to the mechanisms integrating bottom-up and top-down signals during search in natural scenes.Comment: Number of figures: 6 Number of supplementary figures: 1

A role of the claustrum in auditory scene analysis by reflecting sensory change

The biological function of the claustrum remains speculative, despite many years of research. On the basis of its widespread connections it is often hypothesized that the claustrum may have an integrative function mainly reflecting objects rather than the details of sensory stimuli. Given the absence of a clear demonstration of any sensory integration in claustral neurons, however, we propose an alternative, data-driven, hypothesis: namely that the claustrum detects the occurrence of novel or salient sensory events. The detection of new events is critical for behavior and survival, as suddenly appearing objects may require rapid and coordinated reactions. Sounds are of particular relevance in this regard, and our conclusions are based on the analysis of neurons in the auditory zone of the primate claustrum. Specifically, we studied the responses to natural sounds, their preference to various sound categories, and to changes in the auditory scene. In a test for sound-category preference claustral neurons responded to but displayed a clear lack of selectivity between monkey vocalizations, other animal vocalizations or environmental sounds (Esnd). Claustral neurons were however able to detect target sounds embedded in a noisy background and their responses scaled with target signal to noise ratio (SNR). The single trial responses of individual neurons suggest that these neurons detected and reflected the occurrence of a change in the auditory scene. Given its widespread connectivity with sensory, motor and limbic structures the claustrum could play the essential role of identifying the occurrence of important sensory changes and notifying other brain areas—hence contributing to sensory awareness