A computer vision model for visual-object-based attention and eye movements

This is the post-print version of the final paper published in Computer Vision and Image Understanding. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2008 Elsevier B.V.This paper presents a new computational framework for modelling visual-object-based attention and attention-driven eye movements within an integrated system in a biologically inspired approach. Attention operates at multiple levels of visual selection by space, feature, object and group depending on the nature of targets and visual tasks. Attentional shifts and gaze shifts are constructed upon their common process circuits and control mechanisms but also separated from their different function roles, working together to fulfil flexible visual selection tasks in complicated visual environments. The framework integrates the important aspects of human visual attention and eye movements resulting in sophisticated performance in complicated natural scenes. The proposed approach aims at exploring a useful visual selection system for computer vision, especially for usage in cluttered natural visual environments.National Natural Science of Founda- tion of Chin

Changes in visual and sensory-motor resting-state functional connectivity support motor learning by observing.

Motor learning occurs not only through direct first-hand experience but also through observation (Mattar AA, Gribble PL. Neuron 46: 153-160, 2005). When observing the actions of others, we activate many of the same brain regions involved in performing those actions ourselves (Malfait N, Valyear KF, Culham JC, Anton JL, Brown LE, Gribble PL. J Cogn Neurosci 22: 1493-1503, 2010). Links between neural systems for vision and action have been reported in neurophysiological (Strafella AP, Paus T. Neuroreport 11: 2289-2292, 2000; Watkins KE, Strafella AP, Paus T. Neuropsychologia 41: 989-994, 2003), brain imaging (Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ. Eur J Neurosci 13: 400-404, 2001; Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, Rizzolatti G. Science 286: 2526-2528, 1999), and eye tracking (Flanagan JR, Johansson RS. Nature 424: 769-771, 2003) studies. Here we used a force field learning paradigm coupled with resting-state fMRI to investigate the brain areas involved in motor learning by observing. We examined changes in resting-state functional connectivity (FC) after an observational learning task and found a network consisting of V5/MT, cerebellum, and primary motor and somatosensory cortices in which changes in FC were correlated with the amount of motor learning achieved through observation, as assessed behaviorally after resting-state fMRI scans. The observed FC changes in this network are not due to visual attention to motion or observation of movement errors but rather are specifically linked to motor learning. These results support the idea that brain networks linking action observation and motor control also facilitate motor learning

Top-down modulation of the perception of other people in schizophrenia and autism

Accurately and efficiently perceiving social cues such as body movements and facial expressions is important in social interaction. Accurate social perception of this kind does not solely rely on “bottom-up” visual processing but is also subject to modulation by “top-down” signals. For example, if instructed to look for signs of happiness rather than fear, participants are more likely to categorize facial expressions as happy—this prior expectation biases subsequent perception. Top-down modulation is also important in our reactions to others. For example, top-down control over imitation plays an important role in the development of smooth and harmonious social interactions. This paper highlights the importance of top-down modulation in our perception of, and reactions to, others. We discuss evidence that top-down modulation of social perception and imitation is atypical in Autism Spectrum Conditions and in schizophrenia, and we consider the effect this may have on the development of social interactions for individuals with these developmental disorders

Predicting visual fixations on video based on low-level visual features

AbstractTo what extent can a computational model of the bottom–up visual attention predict what an observer is looking at? What is the contribution of the low-level visual features in the attention deployment? To answer these questions, a new spatio-temporal computational model is proposed. This model incorporates several visual features; therefore, a fusion algorithm is required to combine the different saliency maps (achromatic, chromatic and temporal). To quantitatively assess the model performances, eye movements were recorded while naive observers viewed natural dynamic scenes. Four completing metrics have been used. In addition, predictions from the proposed model are compared to the predictions from a state of the art model [Itti’s model (Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259)] and from three non-biologically plausible models (uniform, flicker and centered models). Regardless of the metric used, the proposed model shows significant improvement over the selected benchmarking models (except the centered model). Conclusions are drawn regarding both the influence of low-level visual features over time and the central bias in an eye tracking experiment

Biased Competition in Visual Processing Hierarchies: A Learning Approach Using Multiple Cues

In this contribution, we present a large-scale hierarchical system for object detection fusing bottom-up (signal-driven) processing results with top-down (model or task-driven) attentional modulation. Specifically, we focus on the question of how the autonomous learning of invariant models can be embedded into a performing system and how such models can be used to define object-specific attentional modulation signals. Our system implements bi-directional data flow in a processing hierarchy. The bottom-up data flow proceeds from a preprocessing level to the hypothesis level where object hypotheses created by exhaustive object detection algorithms are represented in a roughly retinotopic way. A competitive selection mechanism is used to determine the most confident hypotheses, which are used on the system level to train multimodal models that link object identity to invariant hypothesis properties. The top-down data flow originates at the system level, where the trained multimodal models are used to obtain space- and feature-based attentional modulation signals, providing biases for the competitive selection process at the hypothesis level. This results in object-specific hypothesis facilitation/suppression in certain image regions which we show to be applicable to different object detection mechanisms. In order to demonstrate the benefits of this approach, we apply the system to the detection of cars in a variety of challenging traffic videos. Evaluating our approach on a publicly available dataset containing approximately 3,500 annotated video images from more than 1 h of driving, we can show strong increases in performance and generalization when compared to object detection in isolation. Furthermore, we compare our results to a late hypothesis rejection approach, showing that early coupling of top-down and bottom-up information is a favorable approach especially when processing resources are constrained

An Attention Based Method For Motion Detection And Estimation

The demand for automated motion detection and object tracking systems has promoted considerable research activity in the field of computer vision. A novel approach to motion detection and estimation based on visual attention is proposed in the paper. Two different thresholding techniques are applied and comparisons are made with Black's motion estimation technique based on the measure of overall derived tracking angle. The method is illustrated on various video data and results show that the new method can extract both motion and shape information

UNA NUEVA METRICA PARA UTILIZAR FILTROS DE CORRELACION EN EL RECONOCIMIENTO DE OBJETOS CON EL ROBOT HUMANOIDE NAO (A NOVEL METRIC TO USE CORRELATION FILTERS IN OBJECT RECOGNITION WITH THE NAO HUMANOID ROBOT)

ResumenEn este trabajo, se propone una nueva métrica para el uso del filtro de función discriminante sintética (Synthetic Discriminant Function, SDF por sus siglas en inglés) en el problema de reconocimiento de objetos. Se realiza una serie de experimentos con el filtro SDF en la plataforma de programación del robot humanoide NAO, que permiten observar un comportamiento de la nueva métrica (Peak to Neighboring Values, PNV por sus siglas en inglés) y predecir comportamientos futuros en situaciones similares. Con los experimentos realizados se concluye que la métrica PNV mejora notablemente la medición del desempeño del filtro, generando mejores resultados que las métricas convencionales, específicamente en los objetos que tienen variaciones en su apariencia, como cambios de escala y de rotación. Calificaciones altas en el desempeño brindan una mayor seguridad para determinar que el objeto ha sido reconocido.Palabras Claves: Peak to Neighboring Values, Reconocimiento de objetos, filtros de correlación, robot NAO. AbstractIn this paper, a new metric is proposed for the use of the Synthetic Discriminant Function (SDF) in the problem of object recognition. A series of experiments are carried out with the SDF filter in the programming platform of the NAO humanoid robot, which allow observing a behavior of the new metric (Peak to Neighboring Values, PNV) and predicting future behaviors in similar situations. With the experiments carried out, it is concluded that the PNV metric significantly improves the measurement of the filter's performance, generating better results than conventional metrics, specifically on objects that have variations in their appearance, such as changes in scale and rotation. High performance ratings provide greater security to determine that the object has been recognized.Keywords: Peak to Neighboring Values, Object Recognition, correlation filters, NAO robot