Predicting visual fixations on video based on low-level visual features

AbstractTo what extent can a computational model of the bottom–up visual attention predict what an observer is looking at? What is the contribution of the low-level visual features in the attention deployment? To answer these questions, a new spatio-temporal computational model is proposed. This model incorporates several visual features; therefore, a fusion algorithm is required to combine the different saliency maps (achromatic, chromatic and temporal). To quantitatively assess the model performances, eye movements were recorded while naive observers viewed natural dynamic scenes. Four completing metrics have been used. In addition, predictions from the proposed model are compared to the predictions from a state of the art model [Itti’s model (Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259)] and from three non-biologically plausible models (uniform, flicker and centered models). Regardless of the metric used, the proposed model shows significant improvement over the selected benchmarking models (except the centered model). Conclusions are drawn regarding both the influence of low-level visual features over time and the central bias in an eye tracking experiment

Construction d'images miniatures avec recadrage automatique basée sur un modèle perceptuel bio-inspiré

Cet article présente un procédé de zoom automatique, destiné à adapter la taille des images pour des dispositifs d'affichage à écran de petite taille (Téléphone mobile...). L'adaptation de la taille des images s'effectue par la sélection des zones les plus intéressantes visuellement. Ces dernières sont déterminées via une approche computationnelle de modélisation de l'attention visuelle. Tout d'abord, les performances de modélisation de l'attention visuelle sont d´eduites par comparaison avec une v´erit´e terrain issue de tests oculom´etrique. Enfin, les performances qualitatives du syst`eme de vision complet, mod`ele d'attention visuelle et proc´ed´e de miniaturisation, sont pr´esent´ees

Light angular momentum of a plane wave diffracted by a two-dimensional simple object.

International audienceThe question of the transfer of angular momentum from light to objects is a key issue in optical tweezers and thus raises issues in physics, chemistry, and biology. Most of the attempts have been performed either with spin angular momentum using light's polarization or using orbital angular momentum with twisted beams. We experimentally demonstrate here the exchange of angular momentum from a linearly polarized plane wave to a two dimensional asymmetric object. More precisely, we show that the diffracted light from a snail-shaped mask carries local orbital angular momentum whose topological charge depends on the symmetry of the object. This opens the way to a means to produce micrometer-sized twisted beams from a high power laser or even pulsed beams

Modeling visual attention on scenes

International audienceLa modélisation computationelle de l'attention visuelle connaît actuellement un essor considérable. Les premières modèles, purement basés sur l'attention dite exogène, permettent de calculer une carte de saillance indiquant les zones d'intérêt visuel d'une image. Cependant, afin d'améliorer cette prédiction, il s'avère nécessaire de prendre en compte des informations de plus haut niveaux relatives à l'attention endogène, c'est à dire des informations liées aux processus cognitifs. Afin de rendre compte de cette problématique, le présent article décrit un certain nombre de modèles exogènes ainsi que des modèles intégrant de la connaissance a priori. Les méthodes d'évaluation des performances sont également décrites. Afin d'aller plus loin dans la modélisation et dans la compréhension des processus cognitifs, de nouvelles perspectives et direction d'études sont exposées

Which Semi-Local Visual Masking Model For Wavelet Based Image Quality Metric?

International audienceProperties and models of the Human Visual System (HVS) are the fundaments for most of efficient objective image or video quality metrics. Among HVS properties, visual masking is a sensitive issue. Many models exist in literature. Simplest models can only predict visibility threshold for very simple cue while for natural images one should consider more complex approaches such as semi-local masking. Our previous work has shown the positive impact of incorporating semi-local masking in image quality metric according to one subjective study. It is important to consolidate this work with different subjective experiments. In this paper, different visual masking models, including contrast masking and semi-local masking, are evaluated according to three subjective studies. These subjective experiments were conducted with different protocols, different types of display devices, different contents and different populations

On The Performance of Human Visual System Based Image Quality Assessment Metric Using Wavelet Domain

International audienceMost of the efficient objective image or video quality metrics are based on properties and models of the Human Visual System (HVS). This paper is dealing with two major drawbacks related to HVS properties used in such metrics applied in the DWT domain : subband decomposition and masking effect. The multi-channel behavior of the HVS can be emulated applying a perceptual subband decomposition. Ideally, this can be performed in the Fourier domain but it requires too much computation cost for many applications. Spatial transform such as DWT is a good alternative to reduce computation effort but the correspondence between the perceptual subbands and the usual wavelet ones is not straightforward. Advantages and limitations of the DWT are discussed, and compared with models based on a DFT. Visual masking is a sensitive issue. Several models exist in literature. Simplest models can only predict visibility threshold for very simple cue while for natural images one should consider more complex approaches such as entropy masking. The main issue relies on finding a revealing measure of the surround influences and an adaptation: should we use the spatial activity, the entropy, the type of texture, etc.? In this paper, different visual masking models using DWT are discussed and compared

How saccadic models help predict where we look during a visual task? Application to visual quality assessment

International audienceIn this paper, we present saccadic models which are an alternative way to predict where observers look at. Compared to saliency models, saccadic models generate plausible visual scan-paths from which saliency maps can be computed. In addition these models have the advantage of being adaptable to different viewing conditions, viewing tasks and types of visual scene. We demonstrate that saccadic models perform better than existing saliency models for predicting where an observer looks at in free-viewing condition and quality-task condition (i.e. when observers have to score the quality of an image). For that, the joint distributions of saccade amplitudes and orientations in both conditions (i.e. free-viewing and quality task) have been estimated from eye tracking data. Thanks to saccadic models, we hope we will be able to improve upon the performance of saliency-based quality metrics, and more generally the capacity to predict where we look within visual scenes when performing visual tasks

Memorability of natural scenes: the role of attention

International audienceThe image memorability consists in the faculty of an image to be recalled after a period of time. Recently, the memorability of an image database was measured and some factors responsible for this memorability were highlighted. In this paper, we investigate the role of visual attention in image memorability around two axis. The first one is experimental and uses results of eye-tracking performed on a set of images of different memorability scores. The second investigation axis is predictive and we show that attention-related features can advantageously replace low-level features in image memorability prediction. From our work it appears that the role of visual attention is important and should be more taken into account along with other low-level features

How to measure the relevance of a retargeting approach?

International audienceMost cell phones today can receive and display video content. Nonetheless, we are still significantly behind the point where premium made for mobile content is mainstream, largely available, and affordable. Significant issues must be overcome. The small screen size is one of them. Indeed, the direct transfer of conventional contents (not specifically shot for mobile devices) will provide a video in which the main characters or objects of interest may become indistinguishable from the rest of the scene. Therefore, it is required to retarget the content. Different solutions exist, either based on distortion of the image, on removal of redundant areas, or cropping. The most efficient ones are based on dynamic adaptation of the cropping window. They significantly improve the viewing experience by zooming in the regions of interest. Currently, there is no common agreement on how to compare different solutions. A retargeting metric is proposed in order to gauge its quality. Eye-tracking experiments, zooming effect through coverage ratio and temporal consistency are introduced and discussed