Real-time computational attention model for dynamic scenes analysis: from implementation to evaluation.

International audienceProviding real time analysis of the huge amount of data generated by computer vision algorithms in interactive applications is still an open problem. It promises great advances across a wide variety of elds. When using dynamics scene analysis algorithms for computer vision, a trade-o must be found between the quality of the results expected, and the amount of computer resources allocated for each task. It is usually a design time decision, implemented through the choice of pre-de ned algorithms and parameters. However, this way of doing limits the generality of the system. Using an adaptive vision system provides a more exible solution as its analysis strategy can be changed according to the new information available. As a consequence, such a system requires some kind of guiding mechanism to explore the scene faster and more e ciently. We propose a visual attention system that it adapts its processing according to the interest (or salience) of each element of the dynamic scene. Somewhere in between hierarchical salience based and competitive distributed, we propose a hierarchical yet competitive and non salience based model. Our original approach allows the generation of attentional focus points without the need of neither saliency map nor explicit inhibition of return mechanism. This new real- time computational model is based on a preys / predators system. The use of this kind of dynamical system is justi ed by an adjustable trade-o between nondeterministic attentional behavior and properties of stability, reproducibility and reactiveness

VIRTUAL INTERACTIONS: CAN EEG HELP MAKE THE DIFFERENCE WITH REAL INTERACTION?

International audienceScience and technology progress fast, but mouse and keyboard are still used to control multimedia devices. One of the limiting factors of gesture based HCIs adoption is the detection of the user's intention to interact. This study tries to make a step in that direction with use of consumer EEG sensor headset. EEG headset records in real-time data that can help to identify intention of the user based on his emotional state. For each subject EEG responses for different stimuli are recorded. Acquiring these data allows to determine the potential of EEG based intention detection. The findings are promising and with proper implementation should allow to building a new type of HCI devices

LLM-based Interaction for Content Generation: A Case Study on the Perception of Employees in an IT department

In the past years, AI has seen many advances in the field of NLP. This has led to the emergence of LLMs, such as the now famous GPT-3.5, which revolutionise the way humans can access or generate content. Current studies on LLM-based generative tools are mainly interested in the performance of such tools in generating relevant content (code, text or image). However, ethical concerns related to the design and use of generative tools seem to be growing, impacting the public acceptability for specific tasks. This paper presents a questionnaire survey to identify the intention to use generative tools by employees of an IT company in the context of their work. This survey is based on empirical models measuring intention to use (TAM by Davis, 1989, and UTAUT2 by Venkatesh and al., 2008). Our results indicate a rather average acceptability of generative tools, although the more useful the tool is perceived to be, the higher the intention to use seems to be. Furthermore, our analyses suggest that the frequency of use of generative tools is likely to be a key factor in understanding how employees perceive these tools in the context of their work. Following on from this work, we plan to investigate the nature of the requests that may be made to these tools by specific audiences.Comment: 14 pages (bibliography inclued), 6 figures, preprint submitted to Work-In-Progress session of ACM IMX'23 Interactive Media Experienc

Study of depth bias of observers in free viewing of still stereoscopic synthetic stimuli

Observers’ fixations exhibit a marked bias towards certain areas on the screen when viewing scenes on computer monitors. For instance, there exists a well-known “center-bias” which means that fixations are biased towards the center of the screen during the viewing of 2D still images. In the viewing of 3D content, stereoscopic displays enhance depth perception by the mean of binocular parallax. This additional depth cue has a great influence on guiding eye movements. Relatively little is known about the impact of binocular parallax on visual attention of the 3D content displayed on stereoscopic screen. Several studies mentioned that people tend to look preferably at the objects located at certain positions in depth. But studies proving or quantifying this depth-bias are still limited. In this paper, we conducted a binocular eye-tracking experiment by showing synthetic stimuli on a stereoscopic display. Observers were required to do a free-viewing task through passive polarized glasses. Gaze positions of both eyes were recorded and the depth of eyes’ fixation was determined. The stimuli used in the experiment were designed in such a way that the center-bias and the depth-bias affect eye movements individually. Results indicate the existence of a depth-bias: objects closer to the viewer attract attention earlier than distant objects, and the number of fixations located on objects varies as a function of objects’ depth. The closest object in a scene always attracts most fixations. The fixation distribution along depth also shows a convergent behavior as the viewing time increases

Exploring the Sensed and Unexpected:Not Looking in Gaze Interaction

Gaze interaction paradigms rely on the user needing to look at objects in the interface to select them or trigger actions. ”Not looking” is an atypical and unexpected interaction to perform, but the eye-tracker can sense it. We illustrate the use of ”not looking” as an interaction dynamic with examples of gaze-enabled games. We created a framework containing a spectrum of five discrete categories for this unexpected use of gaze sensing. For each category, we analyse games that use gaze interaction and make the user look away from the game action up to the extent they close their eyes. The framework is described based on whether specific game events mean the player might not; cannot; should not; must not; or does not look. Finally, we discuss the outcomes of using unexpected gaze interactions and the potential of the proposed framework as a new approach to guide the design of sensing-based interfaces

Computational attention model for adaptive vision

L'analyse temps réel de la masse de données générée par les mécanismes de gestion de la vision dans les applications interactives est un problème toujours ouvert, promettant des avancées importantes dans des domaines aussi variés que la robotique, l’apprentissage à distance ou les nouvelles formes d’interactions avec l’utilisateur, sans clavier ni souris. Dans le cadre général de la vision, les algorithmes d’analyse de scène doivent trouver un compromis entre d'une part la qualité des résultats recherchés et d'autre part la quantité de ressources allouable aux différents tâches. Classiquement, ce choix est effectué à la conception du système (sous la forme de paramètres et d’algorithmes prédéfinis), mais cette solution limite le champ d’application de celui-ci. Une solution plus flexible consiste à utiliser un système de vision adaptatif qui pourra modifier sa stratégie d’analyse en fonction des informations disponibles concernant son contexte d’exécution. En conséquence, ce système doit posséder un mécanisme permettant de guider rapidement et efficacement l’exploration de la scène afin d’obtenir ces informations. Chez l’homme, les mécanismes de l’évolution ont mis en place le système d’attention visuelle. Ce système sélectionne les informations importantes afin de réduire la charge cognitive et les ambiguïtés d’interprétation de la scène. Nous proposons, dans cette thèse, un système d'attention visuelle, dont nous définissons l’architecture et les principes de fonctionnement. Ce dernier devra permettre l’interaction avec un système de vision afin qu’il adapte ses traitements en fonction de l’intérêt de chacun des éléments de la scène, i.e. ce que nous appelons saillance. A la croisée des chemins entre les modèles centralisés et hiérarchiques (ex : [Koch1985], puis [Itti1998]), et les modèles distribués et compétitifs (ex : [Desimone1995], puis [Deco2004, Rolls2006]), nous proposons un modèle hiérarchique, compétitif et non centralisé. Cette approche originale permet de générer un point de focalisation attentionnel à chaque pas de temps sans utiliser de carte de saillance ni de mécanisme explicite d’inhibition de retour. Ce nouveau modèle computationnel d'attention visuelle temps réel est basé sur un système d'équations proies / prédateurs, qui est bien adapté pour l'arbitrage entre un comportement attentionnel non déterministe et des propriétés de stabilité, reproductibilité, et réactivité. L'analyse des expérimentations menées est positive : malgré le comportement non-déterministe des équations proies / prédateurs, ce système possède des propriétés intéressantes de stabilité, reproductibilité, et réactivité, tout en permettant une exploration rapide et efficace de la scène. Ces propriétés ouvrent la possibilité d’aborder différents types d’applications allant de l’évaluation de la complexité d’images et de vidéos à la détection et au suivi d’objets. Enfin, bien qu’il soit destiné à la vision par ordinateur, nous comparons notre modèle au système attentionnel humain et montrons que celui-ci présente un comportement aussi plausible (voire plus en fonction du comportement défini) que les modèles classiques existants.Providing real time analysis of the huge amount of data generated by computer vision algorithms in interactive applications is still an open problem. It promises great advances across a wide variety of fields : robotics, distance education, or new mouse-less and keyboard-less human computer interaction.When using scene analysis algorithms for computer vision, a trade-off must be found between the quality of the results expected, and the amount of computer resources allocated for each task. It is usually a design time decision, implemented through the choice of pre-defined algorithms and parameters. However, this way of doing limits the generality of the system. Using an adaptive vision system provides a more flexible solution as its analysis strategy can be changed according to the information available concerning the execution context. As a consequence, such a system requires some kind of guiding mechanism to explore the scene faster and more efficiently.In human, the mechanisms of evolution have generated the visual attention system which selects the most important information in order to reduce both cognitive load and scene understanding ambiguity.In this thesis, we propose a visual attention system tailored for interacting with a vision system (whose theoretical architecture is given) so that it adapts its processing according to the interest (or salience) of each element of the scene.Somewhere in between hierarchical salience based (ex: [Koch1985], then [Itti1998]) and competitive distributed (ex: [Desimone1995], then [Deco2004, Rolls2006]) models, we propose a hierarchical yet competitive and non salience based model. Our original approach allows the generation of attentional focus points without the need of neither saliency map nor explicit inhibition of return mechanism. This new real-time computational model is based on a preys / predators system. The use of this kind of dynamical system is justified by an adjustable trade-off between nondeterministic attentional behavior and properties of stability, reproducibility and reactiveness.Our experiments shows that despite the non deterministic behavior of preys / predators equations, the system exhibits interesting properties of stability, reproducibility and reactiveness while allowing a fast and efficient exploration of the scene. These properties are useful for addressing different kinds of applications, ranging from image complexity evaluation, to object detection and tracking. Finally, while it is designed for computer vision, we compare our model to human visual attention. We show that it is equally as plausible as existing models (or better, depending on its configuration)

Modèle computationnel d'attention pour la vision adaptative

Providing real time analysis of the huge amount of data generated by computer vision algorithms in interactive applications is still an open problem. It promises great advances across a wide variety of fields : robotics, distance education, or new mouse-less and keyboard-less human computer interaction.When using scene analysis algorithms for computer vision, a trade-off must be found between the quality of the results expected, and the amount of computer resources allocated for each task. It is usually a design time decision, implemented through the choice of pre-defined algorithms and parameters. However, this way of doing limits the generality of the system. Using an adaptive vision system provides a more flexible solution as its analysis strategy can be changed according to the information available concerning the execution context. As a consequence, such a system requires some kind of guiding mechanism to explore the scene faster and more efficiently.In human, the mechanisms of evolution have generated the visual attention system which selects the most important information in order to reduce both cognitive load and scene understanding ambiguity.In this thesis, we propose a visual attention system tailored for interacting with a vision system (whose theoretical architecture is given) so that it adapts its processing according to the interest (or salience) of each element of the scene.Somewhere in between hierarchical salience based (ex: [Koch1985], then [Itti1998]) and competitive distributed (ex: [Desimone1995], then [Deco2004, Rolls2006]) models, we propose a hierarchical yet competitive and non salience based model. Our original approach allows the generation of attentional focus points without the need of neither saliency map nor explicit inhibition of return mechanism. This new real-time computational model is based on a preys / predators system. The use of this kind of dynamical system is justified by an adjustable trade-off between nondeterministic attentional behavior and properties of stability, reproducibility and reactiveness.Our experiments shows that despite the non deterministic behavior of preys / predators equations, the system exhibits interesting properties of stability, reproducibility and reactiveness while allowing a fast and efficient exploration of the scene. These properties are useful for addressing different kinds of applications, ranging from image complexity evaluation, to object detection and tracking. Finally, while it is designed for computer vision, we compare our model to human visual attention. We show that it is equally as plausible as existing models (or better, depending on its configuration).L'analyse temps réel de la masse de données générée par les mécanismes de gestion de la vision dans les applications interactives est un problème toujours ouvert, promettant des avancées importantes dans des domaines aussi variés que la robotique, l’apprentissage à distance ou les nouvelles formes d’interactions avec l’utilisateur, sans clavier ni souris. Dans le cadre général de la vision, les algorithmes d’analyse de scène doivent trouver un compromis entre d'une part la qualité des résultats recherchés et d'autre part la quantité de ressources allouable aux différents tâches. Classiquement, ce choix est effectué à la conception du système (sous la forme de paramètres et d’algorithmes prédéfinis), mais cette solution limite le champ d’application de celui-ci. Une solution plus flexible consiste à utiliser un système de vision adaptatif qui pourra modifier sa stratégie d’analyse en fonction des informations disponibles concernant son contexte d’exécution. En conséquence, ce système doit posséder un mécanisme permettant de guider rapidement et efficacement l’exploration de la scène afin d’obtenir ces informations. Chez l’homme, les mécanismes de l’évolution ont mis en place le système d’attention visuelle. Ce système sélectionne les informations importantes afin de réduire la charge cognitive et les ambiguïtés d’interprétation de la scène. Nous proposons, dans cette thèse, un système d'attention visuelle, dont nous définissons l’architecture et les principes de fonctionnement. Ce dernier devra permettre l’interaction avec un système de vision afin qu’il adapte ses traitements en fonction de l’intérêt de chacun des éléments de la scène, i.e. ce que nous appelons saillance. A la croisée des chemins entre les modèles centralisés et hiérarchiques (ex : [Koch1985], puis [Itti1998]), et les modèles distribués et compétitifs (ex : [Desimone1995], puis [Deco2004, Rolls2006]), nous proposons un modèle hiérarchique, compétitif et non centralisé. Cette approche originale permet de générer un point de focalisation attentionnel à chaque pas de temps sans utiliser de carte de saillance ni de mécanisme explicite d’inhibition de retour. Ce nouveau modèle computationnel d'attention visuelle temps réel est basé sur un système d'équations proies / prédateurs, qui est bien adapté pour l'arbitrage entre un comportement attentionnel non déterministe et des propriétés de stabilité, reproductibilité, et réactivité. L'analyse des expérimentations menées est positive : malgré le comportement non-déterministe des équations proies / prédateurs, ce système possède des propriétés intéressantes de stabilité, reproductibilité, et réactivité, tout en permettant une exploration rapide et efficace de la scène. Ces propriétés ouvrent la possibilité d’aborder différents types d’applications allant de l’évaluation de la complexité d’images et de vidéos à la détection et au suivi d’objets. Enfin, bien qu’il soit destiné à la vision par ordinateur, nous comparons notre modèle au système attentionnel humain et montrons que celui-ci présente un comportement aussi plausible (voire plus en fonction du comportement défini) que les modèles classiques existants

Implémentation et évaluation d’un modèle d’attention pour la vision adaptative

Dans le domaine de l'analyse de scène en vision par ordinateur, un compromis doit être trouvé entre la qualité des résultats attendus et les ressources allouées pour effectuer les traitements. Une solution flexible consiste à utiliser un système de vision adaptatif capable de moduler sa stratégie d'analyse en fonction de l'information disponible et du contexte. Dans cet article, nous décrivons comment concevoir et évaluer un système d'attention visuelle conçu pour interagir avec un système de vision de façon à ce que ce dernier adapte ses traitements en fonction de l'intérêt (de la saillance) de chaque élément de la scène. Nous proposons également un nouvel ensemble de contraintes nommé PAIRED, permettant d'évaluer l'adéquation du modèle à différentes applications. Nous justifions le choix des systèmes dynamiques par leurs propriétés intéressantes pour simuler la compétition entre différentes sources d'informations. Nous présentons enfin une validation à travers différentes métriques montrant que nos résultats sont rapides, hautement configurables et pertinents.In the field of scene analysis for computer vision, a trade-off must be found between the quality of the results expected, and the amount of computer resources allocated for each task. Using an adaptive vision system provides a more flexible solution as its analysis strategy can be changed according to the information available concerning the execution context. We describe how to create and evaluate a visual attention system tailored for interacting with a computer vision system so that it adapts its processing according to the interest (or salience) of each element of the scene. We propose a new set of constraints named PAIRED to evaluate the adequacy of a model with respect to its different applications. We justify why dynamical systems provide good properties for simulating the dynamic competition between different kinds of information. We present different results that demonstrate that our results are fast and highly configurable and plausible

Influence of Emotions on Eye Behavior in Omnidirectional Content

International audienceThe recent development of Virtual Reality (VR) technologies and interactive visual content faces some important challenges in multimedia processing and Quality of Experience (QoE). Considering omnidirectional, also called 360-degree, content , one major research topic is the development of reliable visual attention models. In this paper, we present a new dataset to study the influence of emotions on eye behavior in omnidirec-tional content. This dataset is based on an eye-tracking experiment where 19 observers have assessed emotional valence and arousal dimensions in 360-degree images. Several analyses are then conducted to compare fixation and saccade features, inter-observer saliency congruency and spatial oculomotor biases in positive, neutral and negative content. Results show a significant impact of negative images on visual attention, with more visual agitation and avoidance behavior from larger, longer and faster saccades. However, no obvious difference of eye behavior is found between positive and neutral stimulis on this dataset