Low-cost Geometry-based Eye Gaze Detection using Facial Landmarks Generated through Deep Learning

Introduction: In the realm of human-computer interaction and behavioral research, accurate real-time gaze estimation is critical. Traditional methods often rely on expensive equipment or large datasets, which are impractical in many scenarios. This paper introduces a novel, geometry-based approach to address these challenges, utilizing consumer-grade hardware for broader applicability. Methods: We leverage novel face landmark detection neural networks capable of fast inference on consumer-grade chips to generate accurate and stable 3D landmarks of the face and iris. From these, we derive a small set of geometry-based descriptors, forming an 8-dimensional manifold representing the eye and head movements. These descriptors are then used to formulate linear equations for predicting eye-gaze direction. Results: Our approach demonstrates the ability to predict gaze with an angular error of less than 1.9 degrees, rivaling state-of-the-art systems while operating in real-time and requiring negligible computational resources. Conclusion: The developed method marks a significant step forward in gaze estimation technology, offering a highly accurate, efficient, and accessible alternative to traditional systems. It opens up new possibilities for real-time applications in diverse fields, from gaming to psychological research

iBall: Augmenting Basketball Videos with Gaze-moderated Embedded Visualizations

We present iBall, a basketball video-watching system that leverages gaze-moderated embedded visualizations to facilitate game understanding and engagement of casual fans. Video broadcasting and online video platforms make watching basketball games increasingly accessible. Yet, for new or casual fans, watching basketball videos is often confusing due to their limited basketball knowledge and the lack of accessible, on-demand information to resolve their confusion. To assist casual fans in watching basketball videos, we compared the game-watching behaviors of casual and die-hard fans in a formative study and developed iBall based on the fndings. iBall embeds visualizations into basketball videos using a computer vision pipeline, and automatically adapts the visualizations based on the game context and users' gaze, helping casual fans appreciate basketball games without being overwhelmed. We confrmed the usefulness, usability, and engagement of iBall in a study with 16 casual fans, and further collected feedback from 8 die-hard fans.Comment: ACM CHI2

Eye Tracking Support for Visual Analytics Systems

Visual analytics (VA) research provides helpful solutions for interactive visual data analysis when exploring large and complex datasets. Due to recent advances in eye tracking technology, promising opportunities arise to extend these traditional VA approaches. Therefore, we discuss foundations for eye tracking support in VA systems. We first review and discuss the structure and range of typical VA systems. Based on a widely used VA model, we present five comprehensive examples that cover a wide range of usage scenarios. Then, we demonstrate that the VA model can be used to systematically explore how concrete VA systems could be extended with eye tracking, to create supportive and adaptive analytics systems. This allows us to identify general research and application opportunities, and classify them into research themes. In a call for action, we map the road for future research to broaden the use of eye tracking and advance visual analytics

Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation

This paper proposes an approach to detect information relevance during decision-making from eye movements in order to enable user interface adaptation. This is a challenging task because gaze behavior varies greatly across individual users and tasks and groundtruth data is difficult to obtain. Thus, prior work has mostly focused on simpler target-search tasks or on establishing general interest, where gaze behavior is less complex. From the literature, we identify six metrics that capture different aspects of the gaze behavior during decision-making and combine them in a voting scheme. We empirically show, that this accounts for the large variations in gaze behavior and out-performs standalone metrics. Importantly, it offers an intuitive way to control the amount of detected information, which is crucial for different UI adaptation schemes to succeed. We show the applicability of our approach by developing a room-search application that changes the visual saliency of content detected as relevant. In an empirical study, we show that it detects up to 97% of relevant elements with respect to user self-reporting, which allows us to meaningfully adapt the interface, as confirmed by participants. Our approach is fast, does not need any explicit user input and can be applied independent of task and user.Comment: The first two authors contributed equally to this wor

EYE-AS-AN-INPUT FOR IMPROVING INTERACTIVE INFORMATION RETRIEVAL

In this work, Publication Access Through Tiered Interaction and Exploration (PATTIE) is presented with the eye as an additional input modality. PATTIE is built upon the scatter/gather information retrieval paradigm where users can explore a visual and interactive table-of-contents metaphor for large-scale document collections in an iterative manner. Additionally, the prototype has been integrated with eye-tracking through the web camera and experimental findings are provided to demonstrate a proof-of-concept for interest modeling at the term level and implicit relevance feedback on the gold standard inaugural 2019 Text REtrieval Conference Precision Medicine dataset (TREC PM). Low error rates for gaze tracking, and acceptable performance on binary classification of interest are reported as well as statistically significant increases in precision and recall performance for relevant information on a TREC PM task when PATTIE is used with eye-as-an-input versus a baseline PATTIE system.Doctor of Philosoph

Calibration-free gaze interfaces based on linear smooth pursuit

Since smooth pursuit eye movements can be used without calibration in spontaneous gaze interaction, the intuitiveness of the gaze interface design has been a topic of great interest in the human-computer interaction field. However, since most related research focuses on curved smooth-pursuit trajectories, the design issues of linear trajectories are poorly understood. Hence, this study evaluated the user performance of gaze interfaces based on linear smooth pursuit eye movements. We conducted an experiment to investigate how the number of objects (6, 8, 10, 12, or 15) and object moving speed (7.73 ˚/s vs. 12.89 ˚/s) affect the user performance in a gaze-based interface. Results show that the number and speed of the displayed objects influence users’ performance with the interface. The number of objects significantly affected the correct and false detection rates when selecting objects in the display. Participants’ performance was highest on interfaces containing 6 and 8 objects and decreased for interfaces with 10, 12, and 15 objects. Detection rates and orientation error were significantly influenced by the moving speed of displayed objects. Faster moving speed (12.89 ˚/s) resulted in higher detection rates and smaller orientation error compared to slower moving speeds (7.73 ˚/s). Our findings can help to enable a calibration-free accessible interaction with gaze interfaces.DFG, 414044773, Open Access Publizieren 2019 - 2020 / Technische Universität Berli

Users’ Cognitive Load: A Key Aspect to Successfully Communicate Visual Climate Information

The visual communication of climate information is one of the cornerstones of climate services. It often requires the translation of multidimensional data to visual channels by combining colors, distances, angles, and glyph sizes. However, visualizations including too many layers of complexity can hinder decision-making processes by limiting the cognitive capacity of users, therefore affecting their attention, recognition, and working memory. Methodologies grounded on the fields of user-centered design, user interaction, and cognitive psychology, which are based on the needs of the users, have a lot to contribute to the climate data visualization field. Here, we apply these methodologies to the redesign of an existing climate service tool tailored to the wind energy sector. We quantify the effect of the redesign on the users’ experience performing typical daily tasks, using both quantitative and qualitative indicators that include response time, success ratios, eye-tracking measures, user perceived effort, and comments, among others. Changes in the visual encoding of uncertainty and the use of interactive elements in the redesigned tool reduced the users’ response time by half, significantly improved success ratios, and eased decision-making by filtering nonrelevant information. Our results show that the application of user-centered design, interaction, and cognitive aspects to the design of climate information visualizations reduces the cognitive load of users during tasks performance, thus improving user experience. These aspects are key to successfully communicating climate information in a clearer and more accessible way, making it more understandable for both technical and nontechnical audiences.The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreements 776787 (S2S4E), 776613 (EUCP), and (ClimatEurope). This work was also supported by the MEDSCOPE project. MEDSCOPE is part of ERA4CS, an ERA-NET initiated by JPI Climate, and funded by AEMET (ES), ANR (FR), BSC (ES), CMCC (IT), CNR (IT), IMR (BE), and Météo-France (FR), with co-funding by the European Union (Grant 690462). The research team would like to thank the participants of the test who generously shared their time and opinions for the purposes of this research. This study is a part of the PhD of the corresponding author, Luz Calvo.Peer ReviewedPostprint (published version

Toward a Scalable Census of Dashboard Designs in the Wild: A Case Study with Tableau Public

Dashboards remain ubiquitous artifacts for presenting or reasoning with data across different domains. Yet, there has been little work that provides a quantifiable, systematic, and descriptive overview of dashboard designs at scale. We propose a schematic representation of dashboard designs as node-link graphs to better understand their spatial and interactive structures. We apply our approach to a dataset of 25,620 dashboards curated from Tableau Public to provide a descriptive overview of the core building blocks of dashboards in the wild and derive common dashboard design patterns. To guide future research, we make our dashboard corpus publicly available and discuss its application toward the development of dashboard design tools.Comment: *J. Purich and A. Srinivasan contributed equally to the wor

Survey on Individual Differences in Visualization

Developments in data visualization research have enabled visualization systems to achieve great general usability and application across a variety of domains. These advancements have improved not only people's understanding of data, but also the general understanding of people themselves, and how they interact with visualization systems. In particular, researchers have gradually come to recognize the deficiency of having one-size-fits-all visualization interfaces, as well as the significance of individual differences in the use of data visualization systems. Unfortunately, the absence of comprehensive surveys of the existing literature impedes the development of this research. In this paper, we review the research perspectives, as well as the personality traits and cognitive abilities, visualizations, tasks, and measures investigated in the existing literature. We aim to provide a detailed summary of existing scholarship, produce evidence-based reviews, and spur future inquiry