Eye-tracking as a measure of cognitive effort for post-editing of machine translation

The three measurements for post-editing effort as proposed by Krings (2001) have been adopted by many researchers in subsequent studies and publications. These measurements comprise temporal effort (the speed or productivity rate of post-editing, often measured in words per second or per minute at the segment level), technical effort (the number of actual edits performed by the post-editor, sometimes approximated using the Translation Edit Rate metric (Snover et al. 2006), again usually at the segment level), and cognitive effort. Cognitive effort has been measured using Think-Aloud Protocols, pause measurement, and, increasingly, eye-tracking. This chapter provides a review of studies of post-editing effort using eye-tracking, noting the influence of publications by Danks et al. (1997), and O’Brien (2006, 2008), before describing a single study in detail. The detailed study examines whether predicted effort indicators affect post-editing effort and results were previously published as Moorkens et al. (2015). Most of the eye-tracking data analysed were unused in the previou

Rethinking Eye-blink: Assessing Task Difficulty through Physiological Representation of Spontaneous Blinking

Continuous assessment of task difficulty and mental workload is essential in improving the usability and accessibility of interactive systems. Eye tracking data has often been investigated to achieve this ability, with reports on the limited role of standard blink metrics. Here, we propose a new approach to the analysis of eye-blink responses for automated estimation of task difficulty. The core module is a time-frequency representation of eye-blink, which aims to capture the richness of information reflected on blinking. In our first study, we show that this method significantly improves the sensitivity to task difficulty. We then demonstrate how to form a framework where the represented patterns are analyzed with multi-dimensional Long Short-Term Memory recurrent neural networks for their non-linear mapping onto difficulty-related parameters. This framework outperformed other methods that used hand-engineered features. This approach works with any built-in camera, without requiring specialized devices. We conclude by discussing how Rethinking Eye-blink can benefit real-world applications

Rethinking Eye-blink: Assessing Task Difficulty through Physiological Representation of Spontaneous Blinking

Continuous assessment of task difficulty and mental workload is essential in improving the usability and accessibility of interactive systems. Eye tracking data has often been investigated to achieve this ability, with reports on the limited role of standard blink metrics. Here, we propose a new approach to the analysis of eye-blink responses for automated estimation of task difficulty. The core module is a time-frequency representation of eye-blink, which aims to capture the richness of information reflected on blinking. In our first study, we show that this method significantly improves the sensitivity to task difficulty. We then demonstrate how to form a framework where the represented patterns are analyzed with multi-dimensional Long Short-Term Memory recurrent neural networks for their non-linear mapping onto difficulty-related parameters. This framework outperformed other methods that used hand-engineered features. This approach works with any built-in camera, without requiring specialized devices. We conclude by discussing how Rethinking Eye-blink can benefit real-world applications.Comment: [Accepted version] In Proceedings of CHI Conference on Human Factors in Computing Systems (CHI '21), May 8-13, 2021, Yokohama, Japan. ACM, New York, NY, USA. 19 Pages. https://doi.org/10.1145/3411764.344557

Analysing the Role of Interactivity in User Experience

An experimental investigation into the role of interaction in user experience (UX) with a controlled manipulation of interactivity features (e.g. avatars, interactive video) in a university information website is reported. The more interactive version had better affect and hedonic ratings, even though its perceived usability was worse. Analysis of qualitative data showed users were attracted to the interactive features, although they complained about poor usability. The results of the experiments are discussed to consider the role of interactivity in user experience and the differences between users’ quantitative judgements of UX and their comments on interactive features which reveal different perspectives

Possibilities of eye tracking and EEG integration for visual search on 2D maps

This on-going research paper explores (the possibilities to integrate eye tracking (ET) and electroencephalogram (EEG) for cartographic usability research. While ET, on one hand, provides observations and measurements related to gaze movements, EEG, on the other hand, helps to monitor and measure electrical activity occurring at different locations in the brain with a high temporal resolution. Therefore, combining ET and EEG introduces a holistic approach enabling to measure both overt and covert attention, and additionally, may reveal insights on individual’s different strategies of spatial cognition, if there is any. In this context, we introduce the experimental design settings for visual search task on simplified 2D static maps considering expert and novice participants, outlining methodological proposal and possible analyses. The paper mainly discusses the technical and theoretical issues of ET-EEG integration and mentions potential benefits of implementing EEG in cartographic usability research to indicate its value for future studies

Eye tracking in retrospective think-aloud usability testing: Is there added value?

Eye tracking is the process of recording users’ eye movements while they are looking at the location of an object. In usability testing, this technique is commonly used in combination with think-aloud protocols. This paper presents an experimental study involving 24 participants; with the aim of comparing two variants of retrospective think-aloud (RTA) methods, that is, video-cued RTA method and gaze-cued RTA method, to address the value of having an extra eye-cue in retrospective think-aloud usability testing. Results suggest that both RTA variants are effective in detecting major usability problems. Moreover, the combination of eye tracking techniques and think-aloud protocols can further help evaluators to detect more usability problems, especially minor navigational and comprehension problems. It also helps participants to remember their behavior details, such as what they were looking at on a web page, as mouse movement alone might not be representative of their actual thoughts. Nevertheless, we found that participants might become distracted while seeing their eye movement, which can affect their verbalization performance and, hence, they might experience longer silence periods

An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work.

Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors' efficacy for workplace stress monitoring, without discriminating different tasks. Given the variation in workplace computer activities, this study investigates the efficacy of unobtrusive sensors for stress measurement across a variety of tasks. We present a comparison of five physiological measurements obtained in a lab experiment, where participants completed six different computer tasks, while we measured their stress levels using a chest-band (ECG, respiration), a wristband (PPG and EDA), and an emerging thermal imaging method (perinasal perspiration). We found that thermal imaging can detect increased stress for most participants across all tasks, while wrist and chest sensors were less generalizable across tasks and participants. We summarize the costs and benefits of each sensor stream, and show how some computer use scenarios present usability and reliability challenges for stress monitoring with certain physiological sensors. We provide recommendations for researchers and system builders for measuring stress with physiological sensors during workplace computer use