Stars in their eyes: What eye-tracking reveal about multimedia perceptual quality

Perceptual multimedia quality is of paramount importance to the continued take-up and proliferation of multimedia applications: users will not use and pay for applications if they are perceived to be of low quality. Whilst traditionally distributed multimedia quality has been characterised by Quality of Service (QoS) parameters, these neglect the user perspective of the issue of quality. In order to redress this shortcoming, we characterise the user multimedia perspective using the Quality of Perception (QoP) metric, which encompasses not only a user’s satisfaction with the quality of a multimedia presentation, but also his/her ability to analyse, synthesise and assimilate informational content of multimedia. In recognition of the fact that monitoring eye movements offers insights into visual perception, as well as the associated attention mechanisms and cognitive processes, this paper reports on the results of a study investigating the impact of differing multimedia presentation frame rates on user QoP and eye path data. Our results show that provision of higher frame rates, usually assumed to provide better multimedia presentation quality, do not significantly impact upon the median coordinate value of eye path data. Moreover, higher frame rates do not significantly increase level of participant information assimilation, although they do significantly improve overall user enjoyment and quality perception of the multimedia content being shown

Comparison of Written and Spoken Instruction to Foster Coordination between Diagram and Equation in Undergraduate Physics Education

Visual–graphical representations are used to visualise information and are therefore key components of learning materials. An important type of convention-based representation in everyday contexts as well as in science, technology, engineering, and math (STEM) disciplines are vector field plots. Based on the cognitive theory of multimedia learning, we aim to optimize an instruction with symbolical-mathematical and visual-graphical representations in undergraduate physics education through spoken instruction combined with dynamic visual cues. For this purpose, we conduct a pre-post study with 38 natural science students who are divided into two groups and instructed via different modalities and with visual cues on the graphical interpretation of vector field plots. Afterward, the students rate their cognitive load. During the computer-based experiment, we record the participants’ eye movements. Our results indicate that students with spoken instruction perform better than students with written instruction. This suggests that the modality effect is also applicable to mathematical-symbolical and convention-based visual-graphical representations. The differences in visual strategies imply that spoken instruction might lead to increased effort in organising and integrating information. The finding of the modality effect with higher performance during spoken instruction could be explained by deeper cognitive processing of the material

Visual cues improve students’ understanding of divergence and curl: Evidence from eye movements during reading and problem solving

The coordination of multiple external representations is important for learning, but yet a difficult task for students, requiring instructional support. The subject in this study covers a typical relation in physics between abstract mathematical equations (definitions of divergence and curl) and a visual representation (vector field plot). To support the connection across both representations, two instructions with written explanations, equations, and visual representations (differing only in the presence of visual cues) were designed and their impact on students’ performance was tested. We captured students’ eye movements while they processed the written instruction and solved subsequent coordination tasks. The results show that students instructed with visual cues (VC students) performed better, responded with higher confidence, experienced less mental effort, and rated the instructional quality better than students instructed without cues. Advanced eye-tracking data analysis methods reveal that cognitive integration processes appear in both groups at the same point in time but they are significantly more pronounced for VC students, reflecting a greater attempt to construct a coherent mental representation during the learning process. Furthermore, visual cues increase the fixation count and total fixation duration on relevant information. During problem solving, the saccadic eye movement pattern of VC students is similar to experts in this domain. The outcomes imply that visual cues can be beneficial in coordination tasks, even for students with high domain knowledge. The study strongly confirms an important multimedia design principle in instruction, that is, that highlighting conceptually relevant information shifts attention to relevant information and thus promotes learning and problem solving. Even more, visual cues can positively influence students’ perception of course materials

Visualizing a Task Performer’s Gaze to Foster Observers’ Performance and Learning : a Systematic Literature Review on Eye Movement Modeling Examples

Eye movement modeling examples (EMMEs) are instructional videos (e.g., tutorials) that visualize another person’s gaze location while they demonstrate how to perform a task. This systematic literature review provides a detailed overview of studies on the effects of EMME to foster observers’ performance and learning and highlights their differences in EMME designs. Through a broad, systematic search on four relevant databases, we identified 72 EMME studies (78 experiments). First, we created an overview of the different study backgrounds. Studies most often taught tasks from the domains of sports/physical education, medicine, aviation, and STEM areas and had different rationales for displaying EMME. Next, we outlined how studies differed in terms of participant characteristics, task types, and the design of the EMME materials, which makes it hard to infer how these differences affect performance and learning. Third, we concluded that the vast majority of the experiments showed at least some positive effects of EMME during learning, on tests directly after learning, and tests after a delay. Finally, our results provide a first indication of which EMME characteristics may positively influence learning. Future research should start to more systematically examine the effects of specific EMME design choices for specific participant populations and task types

Towards a framework for attention cueing in instructional animations: Guidelines for research and design

This paper examines the transferability of successful cueing approaches from text and static visualization research to animations. Theories of visual attention and learning as well as empirical evidence for the instructional effectiveness of attention cueing are reviewed and, based on Mayer’s theory of multimedia learning, a framework was developed for classifying three functions for cueing: (1) selection—cues guide attention to specific locations, (2) organization—cues emphasize structure, and (3) integration—cues explicate relations between and within elements. The framework was used to structure the discussion of studies on cueing in animations. It is concluded that attentional cues may facilitate the selection of information in animations and sometimes improve learning, whereas organizational and relational cueing requires more consideration on how to enhance understanding. Consequently, it is suggested to develop cues that work in animations rather than borrowing effective cues from static representations. Guidelines for future research on attention cueing in animations are presented

The interplay between methodologies, tasks and visualisation formats in the study of visual expertise

The author examines the  methodological contributions of the ten unique reviews developed in this special issue on the methodologies for studying visual expertise. Opportunities, research results and lessons, offered by each methodology are analyzed according to four dimensions criteria: (i) The different levels of visual and cognitive processes targeted in the different methods; (ii) The effect of the task gaols and  task design on the outcomes; (iii) The potential effect of the format (dynamic versus static) of the visual material and the interactive features provided in previous studies (iv) The potential modulating effect of learners individual differences. Concluding comments are developed  about the limitations of each method but also about the challenge and promises of designing and using combined and synchronized methods

Design of a Complex Process Information Visualization: A Study of Eye Tracking and its Relationship to Cognitive Learning

Given the prevalence of visual instruction, it is important to ascertain how these visuals enable cognition particularly while having the complexity of information visualization that encompasses multiple concepts. This study was conducted with fifty female undergraduate students in a university in the central region of Saudi Arabia. It sought to understand what pattern and process a viewer’s eyes takes when viewing a complex design of instructional information visualization, and to ascertain if any correlation exists between eye movement, specifically frequent visits to an area of interest (AOI), and cognition. It builds on literature that identifies a link between patterns of viewing and thought to address whether, using new technology, evidence for such a link exists. It also builds on studies that emphasise the importance of eye tracking technology to test hypotheses concerning perceptual processing. The study employs quantitative analysis to answer the research question. The study reveals that no significant correlation exists between a participant’s eye movement and their degree of cognition associated with the material being observed. Recommendations for a further qualitative study are provided in order to clarify the relationship between eye movement, when focused on a complex design of information visualization, and cognitive uptake. Keywords: instructional design, visual instruction, eye tracking, complex instructional information visualizatio

Training Effects of Adaptive Emotive Responses From Animated Agents in Simulated Environments

Humans are distinct from machines in their capacity to emote, stimulate, and express emotions. Because emotions play such an important role in human interactions, human-like agents used in pedagogical roles for simulation-based training should properly reflect emotions. Currently, research concerning the development of this type of agent focuses on basic agent interface characteristics, as well as character building qualities. However, human-like agents should provide emotion-like qualities that are clearly expressed, properly synchronized, and that simulate complex, real-time interactions through adaptive emotion systems. The research conducted for this dissertation was a quantitative investigation using 3 (within) x 2 (between) x 3 (within) factorial design. A total of 56 paid participants consented to complete the study. Independent variables included emotion intensity (i.e., low, moderate, and high emotion), levels of expertise (novice participant versus experienced participant), and number of trials. Dependent measures included visual attention, emotional response towards the animated agents, simulation performance score, and learners\u27 perception of the pedagogical agent persona while participants interacted with a pain assessment and management simulation. While no relationships were indicated between the levels of emotion intensity portrayed by the animated agents and the participants\u27 visual attention, emotional response towards the animated agent, and simulation performance score, there were significant relationships between the level of expertise of the participant and the visual attention, emotional responses, and performance outcomes. The results indicated that nursing students had higher visual attention during their interaction with the animated agents. Additionally, nursing students expressed more neutral facial expression whereas experienced nurses expressed more emotional facial expressions towards the animated agents. The results of the simulation performance scores indicated that nursing students obtained higher performance scores in the pain assessment and management task than experienced nurses. Both groups of participants had a positive perception of the animated agents persona