Visualizing a Task Performer’s Gaze to Foster Observers’ Performance and Learning : a Systematic Literature Review on Eye Movement Modeling Examples

Eye movement modeling examples (EMMEs) are instructional videos (e.g., tutorials) that visualize another person’s gaze location while they demonstrate how to perform a task. This systematic literature review provides a detailed overview of studies on the effects of EMME to foster observers’ performance and learning and highlights their differences in EMME designs. Through a broad, systematic search on four relevant databases, we identified 72 EMME studies (78 experiments). First, we created an overview of the different study backgrounds. Studies most often taught tasks from the domains of sports/physical education, medicine, aviation, and STEM areas and had different rationales for displaying EMME. Next, we outlined how studies differed in terms of participant characteristics, task types, and the design of the EMME materials, which makes it hard to infer how these differences affect performance and learning. Third, we concluded that the vast majority of the experiments showed at least some positive effects of EMME during learning, on tests directly after learning, and tests after a delay. Finally, our results provide a first indication of which EMME characteristics may positively influence learning. Future research should start to more systematically examine the effects of specific EMME design choices for specific participant populations and task types

What is my teacher talking about? Effects of displaying the teacher’s gaze and mouse cursor cues in video lectures on students’ learning

Eye movement modelling examples (EMME) are instructional videos that display a teacher’s eye movements as “gaze cursor” (e.g. a moving dot) superimposed on the learning task. This study investigated if previous findings on the beneficial effects of EMME would extend to online lecture videos and compared the effects of displaying the teacher’s gaze cursor with displaying the more traditional mouse cursor as a tool to guide learners’ attention. Novices (N = 124) studied a pre-recorded video lecture on how to model business processes in a 2 (mouse cursor absent/present) × 2 (gaze cursor absent/present) between-subjects design. Unexpectedly, we did not find significant effects of the presence of gaze or mouse cursors on mental effort and learning. However, participants who watched videos with the gaze cursor found it easier to follow the teacher. Overall, participants responded positively to the gaze cursor, especially when the mouse cursor was not displayed in the video

Inferring task performance and confidence from displays of eye movements

Eye movements reveal what is at the center of people's attention, which is assumed to coincide with what they are thinking about. Eye-movement displays (visualizations of a person's fixations superimposed onto the stimulus, for example, as dots or circles) might provide useful information for diagnosing that person's performance. However, making inferences about a person's task performance based on eye-movement displays requires substantial interpretation. Using graph-comprehension tasks, we investigated to what extent observers (N = 46) could make accurate inferences about a performer's multiple-choice task performance (i.e., chosen answer), confidence, and competence from displays of that person's eye movements. Observers' accuracy when judging which answer the performer chose was above chance level and was higher for displays reflecting confident performance. Observers were also able to infer performers' confidence from the eye-movement displays; moreover, their own task performance and perceived similarity with the performer affected their judgments of the other's competence

Introducing eye movement modeling examples for programming education and the role of teacher's didactic guidance

In this article, we introduce how eye-tracking technology might become a promising tool to teach programming skills, such as debugging with ĝ€-Eye Movement Modeling Examples' (EMME). EMME are tutorial videos that visualize an expert's (e.g., a programming teacher's) eye movements during task performance to guide students' attention, e.g., as a moving dot or circle. We first introduce the general idea behind the EMME method and present studies that showed first promising results regarding the benefits of EMME to support programming education. However, we argue that the instructional design of EMME varies notably across them, as evidence-based guidelines on how to create effective EMME are often lacking. As an example, we present our ongoing research on the effects of different ways to instruct the EMME model prior to video creation. Finally, we highlight open questions for future investigations that could help improving the design of EMME for (programming) education

Inferring task performance and confidence from displays of eye movements

Eye movements reveal what is at the center of people's attention, which is assumed to coincide with what they are thinking about. Eye-movement displays (visualizations of a person's fixations superimposed onto the stimulus, for example, as dots or circles) might provide useful information for diagnosing that person's performance. However, making inferences about a person's task performance based on eye-movement displays requires substantial interpretation. Using graph-comprehension tasks, we investigated to what extent observers (N = 46) could make accurate inferences about a performer's multiple-choice task performance (i.e., chosen answer), confidence, and competence from displays of that person's eye movements. Observers' accuracy when judging which answer the performer chose was above chance level and was higher for displays reflecting confident performance. Observers were also able to infer performers' confidence from the eye-movement displays; moreover, their own task performance and perceived similarity with the performer affected their judgments of the other's competence

How Experts Adapt Their Gaze Behavior When Modeling a Task to Novices

Domain experts regularly teach novice students how to perform a task. This often requires them to adjust their behavior to the less knowledgeable audience and, hence, to behave in a more didactic manner. Eye movement modeling examples (EMMEs) are a contemporary educational tool for displaying experts’ (natural or didactic) problem‐solving behavior as well as their eye movements to learners. While research on expert‐novice communication mainly focused on experts’ changes in explicit, verbal communication behavior, it is as yet unclear whether and how exactly experts adjust their nonverbal behavior. This study first investigated whether and how experts change their eye movements and mouse clicks (that are displayed in EMMEs) when they perform a task naturally versus teach a task didactically. Programming experts and novices initially debugged short computer codes in a natural manner. We first characterized experts’ natural problem‐solving behavior by contrasting it with that of novices. Then, we explored the changes in experts’ behavior when being subsequently instructed to model their task solution didactically. Experts became more similar to novices on measures associated with experts’ automatized processes (i.e., shorter fixation durations, fewer transitions between code and output per click on the run button when behaving didactically). This adaptation might make it easier for novices to follow or imitate the expert behavior. In contrast, experts became less similar to novices for measures associated with more strategic behavior (i.e., code reading linearity, clicks on run button) when behaving didactically

What is my teacher talking about? Effects of displaying the teacher’s gaze and mouse cursor cues in video lectures on students’ learning

Eye movement modelling examples (EMME) are instructional videos that display a teacher’s eye movements as “gaze cursor” (e.g. a moving dot) superimposed on the learning task. This study investigated if previous findings on the beneficial effects of EMME would extend to online lecture videos and compared the effects of displaying the teacher’s gaze cursor with displaying the more traditional mouse cursor as a tool to guide learners’ attention. Novices (N = 124) studied a pre-recorded video lecture on how to model business processes in a 2 (mouse cursor absent/present) × 2 (gaze cursor absent/present) between-subjects design. Unexpectedly, we did not find significant effects of the presence of gaze or mouse cursors on mental effort and learning. However, participants who watched videos with the gaze cursor found it easier to follow the teacher. Overall, participants responded positively to the gaze cursor, especially when the mouse cursor was not displayed in the video

Inferring task performance and confidence from displays of eye movements

Eye movements reveal what is at the center of people's attention, which is assumed to coincide with what they are thinking about. Eye-movement displays (visualizations of a person's fixations superimposed onto the stimulus, for example, as dots or circles) might provide useful information for diagnosing that person's performance. However, making inferences about a person's task performance based on eye-movement displays requires substantial interpretation. Using graph-comprehension tasks, we investigated to what extent observers (N = 46) could make accurate inferences about a performer's multiple-choice task performance (i.e., chosen answer), confidence, and competence from displays of that person's eye movements. Observers' accuracy when judging which answer the performer chose was above chance level and was higher for displays reflecting confident performance. Observers were also able to infer performers' confidence from the eye-movement displays; moreover, their own task performance and perceived similarity with the performer affected their judgments of the other's competence

Introducing eye movement modeling examples for programming education and the role of teacher's didactic guidance

In this article, we introduce how eye-tracking technology might become a promising tool to teach programming skills, such as debugging with ĝ€-Eye Movement Modeling Examples' (EMME). EMME are tutorial videos that visualize an expert's (e.g., a programming teacher's) eye movements during task performance to guide students' attention, e.g., as a moving dot or circle. We first introduce the general idea behind the EMME method and present studies that showed first promising results regarding the benefits of EMME to support programming education. However, we argue that the instructional design of EMME varies notably across them, as evidence-based guidelines on how to create effective EMME are often lacking. As an example, we present our ongoing research on the effects of different ways to instruct the EMME model prior to video creation. Finally, we highlight open questions for future investigations that could help improving the design of EMME for (programming) education