Effects of observing a model’s natural or didactic problem-solving behavior in eye movement modeling examples

Observing how a knowledgeable person demonstrates (or ‘models’) how to perform a task is a natural way of learning and can help with the acquisition of new skills (Van Gog & Rummel, 2010). Over the past years, video-based modeling examples have become easier to generate and disseminate. However, learners are often deprived from important social cues such as eye movements, head turns, or pointing gestures to the referred elements which would, in a traditional classroom setting, guide the learners’ attention to understand the model’s references (Ouwehand, van Gog, & Paas, 2015). One idea for fostering video learning is ‘Eye Movement Modelling Examples’ (Van Gog, Jarodzka, Scheiter, Gerjets, & Paas, 2009). In EMMEs, a model’s eye movements are superimposed onto the task material during execution, for instance as circles or a spotlight. These eye-movement displays could disambiguate the model’s references and thus improve the learners’ understanding. In the field of programming education, studies recently showed promising first results that EMME-based interventions can foster the acquisition of programming skills (Bednarik, Schulte, Budde, Heinemann, & Vrzakova, 2018; Stein & Brennan, 2004). However, clear design guidelines on how to create an optimal EMME are lacking and the creation process of EMMEs differs highly across studies. For instance, model instructions to create EMMEs range from no specific instruction (e.g., Litchfield, Ball, Donovan, Manning, & Crawford, 2010; Nalanagula, Greenstein, & Gramopadhye, 2006) to studies that explicitly prompt the models to adjust their behavior in a didactic manner to a novice audience (e.g., Jarodzka et al., 2012; Jarodzka, van Gog, Dorr, Scheiter, & Gerjets, 2013). Whether, and if so, to what extent, such instructions affect learning outcomes, is unknown. On the one hand, following naturally behaving experts might be more difficult for learners than following didactic examples. On the other hand, natural modeling examples could foster observational learning and the learners could gain insights into the standards their performance should ultimately meet. Our study aims at investigating how displaying either programming experts’ natural or didactic behavior in EMMEs affects learners’ mental effort ratings, video understanding, and debugging performance. This can later provide researchers and practitioners with evidence-based guidelines on how to create effective EMME videos and raise awareness of the importance of model instruction for EMME creation. Data collection will take place in May and June 2019 and we will present preliminary results at the network meeting. References Bednarik, R., Schulte, C., Budde, L., Heinemann, B., & Vrzakova, H. (2018). Eye-movement Modeling Examples in Source Code Comprehension: A Classroom Study. Paper presented at the Proceedings of the 18th Koli Calling International Conference on Computing Education Research. Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M., Scheiter, K., Gerjets, P., & Eika, B. (2012). Conveying clinical reasoning based on visual observation via eye-movement modelling examples. Instructional Science, 40(5), 813-827. Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction, 25, 62-70. Litchfield, D., Ball, L. J., Donovan, T., Manning, D. J., & Crawford, T. (2010). Viewing another person's eye movements improves identification of pulmonary nodules in chest x-ray inspection. Journal of Experimental Psychology: Applied, 16(3), 251. Nalanagula, D., Greenstein, J. S., & Gramopadhye, A. K. (2006). Evaluation of the effect of feedforward training displays of search strategy on visual search performance. International Journal of Industrial Ergonomics, 36(4), 289-300. Ouwehand, K., van Gog, T., & Paas, F. (2015). Designing effective video-based modeling examples using gaze and gesture cues. Educational Technology & Society, 18(4), 78-88. Stein, R., & Brennan, S. E. (2004). Another person's eye gaze as a cue in solving programming problems. Paper presented at the Proceedings of the 6th international conference on Multimodal interfaces. Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25(3), 785-791. Van Gog, T., & Rummel, N. (2010). Example-based learning: Integrating cognitive and social-cognitive research perspectives. Educational Psychology Review, 22(2), 155-174

‘Explain It to a Beginner!’ Effects of Model Instruction on Learning with Eye Movement Modeling Examples

Eye-movement modeling example (EMME) videos show the expert’s eye-movements (i.e., the focus of attention) while h/she demonstrates how to perform a task. EMMEs often have found to guide learners’ attention and foster their learning. One potential influencing factor on EMME’s effectiveness might be its concrete design. However, clear design guidelines on how to create an optimal EMME are missing. For instance, some (expert) models received the instruction to behave ‘didactically’ when creating an EMME. Recently, we (Authors, in prep.) found that the instruction to behave didactically clearly changes (programming) experts' non-verbal behavior and thus, the EMME videos. Consequently, our next study uses EMME videos about code debugging to investigate the effect of model instruction (non-didactical EMME vs. didactical EMME) on novices’ learning and video evaluation. Ideas on the study design will be presented at the ICO NSS19. Our findings may provide researchers and practitioners with guidelines on how to create more effective EMME videos

Effects of observing a model’s natural or didactic problem-solving behavior in eye movement modeling examples

Observing how a knowledgeable person demonstrates (or ‘models’) how to perform a task is a natural way of learning and can help with the acquisition of new skills (Van Gog & Rummel, 2010). Over the past years, video-based modeling examples have become easier to generate and disseminate. However, learners are often deprived from important social cues such as eye movements, head turns, or pointing gestures to the referred elements which would, in a traditional classroom setting, guide the learners’ attention to understand the model’s references (Ouwehand, van Gog, & Paas, 2015). One idea for fostering video learning is ‘Eye Movement Modelling Examples’ (Van Gog, Jarodzka, Scheiter, Gerjets, & Paas, 2009). In EMMEs, a model’s eye movements are superimposed onto the task material during execution, for instance as circles or a spotlight. These eye-movement displays could disambiguate the model’s references and thus improve the learners’ understanding. In the field of programming education, studies recently showed promising first results that EMME-based interventions can foster the acquisition of programming skills (Bednarik, Schulte, Budde, Heinemann, & Vrzakova, 2018; Stein & Brennan, 2004). However, clear design guidelines on how to create an optimal EMME are lacking and the creation process of EMMEs differs highly across studies. For instance, model instructions to create EMMEs range from no specific instruction (e.g., Litchfield, Ball, Donovan, Manning, & Crawford, 2010; Nalanagula, Greenstein, & Gramopadhye, 2006) to studies that explicitly prompt the models to adjust their behavior in a didactic manner to a novice audience (e.g., Jarodzka et al., 2012; Jarodzka, van Gog, Dorr, Scheiter, & Gerjets, 2013). Whether, and if so, to what extent, such instructions affect learning outcomes, is unknown. On the one hand, following naturally behaving experts might be more difficult for learners than following didactic examples. On the other hand, natural modeling examples could foster observational learning and the learners could gain insights into the standards their performance should ultimately meet. Our study aims at investigating how displaying either programming experts’ natural or didactic behavior in EMMEs affects learners’ mental effort ratings, video understanding, and debugging performance. This can later provide researchers and practitioners with evidence-based guidelines on how to create effective EMME videos and raise awareness of the importance of model instruction for EMME creation. Data collection will take place in May and June 2019 and we will present preliminary results at the network meeting. References Bednarik, R., Schulte, C., Budde, L., Heinemann, B., & Vrzakova, H. (2018). Eye-movement Modeling Examples in Source Code Comprehension: A Classroom Study. Paper presented at the Proceedings of the 18th Koli Calling International Conference on Computing Education Research. Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M., Scheiter, K., Gerjets, P., & Eika, B. (2012). Conveying clinical reasoning based on visual observation via eye-movement modelling examples. Instructional Science, 40(5), 813-827. Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction, 25, 62-70. Litchfield, D., Ball, L. J., Donovan, T., Manning, D. J., & Crawford, T. (2010). Viewing another person's eye movements improves identification of pulmonary nodules in chest x-ray inspection. Journal of Experimental Psychology: Applied, 16(3), 251. Nalanagula, D., Greenstein, J. S., & Gramopadhye, A. K. (2006). Evaluation of the effect of feedforward training displays of search strategy on visual search performance. International Journal of Industrial Ergonomics, 36(4), 289-300. Ouwehand, K., van Gog, T., & Paas, F. (2015). Designing effective video-based modeling examples using gaze and gesture cues. Educational Technology & Society, 18(4), 78-88. Stein, R., & Brennan, S. E. (2004). Another person's eye gaze as a cue in solving programming problems. Paper presented at the Proceedings of the 6th international conference on Multimodal interfaces. Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25(3), 785-791. Van Gog, T., & Rummel, N. (2010). Example-based learning: Integrating cognitive and social-cognitive research perspectives. Educational Psychology Review, 22(2), 155-174

Introducing Eye Movement Modeling Examples for Programming Education and the Role of Teacher's Didactic Guidance

In this article, we introduce how eye-tracking technology might become a promising tool to teach programming skills, such as debugging with ‘Eye Movement Modeling Examples’ (EMME). EMME are tutorial videos that visualize an expert's (e.g., a programming teacher's) eye movements during task performance to guide students’ attention, e.g., as a moving dot or circle. We first introduce the general idea behind the EMME method and present studies that showed first promising results regarding the benefits of EMME to support programming education. However, we argue that the instructional design of EMME varies notably across them, as evidence-based guidelines on how to create effective EMME are often lacking. As an example, we present our ongoing research on the effects of different ways to instruct the EMME model prior to video creation. Finally, we highlight open questions for future investigations that could help improving the design of EMME for (programming) education

Expertise-related differences in gaze during code debugging

With the growing demand for programming expertise on the labor market, the investigation and optimization of programming education become increasingly relevant. First, this eye-tracking study investigates how experts’ usual gaze patterns differ from those of novices while debugging short computer codes. Second, this study aims to explore how the expert participants change their gaze behavior when being instructed to explain their previous debugging approach didactically. This explorative investigation will help to answer the questions how experts try to make themselves understandable to novices in instructional videos and what features didactic eye movements possess that could guide a learner’s attention

How Experts Adapt Their Gaze Behavior When Modeling a Task to Novices

Domain experts regularly teach novice students how to perform a task. This often requires them to adjust their behavior to the less knowledgeable audience and, hence, to behave in a more didactic manner. Eye movement modeling examples (EMMEs) are a contemporary educational tool for displaying experts’ (natural or didactic) problem‐solving behavior as well as their eye movements to learners. While research on expert‐novice communication mainly focused on experts’ changes in explicit, verbal communication behavior, it is as yet unclear whether and how exactly experts adjust their nonverbal behavior. This study first investigated whether and how experts change their eye movements and mouse clicks (that are displayed in EMMEs) when they perform a task naturally versus teach a task didactically. Programming experts and novices initially debugged short computer codes in a natural manner. We first characterized experts’ natural problem‐solving behavior by contrasting it with that of novices. Then, we explored the changes in experts’ behavior when being subsequently instructed to model their task solution didactically. Experts became more similar to novices on measures associated with experts’ automatized processes (i.e., shorter fixation durations, fewer transitions between code and output per click on the run button when behaving didactically). This adaptation might make it easier for novices to follow or imitate the expert behavior. In contrast, experts became less similar to novices for measures associated with more strategic behavior (i.e., code reading linearity, clicks on run button) when behaving didactically