A Systematic Literature Review of Empirical Studies on Learning Analytics in Educational Games

Learning analytics (LA) in educational games is considered an emerging practice due to its potential of enhancing the learning process. Growing research on formative assessment has shed light on the ways in which students' meaningful and in-situ learning experiences can be supported through educational games. To understand learners' playful experiences during gameplay, researchers have applied LA, which focuses on understanding students' in-game behaviour trajectories and personal learning needs during play. However, there is a lack of studies exploring how further research on LA in educational games can be conducted. Only a few analyses have discussed how LA has been designed, integrated, and implemented in educational games. Accordingly, this systematic literature review examined how LA in educational games has evolved. The study findings suggest that: (1) there is an increasing need to consider factors such as student modelling, iterative game design and personalisation when designing and implementing LA through educational games; and (2) the use of LA creates several challenges from technical, data management and ethical perspectives. In addition to outlining these findings, this article offers important notes for practitioners, and discusses the implications of the study’s results

New measurement paradigms

This collection of New Measurement Paradigms papers represents a snapshot of the variety of measurement methods in use at the time of writing across several projects funded by the National Science Foundation (US) through its REESE and DR K–12 programs. All of the projects are developing and testing intelligent learning environments that seek to carefully measure and promote student learning, and the purpose of this collection of papers is to describe and illustrate the use of several measurement methods employed to achieve this. The papers are deliberately short because they are designed to introduce the methods in use and not to be a textbook chapter on each method. The New Measurement Paradigms collection is designed to serve as a reference point for researchers who are working in projects that are creating e-learning environments in which there is a need to make judgments about students’ levels of knowledge and skills, or for those interested in this but who have not yet delved into these methods

Developing and Validating Stealth Assessments for an Educational Game to Assess Young Dual Language Immersion Learners\u27 Reading Comprehension

The purpose of this multiple-paper dissertation is to design a digital game and stealth assessments within the game to assess young second language learners\u27 Chinese reading proficiency. In Chapter 2 (Paper 1), I describe the game designed for this dissertation and how it was implemented in a dual language immersion classroom. This study found that the digital game and in-class implementation led to significant vocabulary and reading comprehension gains. Further, seven types of support that students needed while playing the game were identified. In Chapter 3 (Paper 2), I describe how educational data mining approaches, and more specifically, how data-driven explorations, can provide insight into how players interact with the game and further how those interactions relate to proficiency and learning. In this study, I identify time on task and use of an in-game glossing tool as important indicators for learning. In addition, four subgroups of students were identified based on their gameplay styles. Finally, in Chapter 4 (Paper 3), I describe how stealth assessments were designed and validated within the game. This study found that the stealth assessments were significantly correlated with two external measures of reading comprehension

E3: Emotions, Engagement, and Educational Digital Games

The use of educational digital games as a method of instruction for science, technology, engineering, and mathematics has increased in the past decade. While these games provide successfully implemented interactive and fun interfaces, they are not designed to respond or remedy students’ negative affect towards the game dynamics or their educational content. Therefore, this exploratory study investigated the frequent patterns of student emotional and behavioral response to educational digital games. To unveil the sequential occurrence of these affective states, students were assigned to play the game for nine class sessions. During these sessions, their affective and behavioral response was recorded to uncover possible underlying patterns of affect (particularly confusion, frustration, and boredom) and behavior (disengagement). In addition, these affect and behavior frequency pattern data were combined with students’ gameplay data in order to identify patterns of emotions that led to a better performance in the game. The results provide information on possible affect and behavior patterns that could be used in further research on affect and behavior detection in such open-ended digital game environments. Particularly, the findings show that students experience a considerable amount of confusion, frustration, and boredom. Another finding highlights the need for remediation via embedded help, as the students referred to peer help often during their gameplay. However, possibly because of the low quality of the received help, students seemed to become frustrated or disengaged with the environment. Finally, the findings suggest the importance of the decay rate of confusion; students’ gameplay performance was associated with the length of time students remained confused or frustrated. Overall, these findings show that there are interesting patterns related to students who experience relatively negative emotions during their gameplay

Supporting Skill Assessment in Learning Experiences Based on Serious Games Through Process Mining Techniques

Learning experiences based on serious games are employed in multiple contexts. Players carry out multiple interactions during the gameplay to solve the different challenges faced. Those interactions can be registered in logs as large data sets providing the assessment process with objective information about the skills employed. Most assessment methods in learning experiences based on serious games rely on manual approaches, which do not scalewell when the amount of data increases. We propose an automated method to analyse students’ interactions and assess their skills in learning experiences based on serious games. The method takes into account not only the final model obtained by the student, but also the process followed to obtain it, extracted from game logs. The assessment method groups students according to their in-game errors and ingame outcomes. Then, the models for the most and the least successful students are discovered using process mining techniques. Similarities in their behaviour are analysed through conformance checking techniques to compare all the students with the most successful ones. Finally, the similarities found are quantified to build a classification of the students’ assessments. We have employed this method with Computer Science students playing a serious game to solve design problems in a course on databases. The findings show that process mining techniques can palliate the limitations of skill assessment methods in game-based learning experiences

Conceptual Framework for Designing Virtual Field Trip Games

This thesis aimed to provide designing models to explore an alternative solution for a field trip when it becomes impossible for several reasons such as the limitation of cost and time. Virtual field trip games are relatively new means to create virtual field trips in game environments through adding game aspects to learning aspects to enhance the learning experience. The simple combining of game and learning aspects will not guarantee the desired effect of virtual field trips. Theoretical and logical connections should be established to form interweave between both aspects. This thesis proposes a designing framework by establishing three links between game design aspects and learning aspects. The three links are constructed by modelling: the experiential learning theory (ELT), the gameplay, and the game world. ELT modelling quantifies the theory into the internal economy mechanic and balances the levels of game task difficulty with the player’s ability through game machinations, game modelling links the learning process to gameplay, and world modelling connects field environment to game environment. The internal economy mechanic and its components (resources, internal mechanic, feedback loop), formulating equations to define generic player’s interactions and identify indicators to capture evidence of achievements via a mathematical (evaluation) model. The game modelling includes skill models to design two important high-order skills (decision-making and teamwork) and connects them to the evaluation model. The game world is modelled through defining its variables and relationships’ rules to connect both environments (game and field) expanding the evaluation model. The framework is supported by essential learning theories (ELT, task-based learning, some aspects of social learning) and pedagogical aspects (assessment, feedback, field-based structure, high-order skills) and connected to the key game elements (interaction, multimodal presentation, control of choice…etc) of field-based learning along with suitable game mechanics. The two research studies that were conducted as part of this thesis found that the designing framework is useful, usable, and provides connections between learning and game aspects and the designed VFTG based on the framework improved learning performance along with providing motivation and presence. This suggests the effectiveness of the framework

Games, design and assessment : how game designers are doing it right

The fixation of formal education to measure and certify academic attainment, continues to fuel the ubiquity of standardised assessment procedures. As such, educators are reluctant to embrace the same constructivist and novel stance adopted towards learning to the assessment domain and continue to favour summative over formative practices. On the other hand, literature on games in education, suggests that well-designed digital games support and enhance the positive interplay between the different forms and functions of assessment, which are inherent to the learning environment. This paper examines the principles of good game design in light of the constructive interaction that exists between learning and the different functions of assessment in games. For instance, levelling and collecting badges whilst climbing up the ladder of challenges found in games, can be considered as the outcome of a purely summative assessment of the player’s progress, as the information collected during gameplay is solely used to judge and certify the player’s performance. However and more importantly, both for the game designers and the players, the game is continuously producing immediate performance feedback in the form of, amongst others, both on-demand and just-in-time information, which informs the players’ learning and allows them to adjust their actions, thus serving as a formative assessment of gameplay. On analysing the respective literature in the field of game studies and assessment, this paper discusses a number of game elements and core mechanics, under three broad themes, namely (i) adaptivity (ii) feedforward and (iii) distributed cognition, which game designers successfully deploy in good game designs. These are gradually and naturally extended towards the theoretical and practical underpinning of an assessment for learning pedagogy, thus potentially informing and transforming traditional assessment practices into a more playful experience.peer-reviewe

Motivating children to learn effectively: exploring the value of intrinsic integration in educational games

The concept of intrinsic motivation lies at the heart of the user engagement created by digital games. Yet despite this, educational software has traditionally attempted to harness games as extrinsic motivation by using them as a sugar coating for learning content. This article tests the concept of intrinsic integration as a way of creating a more productive relationship between educational games and their learning content. Two studies assessed this approach by designing and evaluating an educational game called Zombie Division to teach mathematics to 7- to 11-year-olds. Study 1 examined the learning gains of 58 children who played either the intrinsic, extrinsic, or control variants of Zombie Division for 2 hr, supported by their classroom teacher. Study 2 compared time on task for the intrinsic and extrinsic variants of the game when 16 children had free choice of which game to play. The results showed that children learned more from the intrinsic version of the game under fixed time limits and spent 7 times longer playing it in free-time situations. Together, these studies offer evidence for the genuine value of an intrinsic approach for creating effective educational games. The theoretical and commercial implications of these findings are discussed

Examining scientific thinking processes in open-ended serious games through gameplay data

Research on scientific problem-solving emphasizes the importance of problem solving and scientific inquiry as central components of the twenty-first century skills. Research has shown that open-ended serious games can facilitate students’ development of specific skills and improve learning performance through scientific problem-solving. However, understanding how students learn these complex skills in a game environment is a major challenge, as much research depends on typical paper-and-pencil assessments and self-reported surveys or other traditional observational and quantitative methods. The participants of the study were 237 sixth graders from two middle schools in the Southwestern area of the United States. The students used an open-ended serious game called Alien Rescue as their science curriculum for three weeks. The purpose of this study is, first, to identify students’ navigation behavior patterns in cognitive processes between at-risk and non-at-risk students within Alien Rescue. To accomplish this purpose, this study intends to use gameplay data by incorporating the integrated method of lag sequential analysis and sequential pattern mining together with a statistical analysis. The findings confirmed that the integrated method helped to explore students’ latent navigation behaviors as well as discover the differences of problem-solving processes between non-at-risk and at-risk students. The second purpose of this study is to examine the relationship between students’ learning performance and their scientific inquiry behaviors, which emerged as students engaged with Probe Design Center in this serious game. The results showed that the game metrics developed in Probe Design Center improved the predictions of both in-game and after-game performance. The cluster analyses with game metrics confirmed four unique groups regarding students’ scientific inquiry behaviors in Probe Design Center. This study concluded that the integrated methods of serious games analytics enabled researchers to investigate in-depth cognitive processes and scientific inquiry behaviors within a specific cognitive tool, Probe Design Center, and discover unique behavior groups across different school settings. The researcher identified the challenges of at-risk students in their cognitive processes and highlighted the support needs for these students. Consequently, this study proposed an interactive dashboard using the data-driven evidences to provide teachers just-in-time information to support students’ cognitive processes.Curriculum and Instructio