Understanding Engagement within the Context of a Safety Critical Game

One of the most frequent arguments for deploying serious games is that they provide an engaging format for student learning. However, engagement is often equated with enjoyment, which may not be the most relevant conceptualization in safety-critical settings, such as law enforcement and healthcare. In these contexts, the term ‘serious’ does not only relate to the non-entertainment purpose of the game but also the environment simulated by the game. In addition, a lack of engagement in a safety critical training setting can have serious ethical implications, leading to significant real-world impacts. However, evaluations of safety-critical games (SCGs) rarely provide an in-depth consideration of player experience. Thus, in relation to simulation game-based training, we are left without a clear understanding of what sort of experience players are having, what factors influence their engagement and how their engagement relates to learning. In order to address these issues, this paper reports on the mixed-method evaluation of a SCG that was developed to support police training. The findings indicate that engagement is supported by the experience situational relevance, due to the player’s experience of real-world authenticity, targeted feedback mechanisms and learning challenges

Heuristic Evaluation for Serious Immersive Games and M-instruction

© Springer International Publishing Switzerland 2016. Two fast growing areas for technology-enhanced learning are serious games and mobile instruction (M-instruction or M-Learning). Serious games are ones that are meant to be more than just entertainment. They have a serious use to educate or promote other types of activity. Immersive Games frequently involve many players interacting in a shared rich and complex-perhaps web-based-mixed reality world, where their circumstances will be multi and varied. Their reality may be augmented and often self-composed, as in a user-defined avatar in a virtual world. M-instruction and M-Learning is learning on the move; much of modern computer use is via smart devices, pads, and laptops. People use these devices all over the place and thus it is a natural extension to want to use these devices where they are to learn. This presents a problem if we wish to evaluate the effectiveness of the pedagogic media they are using. We have no way of knowing their situation, circumstance, education background and motivation, or potentially of the customisation of the final software they are using. Getting to the end user itself may also be problematic; these are learning environments that people will dip into at opportune moments. If access to the end user is hard because of location and user self-personalisation, then one solution is to look at the software before it goes out. Heuristic Evaluation allows us to get User Interface (UI) and User Experience (UX) experts to reflect on the software before it is deployed. The effective use of heuristic evaluation with pedagogical software [1] is extended here, with existing Heuristics Evaluation Methods that make the technique applicable to Serious Immersive Games and mobile instruction (M-instruction). We also consider how existing Heuristic Methods may be adopted. The result represents a new way of making this methodology applicable to this new developing area of learning technology

'I play, therefore I learn?' Measuring the Evolution of Perceived Learning and Game Experience in the Design Flow of a Serious Game

This article explores how the serious game Poverty Is Not a Game (PING) is experienced by high school students in its subsequent design stages. We first focus on the multifaceted construct of game experience and how it is related to serious games. To measure game experience we use the Game Experience Questionnaire and add a perceived learning scale to account for the specificity of serious games in a classroom. Next, the data obtained from testing PING in 22 classrooms are analyzed. Results suggest that the evolution in the different design stages of the game is not just an issue of game experience, but also of usability. Furthermore, little evidence is found indicating that the learning experience changed positively during the different test phases. However, findings show a strong effect of the game experience on perceived learning while the game experience also varies significantly between different classrooms

Academic methods for usability evaluation of serious games: a systematic review

In the last years, there has been an increasing interest in the design of video games as a tool for education, training, health promotion, socialization, etc. Usability, which is a key factor in any video game, becomes even more important in these so-called Bserious games^, where the users’ special characteristics should be considered, and the game efficacy depends on the users’ adherence and engagement. However, evaluation of the usability of this kind of games requires a redefinition of techniques, methods and even terminology. In this paper, we elicit six research questions and conduct a systematic review of the scientific literature, which resulted in the selection of 187 papers that contained the most relevant responses. The conclusions of this systematic review illustrate the general status of current academic usability evaluations of these games and the main trends in the selection of methodologies and how are they applied. This view may be a very valuable foundation for future researchMinisterio de Ciencia e Innovación PROCUR@-IPT-2011-1038-90000

Validation and Psychometric Properties of the Gameplay-Scale for Educative Video Games in Spanish Children

The knowledge of evaluation instruments to determine the level of gameplay of schoolchildren is very important at this time. A systematic review has been carried out in this study. The aim of this paper is to investigate the psychometric properties of a study of a sample of Spanish gamers. Two hundred and thirty-seven children (mean age: 11.2 +/- 1.17 years, range: 10-12 years, 59.5% female) completed the Gameplay-Scale to discover their opinions after a game session with a serious educational game. The final scale consisted of three factors. The fit for factor 1 (usability) was 0.712, the fit for factor 2 was 0.702 (satisfaction), the fit for factor 3 was 0.886 (empathy) and the overall fit was 0.868. A positive and direct relationship could be observed between all the dimensions of the developed scale. The greatest correlation strength is shown between satisfaction and empathy (r = 0.800; p < 0.005), followed by satisfaction and usability (r = 0.180; p < 0.05) and the association between empathy and usability (r = 0.140; p < 0.05). In summary, the results of the present study support the use of the Gameplay-Scale as a valid and reliable measure of the game experience of youth populations. These results demonstrate strong psychometric properties so that the Gameplay-Scale appears to be a valid instrument for children in different contexts where an educational video game is used, analyzing its usability/“playability” in terms of learning to use it, game satisfaction, and empathy.This research is supported by the Andalusian Research Program under the project P11-TIC-7486 co-funded by FEDER, together with TIN2014-56494-C4-3-P and TEC2015-68752 from the Spanish Ministry of Economy and Competitiveness and FEDER also

Evaluating the impact of physical activity apps and wearables: interdisciplinary review

Background: Although many smartphone apps and wearables have been designed to improve physical activity, their rapidly evolving nature and complexity present challenges for evaluating their impact. Traditional methodologies, such as randomized controlled trials (RCTs), can be slow. To keep pace with rapid technological development, evaluations of mobile health technologies must be efficient. Rapid alternative research designs have been proposed, and efficient in-app data collection methods, including in-device sensors and device-generated logs, are available. Along with effectiveness, it is important to measure engagement (ie, users’ interaction and usage behavior) and acceptability (ie, users’ subjective perceptions and experiences) to help explain how and why apps and wearables work. Objectives: This study aimed to (1) explore the extent to which evaluations of physical activity apps and wearables: employ rapid research designs; assess engagement, acceptability, as well as effectiveness; use efficient data collection methods; and (2) describe which dimensions of engagement and acceptability are assessed. Method: An interdisciplinary scoping review using 8 databases from health and computing sciences. Included studies measured physical activity, and evaluated physical activity apps or wearables that provided sensor-based feedback. Results were analyzed using descriptive numerical summaries, chi-square testing, and qualitative thematic analysis. Results: A total of 1829 abstracts were screened, and 858 articles read in full. Of 111 included studies, 61 (55.0%) were published between 2015 and 2017. Most (55.0%, 61/111) were RCTs, and only 2 studies (1.8%) used rapid research designs: 1 single-case design and 1 multiphase optimization strategy. Other research designs included 23 (22.5%) repeated measures designs, 11 (9.9%) nonrandomized group designs, 10 (9.0%) case studies, and 4 (3.6%) observational studies. Less than one-third of the studies (32.0%, 35/111) investigated effectiveness, engagement, and acceptability together. To measure physical activity, most studies (90.1%, 101/111) employed sensors (either in-device [67.6%, 75/111] or external [23.4%, 26/111]). RCTs were more likely to employ external sensors (accelerometers: P=.005). Studies that assessed engagement (52.3%, 58/111) mostly used device-generated logs (91%, 53/58) to measure the frequency, depth, and length of engagement. Studies that assessed acceptability (57.7%, 64/111) most often used questionnaires (64%, 42/64) and/or qualitative methods (53%, 34/64) to explore appreciation, perceived effectiveness and usefulness, satisfaction, intention to continue use, and social acceptability. Some studies (14.4%, 16/111) assessed dimensions more closely related to usability (ie, burden of sensor wear and use, interface complexity, and perceived technical performance). Conclusions: The rapid increase of research into the impact of physical activity apps and wearables means that evaluation guidelines are urgently needed to promote efficiency through the use of rapid research designs, in-device sensors and user-logs to assess effectiveness, engagement, and acceptability. Screening articles was time-consuming because reporting across health and computing sciences lacked standardization. Reporting guidelines are therefore needed to facilitate the synthesis of evidence across disciplines

The Art of Learning Community: Technology and Gamification As A Recipe For Learning Umami

How can arts, technology, mobile apps, and a learning community be used as flavors to create an engaging learning platform to motivate students?School reform efforts purport to create engaged learners that can think creatively beyond the standards, however, teachers struggle with how to reconcile the culture of standardized testing with the learner engagement and motivation that is key to student success. When designing learning experiences that promote creativity, via information and computer technology, teachers need to adopt an ecological approach that encompasses people, practices, values, and technology interacting- with the spotlight being on human activities. The Japanese word umami describes how humans engage all senses to form judgments about their food. This provides an apt metaphor for instructional design. Food should be nourishing, presentable, and delicious- a feast for the senses. This is a worthy standard for any lesson- the goal of “learning umami.” The author proposed to create an analogous process in crafting an online learning community (http://edvislee.wix.com/rehearse-for-life), which consists of a mash-up of tools, apps, content, gamification, and collaboration with artists as “flavors” for engagement. This paper will review the community’s features before and after modifications, discuss design implications and rationale for changes, and make recommendations for additional improvements. The results demonstrate how pedagogy, design, and evaluation can be used to tailor existing apps, tools, services, and content to create a compelling learning community to meet any instructional design challenge