A Mixed Method Approach for Evaluating and Improving the Design of Learning in Puzzle Games

Despite the acknowledgment that learning is a necessary part of all gameplay, the area of Games User Research lacks an established evidence based method through which designers and researchers can understand, assess, and improve how commercial games teach players game-specific skills and information. In this paper, we propose a mixed method procedure that draws together both quantitative and experiential approaches to examine the extent to which players are supported in learning about the game world and mechanics. We demonstrate the method through presenting a case study of the game Portal involving 14 participants, who differed in terms of their gaming expertise. By comparing optimum solutions to puzzles against observed player performance, we illustrate how the method can indicate particular problems with how learning is structured within a game. We argue that the method can highlight where major breakdowns occur and yield design insights that can improve the player experience with puzzle games

Scaffolding Novices to Leverage Auditory Awareness Cues in First-Person Shooters

Today's digital games require the mastery of many different skills. This is accomplished through play itself -- sometimes experientially and other times by using explicit guidance provided by the game designer. Multiplayer games, due to their competitive nature, provide fewer opportunities for designers to guide players into mastering particular skills, and so players must learn and master skills experientially. However, when novices compete against better players -- as they would if they were new to the game -- they can feel overwhelmed by the skill differential. This may hinder the ability of novices to learn experientially, and more importantly, may lead to extended periods of unsatisfying play and missed social play opportunities as they struggle to improve in a competitive context. A game genre that suffers from this problem is the multiplayer first-person shooter (FPS), in which the skill difference between new players and experts who have reached a high level of expertise can be quite large. To succeed in a FPS, players must master a number of skills, the most obvious of which are navigating a complex 3D environment and targeting opponents. To target opponents in a 3D environment, you must also be able to locate them -- a skill known as "opponent location awareness". With the goal of helping novices learn the skill of opponent location awareness, we first conducted an experiment to determine how experts accomplish this important task in multiplayer FPS games. After determining that an understanding of audio cues -- and how to leverage them -- was critical, we designed and evaluated two systems for introducing this skill of locating opponents through audio cues -- an explicit stand-alone training system, and a modified game interface for embedded training. We found that both systems improved accuracy and confidence, but that the explicit training system led to more audio cues being recognized. Our work may help people of disparate skill be able to play together, by scaffolding novices to learn and use a strategy commonly employed by experts

Investigation of Videogame Flow: Effects of Expertise and Challenge

The number of participants in this expertise and videogame flow test totaled 80 from multiple target locations. Participants engaged in various levels of the videogame SuperMario Bros. Twenty experts and twenty novices experienced the easier level of World 1-2 while the other twenty experts and novices were exposed to the more difficult level World 6-1. After gameplay, participants completed a modified survey measuring flow. This survey, along with overall percentage game score, was analyzed. A significant interaction was found between game level (challenge level) and skill levels in perceived immersion, with significant main effects for expertise in perceived skill, for game level (challenge level) in perceived challenge, and for game level (challenge level) in overall percentage score. No significant correlation was found between perceived skill and overall percentage score, between perceived challenge and overall percentage score, or between perceived immersion and overall percentage score. These findings are relevant for understanding videogame flow in videogames of varying degrees of challenge and in players of different expertise levels. Discussion on these findings highlights the purpose of this paper

Solutions to cognitive overload in game-based learning using learning experience design for K-12 education : a review of the literature

Learners especially in K-12 education often encounter learning difficulties in the context of game-based learning. One explanation accounting for such learning difficulties in game-based environments is that cognitive load is not properly managed and overload is imposed on learners’ working memory. Learning Experience Design (LX Design), as an ideal substitute compared to instructional design, has been growing to meet the requirements of game-based learning in the 21st century. Based on the development of game-based learning, various factors (e.g., content and functionality) should be weighed to facilitate an optimal learning experience. This report reviews the literature on the impact of cognitive (over)load on game-based learning based on Cognitive Load Theory (CLT). The report applies one specific User Experience Design (UX Design) model, Garrett’s Elements (Garrett, 2011) to explain how LX Design can provide solutions to learning difficulties caused by cognitive (over)load. The purpose of this report is to conduct a review of literature including empirical studies and theoretical articles from 2007 to present. The findings showed that meaningful learning experience designs for game-based learning, in which three types of cognitive load (i.e., intrinsic, extraneous and germane cognitive load) play an important role, should integrate game design with instructional design principles.Curriculum and Instructio

An experimental study on the effects of a simulation game on students’ clinical cognitive skills and motivation

textabstractSimulation games are becoming increasingly popular in education, but more insight in their critical design features is needed. This study investigated the effects of fidelity of open patient cases in adjunct to an instructional e-module on students’ cognitive skills and motivation. We set up a three-group randomized post-test-only design: a control group working on an e-module; a cases group, combining the e-module with low-fidelity text-based patient cases, and a game group, combining the e-module with a high-fidelity simulation game with the same cases. Participants completed questionnaires on cognitive load and motivation. After a 4-week study period, blinded assessors rated students’ cognitive emergency care skills in two mannequin-based scenarios. In total 61 students participated and were assessed; 16 control group students, 20 cases students and 25 game students. Learning time was 2 h longer for the cases and game groups than for the control group. Acquired cognitive skills did not differ between groups. The game group experienced higher intrinsic and germane cognitive load than the cases group (p = 0.03 and 0.01) and felt more engaged (p < 0.001). Students did not profit from working on open cases (in adjunct to an e-module), which nonetheless challenged them to study longer. The e-module appeared to be very effective, while the high-fidelity game, although engaging, probably distracted students and impeded learning. Medical educators designing motivating and effective skills training for novices should align case complexity and fidelity with students’ proficiency level. The relation between case-fidelity, motivation and skills development is an important field for further study

Learning Dimensions: Lessons from Field Studies

In this paper, we describe work to investigate the creation of engaging programming learning experiences. Background research informed the design of four fieldwork studies involving a range of age groups to explore how programming tasks could best be framed to motivate learners. Our empirical findings from these four studies, described here, contributed to the design of a set of programming "Learning Dimensions" (LDs). The LDs provide educators with insights to support key design decisions for the creation of engaging programming learning experiences. This paper describes the background to the identification of these LDs and how they could address the design and delivery of highly engaging programming learning tasks. A web application has been authored to support educators in the application of the LDs to their lesson design

The World (of Warcraft) through the eyes of an expert

Negative correlations between pupil size and the tendency to look at salient locations were found in recent studies (e.g., Mathôt et al., 2015). It is hypothesized that this negative correlation might be explained by the mental effort put by participants in the task that leads in return to pupil dilation. Here we present an exploratory study on the effect of expertise on eye-movement behavior. Because there is no available standard tool to evaluate WoW players’ expertise, we built an off-game questionnaire testing players’ knowledge about WoW and acquired skills through completed raids, highest rated battlegrounds, Skill Points, etc. Experts (N = 4) and novices (N = 4) in the massively multiplayer online role-playing game World of Warcraft (WoW) viewed 24 designed video segments from the game that differ in regards with their content (i.e, informative locations) and visual complexity (i.e, salient locations). Consistent with previous studies, we found a negative correlation between pupil size and the tendency to look at salient locations (experts, r =  − .17, p < .0001, and novices, r =  − .09, p < .0001). This correlation has been interpreted in terms of mental effort: People are inherently biased to look at salient locations (sharp corners, bright lights, etc.), but are able (i.e., experts) to overcome this bias if they invest sufficient mental effort. Crucially, we observed that this correlation was stronger for expert WoW players than novice players (Z =  − 3.3, p = .0011). This suggests that experts learned to improve control over eye-movement behavior by guiding their eyes towards informative, but potentially low-salient areas of the screen. These findings may contribute to our understanding of what makes an expert an expert

User perception of gaming element effectiveness in a corporate learning application

This Conversion Masters in Information Technology thesis gathered users' perceptions about eight gaming elements to determine their effectiveness on aspects of playability, enjoyment and intrinsic motivation needed in a gamified corporate learning application. The study focused on user opinions about a Progress Bar, Individual Leaderboard, Departmental Leaderboard, Timer, In-Game Currency, Badges, Storyline/Theme and Avatar. A gamification application containing these gaming elements was designed and developed to make the evaluation. The application entailed users learning four Information Technology Infrastructure Library (ITIL) processes needed to manage an information technology department in a telecommunications company. The application design process considered the business goals, rules, target behaviours, time limits, rewards, feedback, levels, storytelling, interest, aesthetics, replay or do-overs, user types, activity cycles, fun mechanisms and development tools needed to create a coherent, addictive, engaging and fun user experience. Player types were determined using the Brainhex online survey. Federoff's Game Playability Heuristics model was used to measure the users' perceptions about the playability of the application. Sweetser and Wyeth's Gameflow model was used to measure perceptions about the gaming elements' contribution toward creating an enjoyable experience. Malone and Lepper's Taxonomy of Intrinsic Motivation for Learning was used to measure the gaming elements' ability to promote an intrinsically motivating learning environment. Masterminds, Achievers, Conquerors and Seekers were the most prominent player types found in the Brainhex online survey for which the gamification application design then catered. The staff in the department play-tested the application to evaluate the gaming elements. Overall the Storyline/Theme, suited to Seekers and Masterminds, ranked as the most effective gaming element in this study. The users perceived artwork as an essential component of a gamified learning application. The Individual Leaderboard, suited to Conquerors, ranked very closely as the second most effective gaming element. The Storyline/Theme and Individual Leaderboard both performed the strongest against the criteria measuring the playability. The Storyline/Theme was by far the strongest from a gameflow perspective and the Individual Leaderboard from a motivation perspective. The Avatars ranked the worst across all the measurement criteria. Based on quiz results, 86 percent of the staff in the department had learned the material from the gamified training prototype developed in this work. The findings from this study will therefore serve as input for developing a full-scale gamification learning application