A Mixed Method Approach for Evaluating and Improving the Design of Learning in Puzzle Games

Despite the acknowledgment that learning is a necessary part of all gameplay, the area of Games User Research lacks an established evidence based method through which designers and researchers can understand, assess, and improve how commercial games teach players game-specific skills and information. In this paper, we propose a mixed method procedure that draws together both quantitative and experiential approaches to examine the extent to which players are supported in learning about the game world and mechanics. We demonstrate the method through presenting a case study of the game Portal involving 14 participants, who differed in terms of their gaming expertise. By comparing optimum solutions to puzzles against observed player performance, we illustrate how the method can indicate particular problems with how learning is structured within a game. We argue that the method can highlight where major breakdowns occur and yield design insights that can improve the player experience with puzzle games

Scaffolding Novices to Leverage Auditory Awareness Cues in First-Person Shooters

Today's digital games require the mastery of many different skills. This is accomplished through play itself -- sometimes experientially and other times by using explicit guidance provided by the game designer. Multiplayer games, due to their competitive nature, provide fewer opportunities for designers to guide players into mastering particular skills, and so players must learn and master skills experientially. However, when novices compete against better players -- as they would if they were new to the game -- they can feel overwhelmed by the skill differential. This may hinder the ability of novices to learn experientially, and more importantly, may lead to extended periods of unsatisfying play and missed social play opportunities as they struggle to improve in a competitive context. A game genre that suffers from this problem is the multiplayer first-person shooter (FPS), in which the skill difference between new players and experts who have reached a high level of expertise can be quite large. To succeed in a FPS, players must master a number of skills, the most obvious of which are navigating a complex 3D environment and targeting opponents. To target opponents in a 3D environment, you must also be able to locate them -- a skill known as "opponent location awareness". With the goal of helping novices learn the skill of opponent location awareness, we first conducted an experiment to determine how experts accomplish this important task in multiplayer FPS games. After determining that an understanding of audio cues -- and how to leverage them -- was critical, we designed and evaluated two systems for introducing this skill of locating opponents through audio cues -- an explicit stand-alone training system, and a modified game interface for embedded training. We found that both systems improved accuracy and confidence, but that the explicit training system led to more audio cues being recognized. Our work may help people of disparate skill be able to play together, by scaffolding novices to learn and use a strategy commonly employed by experts

Investigation of Videogame Flow: Effects of Expertise and Challenge

The number of participants in this expertise and videogame flow test totaled 80 from multiple target locations. Participants engaged in various levels of the videogame SuperMario Bros. Twenty experts and twenty novices experienced the easier level of World 1-2 while the other twenty experts and novices were exposed to the more difficult level World 6-1. After gameplay, participants completed a modified survey measuring flow. This survey, along with overall percentage game score, was analyzed. A significant interaction was found between game level (challenge level) and skill levels in perceived immersion, with significant main effects for expertise in perceived skill, for game level (challenge level) in perceived challenge, and for game level (challenge level) in overall percentage score. No significant correlation was found between perceived skill and overall percentage score, between perceived challenge and overall percentage score, or between perceived immersion and overall percentage score. These findings are relevant for understanding videogame flow in videogames of varying degrees of challenge and in players of different expertise levels. Discussion on these findings highlights the purpose of this paper

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

Chunks in expert memory: Evidence for the magical number four… or is it two?

This study aims to test the divergent predictions of the chunking theory (Chase & Simon, 1973) and template theory (Gobet & Simon, 1996a; 2000) with respect to the number of chunks held in visual short-term memory and the size of chunks used by experts. We presented game and random chessboards in both a copy and a recall task. In a within-subject design, the stimuli were displayed using two presentation media: (a) physical board and pieces, as in Chase and Simon’s (1973) study; and (b) a computer display, as in Gobet and Simon’s (1998) study. Results show that, in most cases, no more than three chunks were replaced in the recall task, as predicted by template theory. In addition, with game positions in the computer condition, Masters replaced very large chunks (up to 15 pieces), again in line with template theory. Overall, the results suggest that the original chunking theory overestimated short-term memory capacity and underestimated the size of chunks used, in particular with Masters. They also suggest that Cowan’s (2001) proposal that STM holds four chunks may be an overestimate

Flow Experience and Challenge-Skill Balance in E-Learning

Flow is an optimal experience resulting in intense engagement in the activity. People achieved flow state when they perceived balance between challenge of the activity and their skill to the activity. The concept of flow can be used to explore students’ learning performance in e-learning environment. The current research aims to empirically explore the influence of challenge-skill balance on the flow experience and the influence of flow experience on learning satisfaction and learning performance in e-learning environment. The current research conducted a quasi-experimental design with questionnaire survey and carried out an electroencephalography (EEG) analysis, a psychophysiological method. The empirical survey results have shown that challenge-skill balance is an antecedent factor affecting learners’ flow experience. Once learners reach flow experience, their learning performance and learning satisfaction would get improved. Besides, the current research also found that flow experience is relative with learners’ attention measured by EEG brainwave signal. Learners’ perception of challenge-skill balance would influence their attention in e- learning activities. The current research is also in the pioneering position that using non-medical purpose EEG device in e-learning research

Understanding Engagement within the Context of a Safety Critical Game

One of the most frequent arguments for deploying serious games is that they provide an engaging format for student learning. However, engagement is often equated with enjoyment, which may not be the most relevant conceptualization in safety-critical settings, such as law enforcement and healthcare. In these contexts, the term ‘serious’ does not only relate to the non-entertainment purpose of the game but also the environment simulated by the game. In addition, a lack of engagement in a safety critical training setting can have serious ethical implications, leading to significant real-world impacts. However, evaluations of safety-critical games (SCGs) rarely provide an in-depth consideration of player experience. Thus, in relation to simulation game-based training, we are left without a clear understanding of what sort of experience players are having, what factors influence their engagement and how their engagement relates to learning. In order to address these issues, this paper reports on the mixed-method evaluation of a SCG that was developed to support police training. The findings indicate that engagement is supported by the experience situational relevance, due to the player’s experience of real-world authenticity, targeted feedback mechanisms and learning challenges

Applying science of learning in education: Infusing psychological science into the curriculum

The field of specialization known as the science of learning is not, in fact, one field. Science of learning is a term that serves as an umbrella for many lines of research, theory, and application. A term with an even wider reach is Learning Sciences (Sawyer, 2006). The present book represents a sliver, albeit a substantial one, of the scholarship on the science of learning and its application in educational settings (Science of Instruction, Mayer 2011). Although much, but not all, of what is presented in this book is focused on learning in college and university settings, teachers of all academic levels may find the recommendations made by chapter authors of service. The overarching theme of this book is on the interplay between the science of learning, the science of instruction, and the science of assessment (Mayer, 2011). The science of learning is a systematic and empirical approach to understanding how people learn. More formally, Mayer (2011) defined the science of learning as the “scientific study of how people learn” (p. 3). The science of instruction (Mayer 2011), informed in part by the science of learning, is also on display throughout the book. Mayer defined the science of instruction as the “scientific study of how to help people learn” (p. 3). Finally, the assessment of student learning (e.g., learning, remembering, transferring knowledge) during and after instruction helps us determine the effectiveness of our instructional methods. Mayer defined the science of assessment as the “scientific study of how to determine what people know” (p.3). Most of the research and applications presented in this book are completed within a science of learning framework. Researchers first conducted research to understand how people learn in certain controlled contexts (i.e., in the laboratory) and then they, or others, began to consider how these understandings could be applied in educational settings. Work on the cognitive load theory of learning, which is discussed in depth in several chapters of this book (e.g., Chew; Lee and Kalyuga; Mayer; Renkl), provides an excellent example that documents how science of learning has led to valuable work on the science of instruction. Most of the work described in this book is based on theory and research in cognitive psychology. We might have selected other topics (and, thus, other authors) that have their research base in behavior analysis, computational modeling and computer science, neuroscience, etc. We made the selections we did because the work of our authors ties together nicely and seemed to us to have direct applicability in academic settings