Serious Game Evaluation as a Meta-game

Purpose – This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach – In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team. Findings – The key result from this study was that during the “seamless evaluation” approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in-role game experience. Practical implications – This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user. Originality/value – Through using seamless evaluation, the authors created an evaluation completely embedded within the “magic circle” of an in-game experience that added value to the user experience whilst also yielding relevant results for the development team

'I play, therefore I learn?' Measuring the Evolution of Perceived Learning and Game Experience in the Design Flow of a Serious Game

This article explores how the serious game Poverty Is Not a Game (PING) is experienced by high school students in its subsequent design stages. We first focus on the multifaceted construct of game experience and how it is related to serious games. To measure game experience we use the Game Experience Questionnaire and add a perceived learning scale to account for the specificity of serious games in a classroom. Next, the data obtained from testing PING in 22 classrooms are analyzed. Results suggest that the evolution in the different design stages of the game is not just an issue of game experience, but also of usability. Furthermore, little evidence is found indicating that the learning experience changed positively during the different test phases. However, findings show a strong effect of the game experience on perceived learning while the game experience also varies significantly between different classrooms

Validation of Serious Games Attributes Using the Technology Acceptance Model

The paper introduces a conceptual model for the design of serious games and uses the Technology Acceptance Model (TAM) for its validation. A specially developed game introduced international students to public transport in Southampton. After completing the game, participants completed a short questionnaire and the data was analysed using structural equation modelling (SEM). The results identified the attributes and combinations of attributes that led the learner to accept and to use the serious game for learning. These findings are relevant in helping game designers and educational practitioners design serious games for effective learning

Application of serious games to sport, health and exercise

Use of interactive entertainment has been exponentially expanded since the last decade. Throughout this 10+ year evolution there has been a concern about turning entertainment properties into serious applications, a.k.a "Serious Games". In this article we present two set of Serious Game applications, an Environment Visualising game which focuses solely on applying serious games to elite Olympic sport and another set of serious games that incorporate an in house developed proprietary input system that can detect most of the human movements which focuses on applying serious games to health and exercise

A Conceptual Teacher-Learner Model for a Collaborative Learning with Serious Games

This paper introduces a conceptual Teacher-Learner framework for a collaborative learning with serious games. An initial study identified twelve attributesof educational serious games that can be used to support effective learning. These attributes are used in the conceptual framework to support learning and pedagogy in combination with a game. A considerable number of serious games have been developed over the last ten years, with varying degrees of success. Due to a lack of clear standards and guidelines for game developers; it is difficult to justify claims that a specific game meets the learner’s requirements and/or expectations. This paper defines a conceptual model for serious games that will contribute to their design and the measurement of achievement in meeting the learners’ requirements

Analysis of the Usefulness of a Serious Game to Raise Awareness about Mental Health Problems in a Sample of High School and University Students: Relationship with Familiarity and Time Spent Playing Video Games

Background: One of the main challenges in the field of mental health today is the stigma towards individuals who have psychological disorders. Aims: This study aims to analyse the usefulness of applying a serious game developed for the purpose of raising awareness among students about mental health problems and analyse whether its usefulness can be influenced by the type of video games or the time that students usually devote to playing with this type of entertainment. Method: The serious game introduces four characters who display the symptoms of different psychological disorders. A total of 530 students participated in the study, 412 of whom comprised the experimental group and 118 the control group, 291 came from secondary school classes and 239 were university students. Results: The findings show that this serious game significantly reduced total stigma among students. Variables like time habitually spent playing video games or video game preference had no bearing on the results. Conclusion: Our findings suggest that the serious game is an appropriated tool to reduce stigma, both in high school and university students, independently of the type of video games that young people usually play, or time spent playing video games

Tabletop prototyping of serious games for ‘soft skills’ training

Serious games offer a relatively low cost, highly engaging alternative to traditional forms of soft skills training. The current paper describes an approach taken to designing a serious game for the training of soft skills. A tabletop prototype of the game was created and evaluated with a group of 24 participants. Initial findings suggest that the game successfully created an environment in which it was advantageous to engage in appropriate collaborative decision making behaviors, as well as providing built-in opportunities for a tutor to guide under-performing groups

Is participatory design associated with the effectiveness of serious digital games for healthy lifestyle promotion? : a meta-analysis

Background: Serious digital games can be effective at changing healthy lifestyles, but large differences in their effectiveness exist. The extent of user involvement in game design may contribute to game effectiveness by creating a better fit with user preferences. Participatory design (PD), which represents active user involvement as informant (ie, users are asked for input and feedback) or codesigner (ie, users as equal partners in the design) early on and throughout the game development, may be associated with higher game effectiveness, as opposed to no user involvement or limited user involvement. Objective: This paper reports the results of a meta-analysis examining the moderating role of PD in the effectiveness of serious digital games for healthy lifestyle promotion. Methods: Four databases were searched for peer-reviewed papers in English that were published or in press before October 2014, using a (group-) randomized controlled trial design. Effectiveness data were derived from another meta-analysis assessing the role of behavior change techniques and game features in serious game effectiveness. Results: A total of 58 games evaluated in 61 studies were included. As previously reported, serious digital games had positive effects on healthy lifestyles and their determinants. Unexpectedly, PD (g=0.075, 95% CI 0.017 to 0.133) throughout game development was related to lower game effectiveness on behavior (Q=6.74, P<.05) than when users were only involved as testers (g=0.520, 95% CI 0.150 to 0.890, P<.01). Games developed with PD (g=0.171, 95% CI 0.061 to 0.281, P<.01) were also related to lower game effectiveness on self-efficacy (Q=7.83, P<.05) than when users were not involved in game design (g=0.384, 95% CI 0.283 to 0.485, P<.001). Some differences were noted depending on age group, publication year of the study, and on the specific role in PD (ie, informant or codesigner), and depending on the game design element. Games developed with PD were more effective in changing behavioral determinants when they included users in design elements on game dynamics (beta=.215, 95% CI .075 to .356, P<.01) and, more specifically, as an informant (beta=.235, 95% CI .079 to .329, P<.01). Involving users as informants in PD to create game levels was also related to higher game effectiveness (Q=7.02, P<.01). Codesign was related to higher effectiveness when used to create the game challenge (Q=11.23, P<.01), but to lower game effectiveness when used to create characters (Q=4.36, P<.05) and the game world (Q=3.99, P<.05). Conclusions: The findings do not support higher effectiveness of games developed with PD. However, significant differences existed among PD games. More support was found for informant roles than for codesign roles. When PD was applied to game dynamics, levels, and game challenge, this was associated with higher effectiveness than when it was applied to game aesthetics. Since user involvement may have an important influence on reach, adoption, and implementation of the intervention, further research and design efforts are needed to enhance effectiveness of serious games developed with PD