Assessment in and of serious games: an overview

There is a consensus that serious games have a significant potential as a tool for instruction. However, their effectiveness in terms of learning outcomes is still understudied mainly due to the complexity involved in assessing intangible measures. A systematic approach—based on established principles and guidelines—is necessary to enhance the design of serious games, and many studies lack a rigorous assessment. An important aspect in the evaluation of serious games, like other educational tools, is user performance assessment. This is an important area of exploration because serious games are intended to evaluate the learning progress as well as the outcomes. This also emphasizes the importance of providing appropriate feedback to the player. Moreover, performance assessment enables adaptivity and personalization to meet individual needs in various aspects, such as learning styles, information provision rates, feedback, and so forth. This paper first reviews related literature regarding the educational effectiveness of serious games. It then discusses how to assess the learning impact of serious games and methods for competence and skill assessment. Finally, it suggests two major directions for future research: characterization of the player's activity and better integration of assessment in games

Designing Sugaropolis:digital games as a medium for conveying transnational narratives

In this paper, the authors present a case study of ‘Sugaropolis’: a two-year practice-based project that involved interdisciplinary co-design and stakeholder evaluation of two digital game prototypes. Drawing on the diverse expertise of the research team (game design and development, human geography, and transnational narratives), the paper aims to contribute to debates about the use of digital games as a medium for representing the past. With an emphasis on design-as-research, we consider how digital games can be (co-)designed to communicate complex histories and geographies in which people, objects, and resources are connected through space and time

Responsible research and innovation in science education: insights from evaluating the impact of using digital media and arts-based methods on RRI values

The European Commission policy approach of Responsible Research and Innovation (RRI) is gaining momentum in European research planning and development as a strategy to align scientific and technological progress with socially desirable and acceptable ends. One of the RRI agendas is science education, aiming to foster future generations' acquisition of skills and values needed to engage in society responsibly. To this end, it is argued that RRI-based science education can benefit from more interdisciplinary methods such as those based on arts and digital technologies. However, the evidence existing on the impact of science education activities using digital media and arts-based methods on RRI values remains underexplored. This article comparatively reviews previous evidence on the evaluation of these activities, from primary to higher education, to examine whether and how RRI-related learning outcomes are evaluated and how these activities impact on students' learning. Forty academic publications were selected and its content analysed according to five RRI values: creative and critical thinking, engagement, inclusiveness, gender equality and integration of ethical issues. When evaluating the impact of digital and arts-based methods in science education activities, creative and critical thinking, engagement and partly inclusiveness are the RRI values mainly addressed. In contrast, gender equality and ethics integration are neglected. Digital-based methods seem to be more focused on students' questioning and inquiry skills, whereas those using arts often examine imagination, curiosity and autonomy. Differences in the evaluation focus between studies on digital media and those on arts partly explain differences in their impact on RRI values, but also result in non-documented outcomes and undermine their potential. Further developments in interdisciplinary approaches to science education following the RRI policy agenda should reinforce the design of the activities as well as procedural aspects of the evaluation research

The seamless integration of Web3D technologies with university curricula to engage the changing student cohort

The increasing tendency of many university students to study at least some courses at a distance limits their opportunities for the interactions fundamental to learning. Online learning can assist but relies heavily on text, which is limiting for some students. The popularity of computer games, especially among the younger students, and the emergence of networked games and game-like virtual worlds offers opportunities for enhanced interaction in educational applications. For virtual worlds to be widely adopted in higher education it is desirable to have approaches to design and development that are responsive to needs and limited in their resource requirements. Ideally it should be possible for academics without technical expertise to adapt virtual worlds to support their teaching needs. This project identified Web3D, a technology that is based on the X3D standards and which presents 3D virtual worlds within common web browsers, as an approach worth exploring for educational application. The broad goals of the project were to produce exemplars of Web3D for educational use, together with development tools and associated resources to support non-technical academic adopters, and to promote an Australian community of practice to support broader adoption of Web3D in education. During the first year of the project exemplar applications were developed and tested. The Web3D technology was found to be still in a relatively early stage of development in which the application of standards did not ensure reliable operation in different environments. Moreover, ab initio development of virtual worlds and associated tools proved to be more demanding of resources than anticipated and was judged unlikely in the near future to result in systems that non-technical academics could use with confidence. In the second year the emphasis moved to assisting academics to plan and implement teaching in existing virtual worlds that provided relatively easy to use tools for customizing an environment. A project officer worked with participating academics to support the teaching of significant elements of courses within Second LifeTM. This approach was more successful in producing examples of good practice that could be shared with and emulated by other academics. Trials were also conducted with ExitRealityTM, a new Australian technology that presents virtual worlds in a web browser. Critical factors in the success of the project included providing secure access to networked computers with the necessary capability; negotiating the complexity of working across education, design of virtual worlds, and technical requirements; and supporting participants with professional development in the technology and appropriate pedagogy for the new environments. Major challenges encountered included working with experimental technologies that are evolving rapidly and deploying new networked applications on secure university networks. The project has prepared the way for future expansion in the use of virtual worlds for teaching at USQ and has contributed to the emergence of a national network of tertiary educators interested in the educational applications of virtual worlds

Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards

Location-based and contextual mobile learning. A STELLAR Small-Scale Study

This study starts from several inputs that the partners have collected from previous and current running research projects and a workshop organised at the STELLAR Alpine Rendevous 2010. In the study, several steps have been taken, firstly a literature review and analysis of existing systems; secondly, mobile learning experts have been involved in a concept mapping study to identify the main challenges that can be solved via mobile learning; and thirdly, an identification of educational patterns based on these examples has been done. Out of this study the partners aim to develop an educational framework for contextual learning as a unifying approach in the field. Therefore one of our central research questions is: how can we investigate, theorise, model and support contextual learning