A systematic literature review of methodology used to measure effectiveness in digital game-based learning

In recent years, a growing number of studies is being conducted into the effectiveness of digital game-based learning (DGBL). Despite this growing interest, however, it remains difficult to draw general conclusions due to the disparities in methods and reporting. Guidelines or a standardized procedure for conducting DGBL effectiveness research would allow to compare results across studies and provide well-founded and more generalizable evidence for the impact of DGBL. This study presents a first step in this process by mapping current practices through a systematic literature review. The review included peer-reviewed journal and conference publications between 2000 and 2012. Other inclusion criteria were that (1) the study’s primary aim was effectiveness measurement of cognitive learning outcomes, (2) the focus was on digital games and (3) a pre-post design with a control group was used. Twenty-five publications were found eligible for this study. Important differences were found in the number of control groups used and the type of intervention implemented in the control group (e.g. traditional classroom teaching, use of multimedia, computer-based learning, paper exercises, other games, or no intervention). Regarding the implementation method of the DGBL intervention in the experimental group, two approaches can be distinguished: stand-alone intervention or as part of a larger program. Moreover, a wide variety of effectiveness measures was used: measures for learning outcomes were complemented with time measurements and/or with self-reported measurements for self-efficacy and motivation. Learning effect calculation also varied, introducing pre-test scores in the analysis, conducting a separate analysis on pre- and post-test scores or conducting an analysis on difference scores. Our study thus indicates that a variety of methods is being used in DGBL effectiveness research opening a discussion regarding the potential and requirements for future procedural guidelines

Learning math as you play: comparing arithmetic performance enhancement induced by game play and paper exercises

One of the promises of video game training is that, compared to traditional training, it can be more engaging and entertaining (Boot et. al., 2008). However, besides entertainment, games have shown to have the potential to impact a larger variety of cognitive abilities. Previous research has consistently shown that several aspects in cognition such as visual short-memory, multitasking and spatial cognition can be enhanced by game play. In a previous study, we found that playing Monkey Tales, a commercial game aimed at training arithmetic skills in children, helped second grade pupils to increase their accuracy in mental calculation as compared to paper exercises or no exercises. The present study aimed to explore how arithmetic performance enhancement induced by game play and paper exercises differs. In order to do this, we compared the performance gains that second graders achieved in a computer test made for assessing their math skills. We performed a combined analysis of the changes in two behavioral measurements: accuracy and reaction times. Children were tested at two points in time: before and after the three week period. We compared the reaction times and the accuracy improvements between these two moments and compared different items types (e. g. understanding tenths, understanding hundreds, even or odd up to 100 among other types). We found indirect evidence suggesting that arithmetic performance enhancement induced by game play and paper exercises might rely on slightly different cognitive mechanisms

Pre-test session impact on the effectiveness assessment of a fire safety game

In recent years, critiques have been formulated regarding current evaluation methods of DGBL (digital game-based learning) effectiveness, putting the validity of certain results in doubt. An important point of discussion in DGBL effectiveness studies is whether or not a pre-test should be administered, as it can lead to practice effects and pre-test sensitization, threatening internal validity of the results. The present study aims at testing if the administration of a pre-test has a direct influence on post-test scores and/or makes participants more receptive to the intervention. For this purpose, an effectiveness study of a fire safety training in a hospital was conducted using a Solomon four-group design. The experimental groups received a game-based intervention (n= 65) of which 34 participants received a pre-test and 31 did not. The control groups received traditional classroom instruction (n=68), of which 39 participants received a pre-test and 29 did not. A 2x2 ANOVA was used to explore the practice effect and the interaction between the pre-test and the intervention. An interaction effect between pre-test and intervention is detected. More specifically, this interaction takes place in the traditional classroom group, indicating pre-test sensitization. In the traditional classroom context, the pre-test makes the participants more sensitive to the content treated in the intervention while administration of a pre-test does not influence outcomes of the DGBL treatment. When the administration of a pre-test influences the control group's receptivity to the treatment, but not the experimental group, results of an effectiveness study may be biased. This is especially relevant in the DGBL field as often, non-significant differences between DGBL and more traditional methods are reported. Therefore, further research should take this into account and look for possible solutions to solve this discrepancy

A branch-and-price algorithm for the aperiodic multi-period service scheduling problem

This paper considers the multi-period service scheduling problem with an aperiodic service policy. In this problem, a set of customers who periodically require service over a finite time horizon is given. To satisfy the service demands, a set of operators is given, each with a fixed capacity in terms of the number of customers that can be served per period. With an aperiodic policy, customers may be served before the period were the service would be due. Two criteria are jointly considered in this problem: the total number of operators, and the total number of ahead-of-time periods. The task is to determine the service periods for each customer in such a way that the service requests of the customers are fulfilled and both criteria are minimized. A new integer programming formulation is proposed, which outperforms an existing formulation. Since the computational effort required to obtain solutions considerably increases with the size of the instances, we also present a reformulation suitable for column generation, which is then integrated within a branch-and-price algorithm. Computational experiments highlight the efficiency of this algorithm for the larger instances.Peer ReviewedPostprint (author's final draft

Heating up the Baryonic Branch with U-duality: a unified picture of conifold black holes

We study different aspects of a U-duality recently presented by Maldacena and Martelli and apply it to non-extremal backgrounds. In particular, starting from new non-extremal wrapped D5 branes we generate new non-extremal generalizations of the Baryonic Branch of the Klebanov-Strassler solution. We also elaborate on different conceptual aspects of these U-dualities, like its action on (extremal and non-extremal) Dp branes, dual models for Yang-Mills-like theories, generic asymptotics and decoupling limit of the generated solutions.Comment: 20 pages plus various appendices. Ten figures. Version to appear in JHE

Towards a new ITU-T recommendation for subjective methods evaluating gaming QoE

This paper reports on activities in Study Group 12 of the International Telecommunication Union (ITU-T SG12) to define a new Recommendation on subjective evaluation methods for gaming Quality of Experience (QoE). It first resumes the structure and content of the current draft which has been proposed to ITU-T SG12 in September 2014 and then critically discusses potential gaming content and evaluation methods for inclusion into the upcoming Recommendation. The aim is to start a discussion amongst experts on potential evaluation methods and their limitations, before finalizing a Recommendation. Such a recommendation might in the end be applied by non -expert users, hence wrong decisions in the evaluation design could negatively affect gaming QoE throughout the evaluation