Validity Threats in Quantitative Data Collection with Games : A Narrative Survey

Background. Games are increasingly used to collect scientific data. Some suggest that game features like high cognitive load may limit the inferences we can draw from such data, yet no systematic overview exists of potential validity threats of game-based methods. Aim. We present a narrative survey of documented and potential threats to validity in using games for quantitative data collection. Method. We combined an unsystematic bottom-up literature review with a systematic top-down application of standard validity threat typologies to games to arrive at a systematisation of game-characteristic validity threats. Results. We identify three game characteristics that potentially impact validity: Games are complex systems, impeding the predictable control and isolation of treatments. They are rich in unwanted variance and diversity. And their social framing can differ from and interact with the framing of research studies or non-game situations they are supposed to represent. The diversity of gamers and their differences to general populations bring further complications. Discussion and Conclusions. The wealth of potential validity threats in game-based research is met by a dearth of systematic methodological studies, leading us to outline several future research directions

Interactive Feedforward for Improving Performance and Maintaining Intrinsic Motivation in VR Exergaming

Exergames commonly use low to moderate intensity exercise protocols. Their effectiveness in implementing high intensity protocols remains uncertain. We propose a method for improving performance while maintaining intrinsic motivation in high intensity VR exergaming. Our method is based on an interactive adaptation of the feedforward method: a psychophysical training technique achieving rapid improvement in performance by exposing participants to self models showing previously unachieved performance levels. We evaluated our method in a cycling-based exergame. Participants competed against (i) a self model which represented their previous speed; (ii) a self model representing their previous speed but increased resistance therefore requiring higher performance to keep up; or (iii) a virtual competitor at the same two levels of performance. We varied participants' awareness of these differences. Interactive feedforward led to improved performance while maintaining intrinsic motivation even when participants were aware of the interventions, and was superior to competing against a virtual competitor

Multi-Site Data Collection and Evaluation in Spoken Language Understanding

The Air Travel Information System (ATIS) domain serves as the common task for DARPA spoken language system research and development. The approaches and results possible in this rapidly growing area are structured by available corpora, annotations of that data, and evaluation methods. Coordination of this crucial infrastructure is the charter of the Multi-Site ATIS Data COllection Working group (MADCOW) . We focus here on selection of training and test data, evaluation of language understanding, and the continuing search for evaluation methods that will correlate well with expected performance of the technology in applications. 1. Introduction Data availability and evaluation procedures structure research possibilities: the type and amount of training data affects the performance of existing algorithms and limits the development of new algorithms; and evaluation procedures document progress, and force research choices in a world of limited resources. The recent rapid progress in spoke..

Game Development for Computer Science Education

Educators have long used digital games as platforms for teaching. Games have several qualities that aren't typically found in traditional homework: they tend to situate problems within a compelling alternate reality that unfolds through entertaining narrative, they draw upon a player's intrinsic motivations more than extrinsic ones, they facilitate deliberate low-intensity practice, and they emphasize a spirit of play instead of work. A quick survey of the Internet and mobile app stores reveals that mathematics and language educators have definitely capitalized on using games as a medium for learning. Ironically, computer science educators, individuals who often teach software development skills that can be used to make games, have relatively few learning games at their disposal. We therefore propose a working group centered on developing freely available games to teach computer science concepts