Engaging Children in Interactive Application Evaluation

nteractive applications designed specifically for children offer great potential for education and play. However, to ascertain that the aims of applications are achieved, child-centred evaluations must be conducted. The design of any evaluation with children requires significant consideration of potential problems with comprehension, cognitive ability, response biases and study attrition. Multidisciplinary R&D project evaluation requirements are often extensive, requiring an all-encompassing and prolonged evaluation design. Discontinuity between the highly engaging interaction experience and themultitude of measures that form the evaluation poses a major issue for the evaluation of interactive applications. In response, we have developed Transmedia Evaluation, a method that aims to maintain engagement throughout the evaluation process. In this paper, the Transmedia Evaluation process is explained and applied to evaluate a learning application for children, MIXER (Moderating Interactions for Cross Cultural Empathic Relationships). Children aged 9-11 (N = 117) used the MIXER application and completed an evaluation battery including pre- and posttest questionnaires, immediate learning assessment and qualitative evaluation. Using Transmedia Evaluation to develop the MIXER evaluation resulted in complete data-sets (100%) for quantitative data (by self-regulated completion) along with rich, high quality qualitative responses. Transmedia Evaluation transformed the evaluation, with children fully engaging in and enjoying their experience. Engaging Children in Interactive Application Evaluation (PDF Download Available). Available from: https://www.researchgate.net/publication/276422985_Engaging_Children_in_Interactive_Application_Evaluation [accessed Feb 8, 2016]

Leadership Curriculum Evaluation: Final Report DFE- RR639

Evaluation report of the National College for Teaching and Leadership's Leadership Curriculum commissioned by Department for Education (2013-2016

Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children

This paper focuses on achieving optimal responses through supporting children’s judgements, using Smiley Face Likert scales as a rating scale for quantitative questions in evaluations. It highlights the need to provide appropriate methods for children to communicate judgements, highlighting that the traditional Smiley Face Likert scale does not provide an appropriate method. The paper outlines a range of studies, identifying that to achieve differentiated data and full use of rating scales by children that faces with positive emotions should be used within Smiley Face Likert scales. The proposed rating method, the Five Degrees of Happiness Smiley Face Likert scale, was used in a large-scale summative evaluation of a Serious Game resulting in variance within and between children, with all points of the scale used

Serious Game Evaluation as a Meta-game

Purpose – This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach – In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team. Findings – The key result from this study was that during the “seamless evaluation” approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in-role game experience. Practical implications – This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user. Originality/value – Through using seamless evaluation, the authors created an evaluation completely embedded within the “magic circle” of an in-game experience that added value to the user experience whilst also yielding relevant results for the development team

Werewolves, cheats, and cultural sensitivity

This paper discusses the design and evaluation of the system MIXER (Moderating Interactions for Cross-Cultural Empathic Relationships), which applies a novel approach to the education of children in cultural sensitivity. MIXER incorporates intelligent affective and interactive characters, including a model of a Theory of Mind mechanism, in a simulated virtual world. We discuss the relevant pedagogical approaches, related work, the underlying mind model used for MIXER agents as well as its innovative interaction interface utilising a tablet computer and a pictorial interaction language. We then consider the evaluation of the system, whether this shows it met its pedagogical objectives, and what can be learned from our results.</p