Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children

This paper focuses on achieving optimal responses through supporting children’s judgements, using Smiley Face Likert scales as a rating scale for quantitative questions in evaluations. It highlights the need to provide appropriate methods for children to communicate judgements, highlighting that the traditional Smiley Face Likert scale does not provide an appropriate method. The paper outlines a range of studies, identifying that to achieve differentiated data and full use of rating scales by children that faces with positive emotions should be used within Smiley Face Likert scales. The proposed rating method, the Five Degrees of Happiness Smiley Face Likert scale, was used in a large-scale summative evaluation of a Serious Game resulting in variance within and between children, with all points of the scale used

Engaging Children in Interactive Application Evaluation

nteractive applications designed specifically for children offer great potential for education and play. However, to ascertain that the aims of applications are achieved, child-centred evaluations must be conducted. The design of any evaluation with children requires significant consideration of potential problems with comprehension, cognitive ability, response biases and study attrition. Multidisciplinary R&D project evaluation requirements are often extensive, requiring an all-encompassing and prolonged evaluation design. Discontinuity between the highly engaging interaction experience and themultitude of measures that form the evaluation poses a major issue for the evaluation of interactive applications. In response, we have developed Transmedia Evaluation, a method that aims to maintain engagement throughout the evaluation process. In this paper, the Transmedia Evaluation process is explained and applied to evaluate a learning application for children, MIXER (Moderating Interactions for Cross Cultural Empathic Relationships). Children aged 9-11 (N = 117) used the MIXER application and completed an evaluation battery including pre- and posttest questionnaires, immediate learning assessment and qualitative evaluation. Using Transmedia Evaluation to develop the MIXER evaluation resulted in complete data-sets (100%) for quantitative data (by self-regulated completion) along with rich, high quality qualitative responses. Transmedia Evaluation transformed the evaluation, with children fully engaging in and enjoying their experience. Engaging Children in Interactive Application Evaluation (PDF Download Available). Available from: https://www.researchgate.net/publication/276422985_Engaging_Children_in_Interactive_Application_Evaluation [accessed Feb 8, 2016]

Enhancing Questionnaire Design Through Participant Engagement to Improve the Outputs of Evaluation.

Questionnaires are habitual choices for many user experience evaluators, providing a well-recognised and accepted, fast and cost effective method of collecting and analysing data. However, despite frequent and widespread use in evaluation, reliance on questionnaires can be problematic. Satisficing, acquiescence bias and straight lining are common response biases associated with questionnaires, typically resulting in suboptimal responses and provision of poor quality data. These problems can relate to a lack of engagement with evaluation tasks, yet there is a lack of previous research that has attempted to alleviate these limitations by making questionnaires more fun or enjoyable to enhance participant engagement. This research seeks to address whether ‘user evaluation questionnaires can be designed to be engaging to improve optimal responding. The aim of this research is to investigate if response quality can be improved through enhancing questionnaire design both to reduce common response biases and to maintain participant engagement. The evaluation context for this study was provided by MIXER, an interactive, narrative-based application for intercultural sensitivity learning, used and evaluated by 9-11 year old children in the classroom context. A series of Participatory Design studies with children investigated engagement and optimal responding with questionnaires. These initial studies informed the design of a series of questionnaires created in the form of three workbooks that were used to evaluate MIXER with over 400 children. 3 A mixed methods approach was used to evaluate the questionnaires. Results demonstrate that by making questionnaire completion more enjoyable data quality is improved. Response biases are reduced, quantitative data are more complete and qualitative responses are more verbose and meaningful compared to standard questionnaires. Further, children reported that completing the questionnaires was a fun and enjoyable activity that they would wish to repeat in the future. As a discipline in its own right, evaluation is under-investigated. Similarly user evaluation is not evaluated with a lack of papers considering this issue in this millennium. Thus, this research provides a significant contribution to the field of evaluation, highlighting that the outputs of user evaluation with questionnaires are improved when participant engagement informs questionnaire design. The result is a more positive evaluation experience for participants and in return a higher standard of data provision for evaluators and R&D teams

Bone marrow CD169+ macrophages promote the retention of hematopoietic stem and progenitor cells in the mesenchymal stem cell niche

Hematopoietic stem cells (HSCs) reside in specialized bone marrow (BM) niches regulated by the sympathetic nervous system (SNS). Here, we have examined whether mononuclear phagocytes modulate the HSC niche. We defined three populations of BM mononuclear phagocytes that include Gr-1hi monocytes (MOs), Gr-1lo MOs, and macrophages (MΦ) based on differential expression of Gr-1, CD115, F4/80, and CD169. Using MO and MΦ conditional depletion models, we found that reductions in BM mononuclear phagocytes led to reduced BM CXCL12 levels, the selective down-regulation of HSC retention genes in Nestin+ niche cells, and egress of HSCs/progenitors to the bloodstream. Furthermore, specific depletion of CD169+ MΦ, which spares BM MOs, was sufficient to induce HSC/progenitor egress. MΦ depletion also enhanced mobilization induced by a CXCR4 antagonist or granulocyte colony-stimulating factor. These results highlight two antagonistic, tightly balanced pathways that regulate maintenance of HSCs/progenitors in the niche during homeostasis, in which MΦ cross talk with the Nestin+ niche cell promotes retention, and in contrast, SNS signals enhance egress. Thus, strategies that target BM MΦ hold the potential to augment stem cell yields in patients that mobilize HSCs/progenitors poorly

Why numbers, invites and visits are not enough: Evaluating the user experience in Social Eco-Systems

Social eco-systems are often evaluated through quantitative data that is automatically logged and analysed. However, where the user’s experience of social eco-systems is evaluated, more explicit intervention approaches are typical, with questionnaires, focus groups and user testing widely used, directly asking the user about their experience. User experience evaluation thus ruptures the social eco-system, occurring as a separate, discrete activity outside of that system. In this paper, we propose that evaluation should be part of the social eco-system adding value to the user experience. We outline an evaluation approach that has been applied within games-based learning environments where the evaluation is seamlessly embedded. We briefly outline our approach to generating and analyzing data highlighting its potential for social eco-system evaluation

Data collection in zones of violence and conflict

A panel session structured around a dialogue between the relative merits and limitations of quantitative and qualitative data collection methods in zones of violence and armed conflict and the potentiality for more mixed methods collaborations