Evaluating First Experiences with an Educational Computer Game: A multi-Method Approach

This paper presents our evaluation approach for a specific case study, namely the evaluation of an early prototype of an educational game with children aged between 12 and 14 years. The main goal of this initial evaluation study was to explore children’s first impressions and experiences of the game on the one hand and to assess the students’ ideas and wishes for the further development of the game on the other hand. The main challenge for the evaluation activities was the selection of the appropriate methodological approach, taking into account children as a special user group. We opted for a combination of different, mainly qualitative and explorative methods that were reported beneficial for work with children in the human-computer interaction (HCI) field. By presenting our multi-method approach, in particular the different steps and procedure within our study, other researchers can get inspirations for follow up activities when evaluating games with children as well as benefit from our experiences in exploring more collaborative methods and methodological combinations

Serious interface design for dental health: Wiimote-based tangible interaction for school children

This paper describes a camera-based approach towards creating a tangible interface for serious games. We introduce our game for dental health targeted at school children which implements the Nintendo WiiMote as infrared camera. Paired with a gesture-recognition system, this combination allows us to apply real-world items as input devices. Thereby, the game tries to address different aspects of dental hygiene along with the improvement of children's motor skills. In our focus group test, we found that tangible interfaces offer great potential for educational purposes and can be used to engage kids in a playful learning process by addressing their childlike curiosity and fostering implicit learning

A Virtual Conversational Agent for Teens with Autism: Experimental Results and Design Lessons

We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere, anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in real-time, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably: 1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations. 2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility

SmallTalk: Using tangible interactions to gather feedback from children

Gathering opinions from young children is challenging and different methods have been explored. In this paper we investigated how tangible devices can be used to gather feedback from children in the context of a theater performance. We introduce SmallTalk, a tangible survey system designed for use within a theater space to capture what children, aged 4 to 9, thought of a live performance they had just seen. We describe how the system was designed to build on previous feedback methods that had been tried; while at the same time meeting the constraints of the challenging theater context. We present results from seven deployments of SmallTalk and based on these we briefly discuss its value as a method for evaluating the theater performance. We then look at how the results validated the system design and present several design implications that more generally relate to tangible feedback systems for children

Emotional recognition in computing

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University 8/4/2010.Emotions are fundamental to human lives and decision-making. Understanding and expression of emotional feeling between people forms an intricate web. This complex interactional phenomena, is a hot topic for research, as new techniques such as brain imaging give us insights about how emotions are tied to human functions. Communication of emotions is mixed with communication of other types of information (such as factual details) and emotions can be consciously or unconsciously displayed. Affective computer systems, using sensors for emotion recognition and able to make emotive responses are under development. The increased potential for emotional interaction with products and services, in many domains, is generating much interest. Emotionally enhanced systems have potential to improve human computer interaction and so to improve how systems are used and what they can deliver. They may also have adverse implications such as creating systems capable of emotional manipulation of users. Affective systems are in their infancy and lack human complexity and capability. This makes it difficult to assess whether human interaction with such systems will actually prove beneficial or desirable to users. By using experimental design, a Wizard of Oz methodology and a game that appeared to respond to the user’s emotional signals with human-like capability, I tested user experience and reactions to a system that appeared affective. To assess users’ behaviour, I developed a novel affective behaviour coding system called ‘affectemes’. I found significant gains in user satisfaction and performance when using an affective system. Those believing the system responded to emotional signals blinked more frequently. If the machine failed to respond to their emotional signals, they increased their efforts to convey emotion, which might be an attempt to ‘repair’ the interaction. This work highlights how very complex and difficult it is to design and evaluate affective systems. I identify many issues for future work, including the unconscious nature of emotions and how they are recognised and displayed with affective systems; issues about the power of emotionally interactive systems and their evaluation; and critical ethical issues. These are important considerations for future design of systems that use emotion recognition in computing.EPSRC project grant (R81374/01

Visitor engagement and learning behaviour in science centres, zoos and aquaria

The purpose of this research was to devise an assessment tool to effectively capture the nature of visitors' learning experiences with live animal exhibits in zoos and aquaria. A comprehensive learning framework was developed and field-tested with a total of 900 visitor. The resulting framework provides researchers and practitioners in zoos and aquaria with a valuable tool to assess the learning impact of exhibits through observable behavioural indicators

'Video-View-Point' - Video analysis to reveal tacit indicators of student nurse competence

For over 30 years, the assessment of the clinical competence of student nurses has been the subject of much theoretical debate, yet the definition of criteria based on observable indicators of competence remains problematic. In practice, however, different assessors will judge and agree, relatively quickly, whether a student is competent or not; whether they have got ‘it’. Articulating what ‘it’ is, is difficult; although ‘it’ appears to be collectively, yet tacitly, understood. These judgements provide the key to the definition of competence. This research solves the dilemma of revealing and investigating these tacit understandings through the video analysis of students in simulated practice. The findings of four initial exploratory studies confirmed that competence is an example of tacitly understood behaviour and identified the limitations of traditional research methods in this context. The practical challenges of analysing video were highlighted, leading to the development of Video-View-Point to solve these problems and to reveal the tacitly understood behaviours. This innovative hybrid research method combines analysis of multiple ‘Think Aloud’ commentaries with the ability to ‘point’ at the subject of interest. The analysis is presented as a time-stamped multimedia dialectic, a visually simple yet sophisticated collage of data which reveals relevant behaviours, including those which are tacitly understood. A bespoke software tool (BigSister) was designed to facilitate the data collection, and was tested against the most similar commercially available technology, an eye tracker. The test of Video-View-Point successfully revealed four tacitly understood indicators of competence: communication, processing clinical information, being in the right place, and being proactive. Video-View-Point offers huge potential for behavioural analysis in other domains