Emotion recognition techniques using physiological signals and video games –Systematic review–

Emotion recognition systems from physiological signals are innovative techniques that allow studying the behavior and reaction of an individual when exposed to information that may evoke emotional reactions through multimedia tools, for example, video games. This type of approach is used to identify the behavior of an individual in different fields, such as medicine, education, psychology, etc., in order to assess the effect that the content has on the individual that is interacting with it. This article shows a systematic review of articles that report studies on emotion recognition with physiological signals and video games, between January 2010 and April 2016. We searched in eight databases, and found 15 articles that met the selection criteria. With this systematic review, we found that the use of video games as emotion stimulation tools has become an innovative field of study, due to their potential to involve stories and multimedia tools that can interact directly with the person in fields like rehabilitation. We detected clear examples where video games and physiological signal measurement became an important approach in rehabilitation processes, for example, in Posttraumatic Stress Disorder (PTSD) treatments

Biometric storyboards: a games user research approach for improving qualitative evaluations of player experience

Developing video games is an iterative and demanding process. It is difficult to achieve the goal of most video games — to be enjoyable, engaging and to create revenue for game developers — because of many hard-to-evaluate factors, such as the different ways players can interact with the game. Understanding how players behave during gameplay is of vital importance to developers and can be uncovered in user tests as part of game development. This can help developers to identify and resolve any potential problem areas before release, leading to a better player experience and possibly higher game review scores and sales. However, traditional user testing methods were developed for function and efficiency oriented applications. Hence, many traditional user testing methods cannot be applied in the same way for video game evaluation. This thesis presents an investigation into the contributions of physiological measurements in user testing within games user research (GUR). GUR specifically studies the interaction between a game and users (players) with the aim to provide feedback for developers to help them to optimise the game design of their title. An evaluation technique called Biometric Storyboards is developed, which visualises the relationships between game events, player feedback and changes in a player’s physiological state. Biometric Storyboards contributes to the field of human-computer interaction and GUR in three important areas: (1) visualising mixedmeasures of player experience, (2) deconstructing game design by analysing game events and pace, (3) incremental improvement of classic user research techniques (such as interviews and physiological measurements). These contributions are described in practical case studies, interviews with game developers and laboratory experiments. The results show this evaluation approach can enable games user researchers to increase the plausibility and persuasiveness of their reports and facilitate developers to better deliver their design goals. Biometric Storyboards is not aimed at replacing existing methods, but to extend them with mixed methods visualisations, to provide powerful tools for games user researchers and developers to better understand and communicate player needs, interactions and experiences. The contributions of this thesis are directly applicable for user researchers and game developers, as well as for researchers in user experience evaluation in entertainment systems

Measuring Behavior 2018 Conference Proceedings

These proceedings contain the papers presented at Measuring Behavior 2018, the 11th International Conference on Methods and Techniques in Behavioral Research. The conference was organised by Manchester Metropolitan University, in collaboration with Noldus Information Technology. The conference was held during June 5th – 8th, 2018 in Manchester, UK. Building on the format that has emerged from previous meetings, we hosted a fascinating program about a wide variety of methodological aspects of the behavioral sciences. We had scientific presentations scheduled into seven general oral sessions and fifteen symposia, which covered a topical spread from rodent to human behavior. We had fourteen demonstrations, in which academics and companies demonstrated their latest prototypes. The scientific program also contained three workshops, one tutorial and a number of scientific discussion sessions. We also had scientific tours of our facilities at Manchester Metropolitan Univeristy, and the nearby British Cycling Velodrome. We hope this proceedings caters for many of your interests and we look forward to seeing and hearing more of your contributions

Mediated participatory design for contextually aware in-vehicle user-experiences with autonomous vehicles

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThis study reports on the empirical findings of a series of participatory design workshops for the development of a supportive automotive user experience design system. Identifying and addressing this area with traditional research methods is problematic due to the different user experience (UX) design perspectives that might conflict and the related limitations of the automotive domain. Consequently, we deploy a pragmatic epistemological paradigm and apply participatory prototyping methods to resolve this problem. We conduct two iterations of design and evaluation with 19 user experience (UX) designers through individual participatory prototyping activities to gain insights into their explicit, observable, tacit and latent needs. We describe the design of a toolkit tailored to the character of the study to be used in relevant studies of ill-defined or wicked problems. The participatory design activities initially allowed us to explore the motivation to use different technologies, the system’s architecture, detailed features of interactivity, and to describe our users’ needs. As a result, our first analysis of data led us to design implications that translate participants’ needs into UX goals. We use these UX goals for the design of goal-directed personas and scenarios of use as actionable insights to develop our system. A medium-fidelity functional prototype of our system was then evaluated, while contextually aware automotive UX practitioners criticised our design decisions. Some of the essential findings when supporting the contextual understanding are generating new knowledge to inform both theory and practice. The results propose that most automotive UX designers are ready to adopt technologies that use sensitive physiological measures such as eyes, face, body tracking using cameras and computer vision. In contrast, non-automotive UX designers who empathise with the passengers and the drivers and perceive the in-vehicle space as something more private are suggesting that this might affect people’s trust. The majority agrees to collect data and communicate with the users using implicit and explicit context, as a way to support UX design in the autonomous vehicles would require the consent of the passengers. Even though UX designers suggested a general interest in the social and temporal context of the interactions, the limitations of privacy and safety in the vehicle limit them in collecting task-related contextual data leaving the social, temporal, and physical context unexplored. Safety is arguably a factor that will not restrict the future of autonomous driving experiences research and design since there is no cognitive demand on level five autonomy which hands the passengers with plenty of other options when not driving, assuming that they are ready to trust a fully automated system. However, our study does not provide us with a direction on the privacy of autonomous vehicle experiences and whether privacy will continue being a limitation in the context of self-driving vehicles. Thus, we would recommend further research on trust and privacy in fully automated vehicles. We conclude by discussing the design implications and functional tools of our system, including 1) a video tagging tool that supports saving an occurrence identified momentarily on real-time video. 2) A privacy call-wall which uses implicit and explicit context to avoid intrusiveness in private situations. 3) A human-like avatar tool for mitigating privacy issues, and 4) an interactive interviewing tool to support communication between UXers and the passengers of autonomous vehicles. Finally, 5) exploration tools, including a tool for searching participants’ characteristics and target groups of people. We further inform the body of knowledge in participatory UX and HCI methods about the advantages of our methodological approach and the limitations of using it. We discuss why involving non-experts in co-design activities using toolkits tailored to the domain of interest is valuable. Furthermore, we extensively address how, and we give directions for the design of similar toolkits by describing the toolkit that we designed and applied in our study. Conclusively we discuss the broader implications of trust and privacy in other domains and how this related to our findings

Modelling music selection in everyday life with applications for psychology-informed music recommender systems

Music is a highly functional and utilitarian resource. It enables people to regulate emotions, reduce distractions, stimulate physical action, and connect with others. However, with technologically facilitated ubiquitous listening now commonplace, new problems have emerged. The main problem is that of choice: how, given millions of songs to choose from, should providers curate listening experiences? To resolve this, many online platforms employ recommender systems, and there have been concerted efforts to orientate these systems in such a way that they are responsive to the short-term, dynamic needs of listeners in everyday situations. However, there is increasing scrutiny around the impact of automated recommender systems in terms of interpretability and data usage. To this end, researchers have begun exploring ways of integrating knowledge about user behaviours into the recommendation process, rather than through purely data-driven approaches. This thesis aims to bridge these strands of intrigue by exploring an approach to generating situationally determined recommendations, based on an understanding of how and why contextual factors influence music selection in everyday life. This is achieved through three studies, in which contexts, functions, and content of listeners’ music selections are triangulated to make inferences and estimates of situationally congruent musical characteristics. Firstly, a psychometric structure of the functions of music listening is generated. Secondly, this is triangulated with contextual factors and audio features of music selection. Finally, this is supplemented with an exploratory approach to generating recommendations through the explanatory model. These three studies result in both: a preliminary model of goal-orientated music listening that can be deployed by recommender procedures; and provides an exemplar methodology of how to construct behavioural models that can drive such systems. This thesis therefore holds relevance to both psychological research and those interested in music curation techniques

Collaborative learning with affective artificial study companions in a virtual learning environment

This research has been carried out in conjunction with Chapeltown and Harehills Assisted Learning Computer School (CHALCS) and local schools. CHALCS is an 'out-of-hours' school in a deprived inner-city community where unemployment is high and many children are failing to meet their educational potential. As the name implies CHALCS provides students with access to computers to support their learning. CHALCS relies on many volunteer tutors and specialist tutors are in short supply. This is especially true for subjects such as Advanced Level Physics with low numbers of students. This research aimed to investigate the feasibility of providing online study skills support to pupils at CHALCS and a local school. Research suggests that collaborative learning that prompts students to explain and justify their understanding can encourage deeper learning. As a potentially effective way of motivating deeper learning from hypertext course notes in a Virtual Learning Environment (VLE), this research investigates the feasibility of designing an artificial Agent capable of collaborating with the learner to jointly construct summary notes. Hypertext course notes covering a portion of the Advanced Level Physics curriculum were designed and uploaded into a WebCT based VLE. A specialist tutor validated the content of the course notes before the ease of use of the VLE was tested with target students. A study was then conducted to develop a model of the kinds of help students required in writing summary notes from the course-notes. Based on the derived process model of summarisation and an analysis of the content structure of the course notes, strategies for summarising the text were devised. An Animated Pedagogical Agent was designed incorporating these strategies. Two versions of the agent with opposing 'Affectations' (giving the appearance of different characters) were evaluated with users. It was therefore possible to test which artificial 'character' students preferred. From the evaluation study some conclusions are made concerning the effect of the two opposite characterisations on student perceptions of the agent and the degree to which it was helpful as a learning companion. Some recommendations for future work are then made