Affective games:a multimodal classification system

Affective gaming is a relatively new field of research that exploits human emotions to influence gameplay for an enhanced player experience. Changes in player’s psychology reflect on their behaviour and physiology, hence recognition of such variation is a core element in affective games. Complementary sources of affect offer more reliable recognition, especially in contexts where one modality is partial or unavailable. As a multimodal recognition system, affect-aware games are subject to the practical difficulties met by traditional trained classifiers. In addition, inherited game-related challenges in terms of data collection and performance arise while attempting to sustain an acceptable level of immersion. Most existing scenarios employ sensors that offer limited freedom of movement resulting in less realistic experiences. Recent advances now offer technology that allows players to communicate more freely and naturally with the game, and furthermore, control it without the use of input devices. However, the affective game industry is still in its infancy and definitely needs to catch up with the current life-like level of adaptation provided by graphics and animation

The Effects of Displayed Violence and Game Speed in First-Person Shooters on Physiological Arousal and Aggressive Behavior

Many studies have been conducted to examine the effects of displayed violence in digital games on outcomes like aggressive behavior and physiological arousal. However, they often lack a proper manipulation of the relevant factors and control of confounding variables. In this study, the displayed violence and game speed of a recent first-person shooter game were varied systematically using the technique of modding, so that effects could be explained properly by the respective manipulations. Aggressive behavior was measured with the standardized version of the Competitive Reaction Time Task or CRTT (Ferguson et al., 2008}. Physiological arousal was operationalized with four measurements: galvanic skin response (GSR), heart rate (HR), body movement, force on mouse and keyboard. A total of N = 87 participants played in one of four game conditions (low- vs. high-violence, normal- vs. high speed) while physiological measurements were taken with finger clips, force sensors on input devices (mouse and keyboard), and a Nintendo Wii balance board on the chair they sat on. After play, their aggressive behavior was measured with the CRTT. The results of the study do not support the hypothesis that playing digital games increases aggressive behavior. There were no significant differences in GSR and HR, but with a higher game speed, participants showed less overall body movement, most likely to meet the game’s higher demands on cognitive and motor capacities. Also, higher game speed and displayed violence caused an increase in applied force on mouse and keyboard. Previous experience with digital games did not moderate any of these findings. Moreover, it provides further evidence that the CRTT should only be used in a standardized way as a measurement for aggression, if at all. Using all 7 different published (though not validated) ways to calculate levels of aggression from the raw data, “evidence” was found that playing a violent digital game increases, decreases, or does not change aggression at all. Thus, the present study does extend previous research. Firstly, it shows the methodological advantages of modding in digital game research to accomplish the principles of psychological (laboratory) experiments by manipulating relevant variables and controlling all others. It also demonstrates the test-theoretical problems of the highly diverse use of the CRTT. It provides evidence that for a meaningful interpretation of effects of displayed violence in digital games, there are other game characteristics that should be controlled for since they might have an effect on relevant outcome variables. Further research needs to identify more of those game features, and it should also improve the understanding of the different measures for physiological arousal and their interrelatedness

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Multimodal Observation and Interpretation of Subjects Engaged in Problem Solving

In this paper we present the first results of a pilot experiment in the capture and interpretation of multimodal signals of human experts engaged in solving challenging chess problems. Our goal is to investigate the extent to which observations of eye-gaze, posture, emotion and other physiological signals can be used to model the cognitive state of subjects, and to explore the integration of multiple sensor modalities to improve the reliability of detection of human displays of awareness and emotion. We observed chess players engaged in problems of increasing difficulty while recording their behavior. Such recordings can be used to estimate a participant's awareness of the current situation and to predict ability to respond effectively to challenging situations. Results show that a multimodal approach is more accurate than a unimodal one. By combining body posture, visual attention and emotion, the multimodal approach can reach up to 93% of accuracy when determining player's chess expertise while unimodal approach reaches 86%. Finally this experiment validates the use of our equipment as a general and reproducible tool for the study of participants engaged in screen-based interaction and/or problem solving

xDelia final report: emotion-centred financial decision making and learning

xDelia is a 3-year pan-European project building on the knowledge, skills, and competences of seven partner organisations from a variety of research disciplines and from business. The principal objective of xDelia is to develop technology-enhanced learning approaches that help improve the financial decision making of investors who trade frequently using an electronic trading platform. We focus on emotions, and how they affect maladaptive decision biases and trading performance. Our earlier field work with traders has shown that the development of emotion regulation skills is a key facet of trader expertise. For that reason we consider expert traders our benchmark for adaptive behaviour rather than normative rationality. Our goal is to provide investors with the tools and techniques to develop greater self-awareness of internal states, increase their ability to reflect critically on emotion-informed choices, develop emotion management skills, and support the transfer of these skills to the real-world practice setting of financial trading. This report provides a comprehensive overview of what xDelia is about and what we have achieved over the life of the project. In the sections that follow, we explain the decision problems investors are faced with in a fast paced environment and the limitations of traditional approaches to reduce cognitive errors; introduce an alternative, technology-enhanced learning approach of diagnosis and feedback, skill development, and transfer; describe the learning intervention comprising twelve autonomous learning elements that we have developed; and present evidence from thirty-five studies we have conducted on learning effects and stakeholder acceptance

Physiological Self Regulation with Biofeedback Games

Mental stress is a global epidemic that can have serious health consequences including cardiovascular diseases and diabetes. Several techniques are available to teach stress self-regulation skills including therapy, meditation, deep breathing, and biofeedback. While effective, these methods suffer from high drop-outs due to the monotonic nature of the exercises and are generally practiced in quiet relaxed environment, which may not transfer to real-world scenarios. To address these issues, this dissertation presents a novel intervention for stress training using games and wearable sensors. The approach consists of monitoring the user’s physiological signals during gameplay, mapping them into estimates of stress levels, and adapting the game in a way that promotes states of low arousal. This approach offers two key advantages. First, it allows users to focus on the gameplay rather than on monitoring their physiological signals, which makes the training far more engaging. More importantly, it teaches users to self-regulate their stress response, while performing a task designed to increase arousal. Within this broad framework, this dissertation studies three specific problems. First, the dissertation evaluates three physiological signals (breathing rate, heart rate variability, and electrodermal activity) that span across the dimensions of degrees of selectivity in measuring arousal and voluntary control in their effectiveness in lowering arousal. This will identify the signal appropriate for game based stress training and the associated bio-signal processing techniques for real-time arousal estimation. Second, this dissertation investigates different methods of biofeedback presentation e.g. visual feedback and game adaptation during gameplay. Selection of appropriate biofeedback mechanism is critical since it provides the necessary information to improve the perception of visceral states (e.g. stress) to the user. Furthermore, these modalities facilitate skill acquisition in distinct ways (i.e., top-down and bottom-up learning) and influence retention of skills. Third, this dissertation studies reinforcement scheduling in a game and its effect on skill learning and retention. A reinforcement schedule determines which occurrences of the target response are reinforced. This study focuses on continuous and partial reinforcement schedules in GBF and their effect on resistance to extinction (i.e. ability to retain learned skills) after the biofeedback is removed. The main contribution of this dissertation is in demonstrating that stress self-regulation training can be embedded in videogames and help individuals develop more adaptive responses to reduce physiological stress encountered both at home and work

Inside the brain of an elite athlete: The neural processes that support high achievement in sports

Events like the World Championships in athletics and the Olympic Games raise the public profile of competitive sports. They may also leave us wondering what sets the competitors in these events apart from those of us who simply watch. Here we attempt to link neural and cognitive processes that have been found to be important for elite performance with computational and physiological theories inspired by much simpler laboratory tasks. In this way we hope to inspire neuroscientists to consider how their basic research might help to explain sporting skill at the highest levels of performance

Neuroeconomics: How Neuroscience Can Inform Economics

Neuroeconomics uses knowledge about brain mechanisms to inform economic analysis, and roots economics in biology. It opens up the "black box" of the brain, much as organizational economics adds detail to the theory of the firm. Neuroscientists use many tools— including brain imaging, behavior of patients with localized brain lesions, animal behavior, and recording single neuron activity. The key insight for economics is that the brain is composed of multiple systems which interact. Controlled systems ("executive function") interrupt automatic ones. Emotions and cognition both guide decisions. Just as prices and allocations emerge from the interaction of two processes—supply and demand— individual decisions can be modeled as the result of two (or more) processes interacting. Indeed, "dual-process" models of this sort are better rooted in neuroscientific fact, and more empirically accurate, than single-process models (such as utility-maximization). We discuss how brain evidence complicates standard assumptions about basic preference, to include homeostasis and other kinds of state-dependence. We also discuss applications to intertemporal choice, risk and decision making, and game theory. Intertemporal choice appears to be domain-specific and heavily influenced by emotion. The simplified ß-d of quasi-hyperbolic discounting is supported by activation in distinct regions of limbic and cortical systems. In risky decision, imaging data tentatively support the idea that gains and losses are coded separately, and that ambiguity is distinct from risk, because it activates fear and discomfort regions. (Ironically, lesion patients who do not receive fear signals in prefrontal cortex are "rationally" neutral toward ambiguity.) Game theory studies show the effect of brain regions implicated in "theory of mind", correlates of strategic skill, and effects of hormones and other biological variables. Finally, economics can contribute to neuroscience because simple rational-choice models are useful for understanding highly-evolved behavior like motor actions that earn rewards, and Bayesian integration of sensorimotor information