Affective games:a multimodal classification system

Affective gaming is a relatively new field of research that exploits human emotions to influence gameplay for an enhanced player experience. Changes in player’s psychology reflect on their behaviour and physiology, hence recognition of such variation is a core element in affective games. Complementary sources of affect offer more reliable recognition, especially in contexts where one modality is partial or unavailable. As a multimodal recognition system, affect-aware games are subject to the practical difficulties met by traditional trained classifiers. In addition, inherited game-related challenges in terms of data collection and performance arise while attempting to sustain an acceptable level of immersion. Most existing scenarios employ sensors that offer limited freedom of movement resulting in less realistic experiences. Recent advances now offer technology that allows players to communicate more freely and naturally with the game, and furthermore, control it without the use of input devices. However, the affective game industry is still in its infancy and definitely needs to catch up with the current life-like level of adaptation provided by graphics and animation

Psychophysiology in games

Psychophysiology is the study of the relationship between psychology and its physiological manifestations. That relationship is of particular importance for both game design and ultimately gameplaying. Players’ psychophysiology offers a gateway towards a better understanding of playing behavior and experience. That knowledge can, in turn, be beneficial for the player as it allows designers to make better games for them; either explicitly by altering the game during play or implicitly during the game design process. This chapter argues for the importance of physiology for the investigation of player affect in games, reviews the current state of the art in sensor technology and outlines the key phases for the application of psychophysiology in games.The work is supported, in part, by the EU-funded FP7 ICT iLearnRWproject (project no: 318803).peer-reviewe

Fusion of heart rate variability and salivary cortisol for stress response identification based on adverse childhood experience

Adverse childhood experiences have been suggested to cause changes in physiological processes and can determine the magnitude of the stress response which might have a significant impact on health later in life. To detect the stress response, biomarkers that represent both the Autonomic Nervous System (ANS) and Hypothalamic-Pituitary-Adrenal (HPA) axis are proposed. Among the available biomarkers, Heart Rate Variability (HRV) has been proven as a powerful biomarker that represents ANS. Meanwhile, salivary cortisol has been suggested as a biomarker that reflects the HPA axis. Even though many studies used multiple biomarkers to measure the stress response, the results for each biomarker were analyzed separately. Therefore, the objective of this study is to propose a fusion of ANS and HPA axis biomarkers in order to classify the stress response based on adverse childhood experience. Electrocardiograph, blood pressure (BP), pulse rate (PR), and salivary cortisol (SCort) measures were collected from 23 healthy participants; 11 participants had adverse childhood experience while the remaining 12 acted as the no adversity control group. HRV was then computed from the ECG and the HRV features were extracted. Next, the selected HRV features were combined with the other biomarkers using Euclidean distance (ed) and serial fusion, and the performance of the fused features was compared using Support Vector Machine. From the result, HRV-SCort using Euclidean distance achieved the most satisfactory performance with 80.0% accuracy, 83.3% sensitivity, and 78.3% specificity. Furthermore, the performance of the stress response classification of the fused biomarker, HRV-SCort, outperformed that of the single biomarkers: HRV (61% Accuracy), Cort (59.4% Accuracy), BP (78.3% accuracy), and PR (53.3% accuracy). From this study, it was proven that the fused biomarkers that represent both ANS and HPA (HRV-SCort) able to demonstrate a better classification performance in discriminating the stress response. Furthermore, a new approach for classification of stress response using Euclidean distance and SVM named as ed-SVM was proven to be an effective method for the HRV-SCort in classifying the stress response from PASAT. The robustness of this method is crucial in contributing to the effectiveness of the stress response measures and could further be used as an indicator for future health

Stress response index for traumatic childhood experience based on the fusion of hypothalamus pituitary adrenocorticol and autonomic nervous system biomarkers

Stress occurring in the early days of an individual was often assumed to cause several health consequences. A number of reports indicated that having to deal with unfavourable events or distress situation at a young age could tweak stress responses leading to a broad spectrum of poor mental and physical health condition. Therefore, changes identified within stress response were recommended to be taken as a measure in regulating and managing such health situation. This study combines the biomarker that represents both autonomic nervous system (ANS) and hypothalamic-pituitary-adrenocorticol (HPA) as a single measure to classify the stress response based on traumatic childhood experience and propose a stress response index as a future health indicator. Electrocardiograph (ECG), blood pressure, pulse rate and salivary cortisol (SCort) were collected from 12 participants who had traumatic childhood experience while the remaining 11 acted as the control group. The recording session was done during a Paced Auditory Serial Addition Test (PASAT). HRV was then computed from the ECG and the HRV features were extracted. Next, the best HRV features were selected using Genetic Algorithm (GA). Biomarkers such as BP, PR and SCort were then integrated with 12 HRV features picked from GA. The integrations were conducted using two fusion methods which are Euclidean distance and serial fusion. The differences in reaction of the fused features were then identified. Based on the result, the Euclidean distance (ed) which is the fused feature by the parallel fusion, displayed the most efficient reaction with accuracy, sensitivity, and specificity at 80.0%, 83.3% and 78.3%, respectively. Support Vector Machine (SVM) was utilized to attain such result. The fused feature performance was then fed into SVM which produced indexes on stress responses. The result retrieved from these indexes acts as a measure in handling future health deliverability and perhaps could eventually enhance the health care platform for midlife individuals

A Framework for Psychophysiological Classification within a Cultural Heritage Context Using Interest

This article presents a psychophysiological construct of interest as a knowledge emotion and illustrates the importance of interest detection in a cultural heritage context. The objective of this work is to measure and classify psychophysiological reactivity in response to cultural heritage material presented as visual and audio. We present a data processing and classification framework for the classification of interest. Two studies are reported, adopting a subject-dependent approach to classify psychophysiological signals using mobile physiological sensors and the support vector machine learning algorithm. The results show that it is possible to reliably infer a state of interest from cultural heritage material using psychophysiological feature data and a machine learning approach, informing future work for the development of a real-time physiological computing system for use within an adaptive cultural heritage experience designed to adapt the provision of information to sustain the interest of the visitor

Aerospace medicine and biology. A continuing bibliography (supplement 231)

This bibliography lists 284 reports, articles, and other documents introduced into the NASA scientific and technical information system in March 1982

Feature Space Augmentation: Improving Prediction Accuracy of Classical Problems in Cognitive Science and Computer Vison

The prediction accuracy in many classical problems across multiple domains has seen a rise since computational tools such as multi-layer neural nets and complex machine learning algorithms have become widely accessible to the research community. In this research, we take a step back and examine the feature space in two problems from very different domains. We show that novel augmentation to the feature space yields higher performance. Emotion Recognition in Adults from a Control Group: The objective is to quantify the emotional state of an individual at any time using data collected by wearable sensors. We define emotional state as a mixture of amusement, anger, disgust, fear, sadness, anxiety and neutral and their respective levels at any time. The generated model predicts an individual’s dominant state and generates an emotional spectrum, 1x7 vector indicating levels of each emotional state and anxiety. We present an iterative learning framework that alters the feature space uniquely to an individual’s emotion perception, and predicts the emotional state using the individual specific feature space. Hybrid Feature Space for Image Classification: The objective is to improve the accuracy of existing image recognition by leveraging text features from the images. As humans, we perceive objects using colors, dimensions, geometry and any textual information we can gather. Current image recognition algorithms rely exclusively on the first 3 and do not use the textual information. This study develops and tests an approach that trains a classifier on a hybrid text based feature space that has comparable accuracy to the state of the art CNN’s while being significantly inexpensive computationally. Moreover, when combined with CNN’S the approach yields a statistically significant boost in accuracy. Both models are validated using cross validation and holdout validation, and are evaluated against the state of the art

Exploring the Biocybernetic loop: Classifying Psychophysiological Responses to Cultural Artefacts using Physiological Computing

The aim of this research project was to provide a bio-sensing component for a real-time adaptive technology in the context of cultural heritage. The proposed system was designed to infer the interest or intention of the user and to augment elements of the cultural heritage experience interactively through implicit interaction. Implicit interaction in this context is the process whereby the system observes the user while they interact with artefacts; recording psychophysiological responses to cultural heritage artefacts or materials and acting upon these responses to drive adaptations in content in real-time.Real-time biocybernetic control is the central component of physiological computing wherein physiological data are converted into a control input for a technological system. At its core the bio-sensing component is a biocybernetic control loop that utilises an inference of user interest as its primary driver. A biocybernetic loop is composed of four main stages: inference, classification, adaptation and interaction. The programme of research described in this thesis is concerned primarily with exploration of the inference and classification elements of the biocybernetic loop but also encompasses an element of adaptation and interaction. These elements are explored first through literature review and discussion (presented in chapters 1-5) and then through experimental studies (presented in chapters 7-11)

Towards disappearing user interfaces for ubiquitous computing: human enhancement from sixth sense to super senses

The enhancement of human senses electronically is possible when pervasive computers interact unnoticeably with humans in Ubiquitous Computing. The design of computer user interfaces towards “disappearing” forces the interaction with humans using a content rather than a menu driven approach, thus the emerging requirement for huge number of non-technical users interfacing intuitively with billions of computers in the Internet of Things is met. Learning to use particular applications in Ubiquitous Computing is either too slow or sometimes impossible so the design of user interfaces must be naturally enough to facilitate intuitive human behaviours. Although humans from different racial, cultural and ethnic backgrounds own the same physiological sensory system, the perception to the same stimuli outside the human bodies can be different. A novel taxonomy for Disappearing User Interfaces (DUIs) to stimulate human senses and to capture human responses is proposed. Furthermore, applications of DUIs are reviewed. DUIs with sensor and data fusion to simulate the Sixth Sense is explored. Enhancement of human senses through DUIs and Context Awareness is discussed as the groundwork enabling smarter wearable devices for interfacing with human emotional memories