3,478 research outputs found
Multi-score Learning for Affect Recognition: the Case of Body Postures
An important challenge in building automatic affective state
recognition systems is establishing the ground truth. When the groundtruth
is not available, observers are often used to label training and testing
sets. Unfortunately, inter-rater reliability between observers tends to
vary from fair to moderate when dealing with naturalistic expressions.
Nevertheless, the most common approach used is to label each expression
with the most frequent label assigned by the observers to that expression.
In this paper, we propose a general pattern recognition framework
that takes into account the variability between observers for automatic
affect recognition. This leads to what we term a multi-score learning
problem in which a single expression is associated with multiple values
representing the scores of each available emotion label. We also propose
several performance measurements and pattern recognition methods for
this framework, and report the experimental results obtained when testing
and comparing these methods on two affective posture datasets
Affective games:a multimodal classification system
Affective gaming is a relatively new field of research that exploits human emotions to influence gameplay for an enhanced player experience. Changes in player’s psychology reflect on their behaviour and physiology, hence recognition of such variation is a core element in affective games. Complementary sources of affect offer more reliable recognition, especially in contexts where one modality is partial or unavailable. As a multimodal recognition system, affect-aware games are subject to the practical difficulties met by traditional trained classifiers. In addition, inherited game-related challenges in terms of data collection and performance arise while attempting to sustain an acceptable level of immersion. Most existing scenarios employ sensors that offer limited freedom of movement resulting in less realistic experiences. Recent advances now offer technology that allows players to communicate more freely and naturally with the game, and furthermore, control it without the use of input devices. However, the affective game industry is still in its infancy and definitely needs to catch up with the current life-like level of adaptation provided by graphics and animation
An original framework for understanding human actions and body language by using deep neural networks
The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour.
By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way.
These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively.
While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements;
both are essential tasks in many computer vision applications, including event recognition, and video surveillance.
In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided.
The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements.
All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods
Naturalistic Affective Expression Classification by a Multi-Stage Approach Based on Hidden Markov Models
In naturalistic behaviour, the affective states of a person
change at a rate much slower than the typical rate at which video or
audio is recorded (e.g. 25fps for video). Hence, there is a high probability
that consecutive recorded instants of expressions represent a same
affective content. In this paper, a multi-stage automatic affective expression
recognition system is proposed which uses Hidden Markov Models
(HMMs) to take into account this temporal relationship and finalize the
classification process. The hidden states of the HMMs are associated
with the levels of affective dimensions to convert the classification problem
into a best path finding problem in HMM. The system was tested on
the audio data of the Audio/Visual Emotion Challenge (AVEC) datasets
showing performance significantly above that of a one-stage classification
system that does not take into account the temporal relationship, as well
as above the baseline set provided by this Challenge. Due to the generality
of the approach, this system could be applied to other types of
affective modalities
What does touch tell us about emotions in touchscreen-based gameplay?
This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2012 ACM. It is posted here by permission of ACM for your personal use. Not for redistribution.Nowadays, more and more people play games on touch-screen mobile phones. This phenomenon raises a very interesting question: does touch behaviour reflect the player’s emotional state? If possible, this would not only be a valuable evaluation indicator for game designers, but also for real-time personalization of the game experience. Psychology studies on acted touch behaviour show the existence of discriminative affective profiles. In this paper, finger-stroke features during gameplay on an iPod were extracted and their discriminative power analysed. Based on touch-behaviour, machine learning algorithms were used to build systems for automatically discriminating between four emotional states (Excited, Relaxed, Frustrated, Bored), two levels of arousal and two levels of valence. The results were very interesting reaching between 69% and 77% of correct discrimination between the four emotional states. Higher results (~89%) were obtained for discriminating between two levels of arousal and two levels of valence
Multimodal database of emotional speech, video and gestures
People express emotions through different modalities. Integration of verbal and non-verbal communication channels creates a system in which the message is easier to understand. Expanding the focus
to several expression forms can facilitate research on emotion recognition as well as human-machine interaction. In this article, the authors
present a Polish emotional database composed of three modalities: facial expressions, body movement and gestures, and speech. The corpora
contains recordings registered in studio conditions, acted out by 16 professional actors (8 male and 8 female). The data is labeled with six basic
emotions categories, according to Ekman’s emotion categories. To check
the quality of performance, all recordings are evaluated by experts and
volunteers. The database is available to academic community and might
be useful in the study on audio-visual emotion recognition
- …