8,072 research outputs found
Fusion of Learned Multi-Modal Representations and Dense Trajectories for Emotional Analysis in Videos
When designing a video affective content analysis algorithm, one of the most important steps is the selection of discriminative features for the effective representation of video segments. The majority of existing affective content analysis methods either use low-level audio-visual features or generate handcrafted higher level representations based on these low-level features. We propose in this work to use deep learning methods, in particular convolutional neural networks (CNNs), in order to automatically learn and extract mid-level representations from raw data. To this end, we exploit the audio and visual modality of videos by employing Mel-Frequency Cepstral Coefficients (MFCC) and color values in the HSV color space. We also incorporate dense trajectory based motion features in order to further enhance the performance of the analysis. By means of multi-class support vector machines (SVMs) and fusion mechanisms, music video clips are classified into one of four affective categories representing the four quadrants of the Valence-Arousal (VA) space. Results obtained on a subset of the DEAP dataset show (1) that higher level representations perform better than low-level features, and (2) that incorporating motion information leads to a notable performance gain, independently from the chosen representation
Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring
How to fuse multi-channel neurophysiological signals for emotion recognition is emerging as a hot research topic in community of Computational Psychophysiology. Nevertheless, prior feature engineering based approaches require extracting various domain knowledge related features at a high time cost. Moreover, traditional fusion method cannot fully utilise correlation information between different channels and frequency components. In this paper, we design a hybrid deep learning model, in which the 'Convolutional Neural Network (CNN)' is utilised for extracting task-related features, as well as mining inter-channel and inter-frequency correlation, besides, the 'Recurrent Neural Network (RNN)' is concatenated for integrating contextual information from the frame cube sequence. Experiments are carried out in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Experimental results demonstrate that the proposed framework outperforms the classical methods, with regard to both of the emotional dimensions of Valence and Arousal
No strings attached:physiological monitoring of rhesus monkeys (Macaca mulatta) with thermal imaging
The ordinal nature of emotions
Representing computationally everyday emotional
states is a challenging task and, arguably, one of the most fundamental
for affective computing. Standard practice in emotion annotation
is to ask humans to assign an absolute value of intensity
to each emotional behavior they observe. Psychological theories
and evidence from multiple disciplines including neuroscience,
economics and artificial intelligence, however, suggest that the
task of assigning reference-based (relative) values to subjective
notions is better aligned with the underlying representations
than assigning absolute values. Evidence also shows that we
use reference points, or else anchors, against which we evaluate
values such as the emotional state of a stimulus; suggesting
again that ordinal labels are a more suitable way to represent
emotions. This paper draws together the theoretical reasons to
favor relative over absolute labels for representing and annotating
emotion, reviewing the literature across several disciplines. We
go on to discuss good and bad practices of treating ordinal
and other forms of annotation data, and make the case for
preference learning methods as the appropriate approach for
treating ordinal labels. We finally discuss the advantages of
relative annotation with respect to both reliability and validity
through a number of case studies in affective computing, and
address common objections to the use of ordinal data. Overall,
the thesis that emotions are by nature relative is supported by
both theoretical arguments and evidence, and opens new horizons
for the way emotions are viewed, represented and analyzed
computationally.peer-reviewe
ELVIS: Entertainment-led video summaries
© ACM, 2010. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Multimedia Computing, Communications, and Applications, 6(3): Article no. 17 (2010) http://doi.acm.org/10.1145/1823746.1823751Video summaries present the user with a condensed and succinct representation of the content of a video stream. Usually this is achieved by attaching degrees of importance to low-level image, audio and text features. However, video content elicits strong and measurable physiological responses in the user, which are potentially rich indicators of what video content is memorable to or emotionally engaging for an individual user. This article proposes a technique that exploits such physiological responses to a given video stream by a given user to produce Entertainment-Led VIdeo Summaries (ELVIS). ELVIS is made up of five analysis phases which correspond to the analyses of five physiological response measures: electro-dermal response (EDR), heart rate (HR), blood volume pulse (BVP), respiration rate (RR), and respiration amplitude (RA). Through these analyses, the temporal locations of the most entertaining video subsegments, as they occur within the video stream as a whole, are automatically identified. The effectiveness of the ELVIS technique is verified through a statistical analysis of data collected during a set of user trials. Our results show that ELVIS is more consistent than RANDOM, EDR, HR, BVP, RR and RA selections in identifying the most entertaining video subsegments for content in the comedy, horror/comedy, and horror genres. Subjective user reports also reveal that ELVIS video summaries are comparatively easy to understand, enjoyable, and informative
Neurophysiological Profile of Antismoking Campaigns
Over the past few decades, antismoking public service announcements (PSAs) have been used by governments to promote healthy
behaviours in citizens, for instance, against drinking before the drive and against smoke. Effectiveness of such PSAs has been
suggested especially for young persons. By now, PSAs efficacy is still mainly assessed through traditional methods (questionnaires
and metrics) and could be performed only after the PSAs broadcasting, leading to waste of economic resources and time in the
case of Ineffective PSAs. One possible countermeasure to such ineffective use of PSAs could be promoted by the evaluation of the
cerebral reaction to the PSA of particular segments of population (e.g., old, young, and heavy smokers). In addition, it is crucial to
gather such cerebral activity in front of PSAs that have been assessed to be effective against smoke (Effective PSAs), comparing
results to the cerebral reactions to PSAs that have been certified to be not effective (Ineffective PSAs). &e eventual differences
between the cerebral responses toward the two PSA groups will provide crucial information about the possible outcome of new
PSAs before to its broadcasting. &is study focused on adult population, by investigating the cerebral reaction to the vision of
different PSA images, which have already been shown to be Effective and Ineffective for the promotion of an antismoking
behaviour. Results showed how variables as gender and smoking habits can influence the perception of PSA images, and how
different communication styles of the antismoking campaigns could facilitate the comprehension of PSA’s message and then
enhance the related impac
Emotion Interaction With Virtual Reality Using Hybrid Emotion Classification Technique Toward Brain Signals
Human computer interaction (HCI) considered main aspect in virtual reality (VR) especially in the context of emotion, where users can interact with virtual reality through their emotions and it could be expressed in virtual reality. Last decade many researchers focused on emotion classification in order to employ emotion in interaction with virtual reality, the classification will be done based on Electroencephalogram (EEG) brain signals. This paper provides a new hybrid emotion classification method by combining self- assessment, arousal valence dimension and variance of brain hemisphere activity to classify users’ emotions. Self-assessment considered a standard technique used for assessing emotion, arousal valence emotion dimension model is an emotion classifier with regards to aroused emotions and brain hemisphere activity that classifies emotion with regards to right and left hemisphere. This method can classify human emotions, two basic emotions is highlighted i.e. happy and sad. EEG brain signals are used to interpret the users’ emotional. Emotion interaction is expressed by 3D model walking expression in VR. The results show that the hybrid method classifies the highlighted emotions in different circumstances, and how the 3D model changes its walking style according to the classified users’ emotions. Finally, the outcome is believed to afford new technique on classifying emotions with feedback through 3D virtual model walking expression
Towards emotional interaction: using movies to automatically learn users’ emotional states
The HCI community is actively seeking novel methodologies to gain insight into the user's experience during interaction with both the application and the content. We propose an emotional recognition engine capable of automatically recognizing a set of human emotional states using psychophysiological measures of the autonomous nervous system, including galvanic skin response, respiration, and heart rate. A novel pattern recognition system, based on discriminant analysis and support vector machine classifiers is trained using movies' scenes selected to induce emotions ranging from the positive to the negative valence dimension, including happiness, anger, disgust, sadness, and fear. In this paper we introduce an emotion recognition system and evaluate its accuracy by presenting the results of an experiment conducted with three physiologic sensors.info:eu-repo/semantics/publishedVersio
- …