4,795 research outputs found
A physiologically-adapted gold standard for arousal during stress
Emotion is an inherently subjective psychophysiological human-state and to
produce an agreed-upon representation (gold standard) for continuous emotion
requires a time-consuming and costly training procedure of multiple human
annotators. There is strong evidence in the literature that physiological
signals are sufficient objective markers for states of emotion, particularly
arousal. In this contribution, we utilise a dataset which includes continuous
emotion and physiological signals - Heartbeats per Minute (BPM), Electrodermal
Activity (EDA), and Respiration-rate - captured during a stress inducing
scenario (Trier Social Stress Test). We utilise a Long Short-Term Memory,
Recurrent Neural Network to explore the benefit of fusing these physiological
signals with arousal as the target, learning from various audio, video, and
textual based features. We utilise the state-of-the-art MuSe-Toolbox to
consider both annotation delay and inter-rater agreement weighting when fusing
the target signals. An improvement in Concordance Correlation Coefficient (CCC)
is seen across features sets when fusing EDA with arousal, compared to the
arousal only gold standard results. Additionally, BERT-based textual features'
results improved for arousal plus all physiological signals, obtaining up to
.3344 CCC compared to .2118 CCC for arousal only. Multimodal fusion also
improves overall CCC with audio plus video features obtaining up to .6157 CCC
to recognize arousal plus EDA and BPM
Can Brain Signals Reveal Inner Alignment with Human Languages?
Brain Signals, such as Electroencephalography (EEG), and human languages have
been widely explored independently for many downstream tasks, however, the
connection between them has not been well explored. In this study, we explore
the relationship and dependency between EEG and language. To study at the
representation level, we introduced \textbf{MTAM}, a \textbf{M}ultimodal
\textbf{T}ransformer \textbf{A}lignment \textbf{M}odel, to observe coordinated
representations between the two modalities. We used various relationship
alignment-seeking techniques, such as Canonical Correlation Analysis and
Wasserstein Distance, as loss functions to transfigure features. On downstream
applications, sentiment analysis and relation detection, we achieved new
state-of-the-art results on two datasets, ZuCo and K-EmoCon. Our method
achieved an F1-score improvement of 1.7% on K-EmoCon and 9.3% on Zuco datasets
for sentiment analysis, and 7.4% on ZuCo for relation detection. In addition,
we provide interpretations of the performance improvement: (1) feature
distribution shows the effectiveness of the alignment module for discovering
and encoding the relationship between EEG and language; (2) alignment weights
show the influence of different language semantics as well as EEG frequency
features; (3) brain topographical maps provide an intuitive demonstration of
the connectivity in the brain regions. Our code is available at
\url{https://github.com/Jason-Qiu/EEG_Language_Alignment}.Comment: EMNLP 2023 Finding
The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress
The Multimodal Sentiment Analysis Challenge (MuSe) 2022 is dedicated to
multimodal sentiment and emotion recognition. For this year's challenge, we
feature three datasets: (i) the Passau Spontaneous Football Coach Humor
(Passau-SFCH) dataset that contains audio-visual recordings of German football
coaches, labelled for the presence of humour; (ii) the Hume-Reaction dataset in
which reactions of individuals to emotional stimuli have been annotated with
respect to seven emotional expression intensities, and (iii) the Ulm-Trier
Social Stress Test (Ulm-TSST) dataset comprising of audio-visual data labelled
with continuous emotion values (arousal and valence) of people in stressful
dispositions. Using the introduced datasets, MuSe 2022 2022 addresses three
contemporary affective computing problems: in the Humor Detection Sub-Challenge
(MuSe-Humor), spontaneous humour has to be recognised; in the Emotional
Reactions Sub-Challenge (MuSe-Reaction), seven fine-grained `in-the-wild'
emotions have to be predicted; and in the Emotional Stress Sub-Challenge
(MuSe-Stress), a continuous prediction of stressed emotion values is featured.
The challenge is designed to attract different research communities,
encouraging a fusion of their disciplines. Mainly, MuSe 2022 targets the
communities of audio-visual emotion recognition, health informatics, and
symbolic sentiment analysis. This baseline paper describes the datasets as well
as the feature sets extracted from them. A recurrent neural network with LSTM
cells is used to set competitive baseline results on the test partitions for
each sub-challenge. We report an Area Under the Curve (AUC) of .8480 for
MuSe-Humor; .2801 mean (from 7-classes) Pearson's Correlations Coefficient for
MuSe-Reaction, as well as .4931 Concordance Correlation Coefficient (CCC) and
.4761 for valence and arousal in MuSe-Stress, respectively.Comment: Preliminary baseline paper for the 3rd Multimodal Sentiment Analysis
Challenge (MuSe) 2022, a full-day workshop at ACM Multimedia 202
Predictive biometrics: A review and analysis of predicting personal characteristics from biometric data
Interest in the exploitation of soft biometrics information has continued to develop over the last decade or so. In comparison with traditional biometrics, which focuses principally on person identification, the idea of soft biometrics processing is to study the utilisation of more general information regarding a system user, which is not necessarily unique. There are increasing indications that this type of data will have great value in providing complementary information for user authentication. However, the authors have also seen a growing interest in broadening the predictive capabilities of biometric data, encompassing both easily definable characteristics such as subject age and, most recently, `higher level' characteristics such as emotional or mental states. This study will present a selective review of the predictive capabilities, in the widest sense, of biometric data processing, providing an analysis of the key issues still adequately to be addressed if this concept of predictive biometrics is to be fully exploited in the future
Recommended from our members
The Role of Landscapes and Landmarks in Bee Navigation: A Review.
The ability of animals to explore landmarks in their environment is essential to their fitness. Landmarks are widely recognized to play a key role in navigation by providing information in multiple sensory modalities. However, what is a landmark? We propose that animals use a hierarchy of information based upon its utility and salience when an animal is in a given motivational state. Focusing on honeybees, we suggest that foragers choose landmarks based upon their relative uniqueness, conspicuousness, stability, and context. We also propose that it is useful to distinguish between landmarks that provide sensory input that changes ("near") or does not change ("far") as the receiver uses these landmarks to navigate. However, we recognize that this distinction occurs on a continuum and is not a clear-cut dichotomy. We review the rich literature on landmarks, focusing on recent studies that have illuminated our understanding of the kinds of information that bees use, how they use it, potential mechanisms, and future research directions
- …