1,508 research outputs found
Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge
The task of emotion recognition in conversations (ERC) benefits from the
availability of multiple modalities, as provided, for example, in the
video-based Multimodal EmotionLines Dataset (MELD). However, only a few
research approaches use both acoustic and visual information from the MELD
videos. There are two reasons for this: First, label-to-video alignments in
MELD are noisy, making those videos an unreliable source of emotional speech
data. Second, conversations can involve several people in the same scene, which
requires the localisation of the utterance source. In this paper, we introduce
MELD with Fixed Audiovisual Information via Realignment (MELD-FAIR) by using
recent active speaker detection and automatic speech recognition models, we are
able to realign the videos of MELD and capture the facial expressions from
speakers in 96.92% of the utterances provided in MELD. Experiments with a
self-supervised voice recognition model indicate that the realigned MELD-FAIR
videos more closely match the transcribed utterances given in the MELD dataset.
Finally, we devise a model for emotion recognition in conversations trained on
the realigned MELD-FAIR videos, which outperforms state-of-the-art models for
ERC based on vision alone. This indicates that localising the source of
speaking activities is indeed effective for extracting facial expressions from
the uttering speakers and that faces provide more informative visual cues than
the visual features state-of-the-art models have been using so far. The
MELD-FAIR realignment data, and the code of the realignment procedure and of
the emotional recognition, are available at
https://github.com/knowledgetechnologyuhh/MELD-FAIR.Comment: 17 pages, 8 figures, 7 tables, Published in Neurocomputin
BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis
Sentiment analysis in conversations has gained increasing attention in recent
years for the growing amount of applications it can serve, e.g., sentiment
analysis, recommender systems, and human-robot interaction. The main difference
between conversational sentiment analysis and single sentence sentiment
analysis is the existence of context information which may influence the
sentiment of an utterance in a dialogue. How to effectively encode contextual
information in dialogues, however, remains a challenge. Existing approaches
employ complicated deep learning structures to distinguish different parties in
a conversation and then model the context information. In this paper, we
propose a fast, compact and parameter-efficient party-ignorant framework named
bidirectional emotional recurrent unit for conversational sentiment analysis.
In our system, a generalized neural tensor block followed by a two-channel
classifier is designed to perform context compositionality and sentiment
classification, respectively. Extensive experiments on three standard datasets
demonstrate that our model outperforms the state of the art in most cases.Comment: 9 pages, 7 figure
Affect-LM: A Neural Language Model for Customizable Affective Text Generation
Human verbal communication includes affective messages which are conveyed
through use of emotionally colored words. There has been a lot of research in
this direction but the problem of integrating state-of-the-art neural language
models with affective information remains an area ripe for exploration. In this
paper, we propose an extension to an LSTM (Long Short-Term Memory) language
model for generating conversational text, conditioned on affect categories. Our
proposed model, Affect-LM enables us to customize the degree of emotional
content in generated sentences through an additional design parameter.
Perception studies conducted using Amazon Mechanical Turk show that Affect-LM
generates naturally looking emotional sentences without sacrificing grammatical
correctness. Affect-LM also learns affect-discriminative word representations,
and perplexity experiments show that additional affective information in
conversational text can improve language model prediction
Emotion Recognition in Conversation using Probabilistic Soft Logic
Creating agents that can both appropriately respond to conversations and
understand complex human linguistic tendencies and social cues has been a long
standing challenge in the NLP community. A recent pillar of research revolves
around emotion recognition in conversation (ERC); a sub-field of emotion
recognition that focuses on conversations or dialogues that contain two or more
utterances. In this work, we explore an approach to ERC that exploits the use
of neural embeddings along with complex structures in dialogues. We implement
our approach in a framework called Probabilistic Soft Logic (PSL), a
declarative templating language that uses first-order like logical rules, that
when combined with data, define a particular class of graphical model.
Additionally, PSL provides functionality for the incorporation of results from
neural models into PSL models. This allows our model to take advantage of
advanced neural methods, such as sentence embeddings, and logical reasoning
over the structure of a dialogue. We compare our method with state-of-the-art
purely neural ERC systems, and see almost a 20% improvement. With these
results, we provide an extensive qualitative and quantitative analysis over the
DailyDialog conversation dataset
- …