129 research outputs found
Modeling Empathy and Distress in Reaction to News Stories
Computational detection and understanding of empathy is an important factor
in advancing human-computer interaction. Yet to date, text-based empathy
prediction has the following major limitations: It underestimates the
psychological complexity of the phenomenon, adheres to a weak notion of ground
truth where empathic states are ascribed by third parties, and lacks a shared
corpus. In contrast, this contribution presents the first publicly available
gold standard for empathy prediction. It is constructed using a novel
annotation methodology which reliably captures empathy assessments by the
writer of a statement using multi-item scales. This is also the first
computational work distinguishing between multiple forms of empathy, empathic
concern, and personal distress, as recognized throughout psychology. Finally,
we present experimental results for three different predictive models, of which
a CNN performs the best.Comment: To appear at EMNLP 201
What changed your mind : the roles of dynamic topics and discourse in argumentation process
In our world with full of uncertainty, debates and argumentation contribute to the progress of science and society. Despite of the in- creasing attention to characterize human arguments, most progress made so far focus on the debate outcome, largely ignoring the dynamic patterns in argumentation processes. This paper presents a study that automatically analyzes the key factors in argument persuasiveness, beyond simply predicting who will persuade whom. Specifically, we propose a novel neural model that is able to dynamically track the changes of latent topics and discourse in argumentative conversations, allowing the investigation of their roles in influencing the outcomes of persuasion. Extensive experiments have been conducted on argumentative conversations on both social media and supreme court. The results show that our model outperforms state-of-the-art models in identifying persuasive arguments via explicitly exploring dynamic factors of topic and discourse. We further analyze the effects of topics and discourse on persuasiveness, and find that they are both useful -- topics provide concrete evidence while superior discourse styles may bias participants, especially in social media arguments. In addition, we draw some findings from our empirical results, which will help people better engage in future persuasive conversations
Emotion Embeddings \unicode{x2014} Learning Stable and Homogeneous Abstractions from Heterogeneous Affective Datasets
Human emotion is expressed in many communication modalities and media formats
and so their computational study is equally diversified into natural language
processing, audio signal analysis, computer vision, etc. Similarly, the large
variety of representation formats used in previous research to describe
emotions (polarity scales, basic emotion categories, dimensional approaches,
appraisal theory, etc.) have led to an ever proliferating diversity of
datasets, predictive models, and software tools for emotion analysis. Because
of these two distinct types of heterogeneity, at the expressional and
representational level, there is a dire need to unify previous work on
increasingly diverging data and label types. This article presents such a
unifying computational model. We propose a training procedure that learns a
shared latent representation for emotions, so-called emotion embeddings,
independent of different natural languages, communication modalities, media or
representation label formats, and even disparate model architectures.
Experiments on a wide range of heterogeneous affective datasets indicate that
this approach yields the desired interoperability for the sake of reusability,
interpretability and flexibility, without penalizing prediction quality. Code
and data are archived under https://doi.org/10.5281/zenodo.7405327 .Comment: 18 pages, 6 figure
Keyword Embeddings for Query Suggestion
Nowadays, search engine users commonly rely on query suggestions to improve
their initial inputs. Current systems are very good at recommending lexical
adaptations or spelling corrections to users' queries. However, they often
struggle to suggest semantically related keywords given a user's query. The
construction of a detailed query is crucial in some tasks, such as legal
retrieval or academic search. In these scenarios, keyword suggestion methods
are critical to guide the user during the query formulation. This paper
proposes two novel models for the keyword suggestion task trained on scientific
literature. Our techniques adapt the architecture of Word2Vec and FastText to
generate keyword embeddings by leveraging documents' keyword co-occurrence.
Along with these models, we also present a specially tailored negative sampling
approach that exploits how keywords appear in academic publications. We devise
a ranking-based evaluation methodology following both known-item and ad-hoc
search scenarios. Finally, we evaluate our proposals against the
state-of-the-art word and sentence embedding models showing considerable
improvements over the baselines for the tasks
Automatic Curriculum Learning With Over-repetition Penalty for Dialogue Policy Learning
Dialogue policy learning based on reinforcement learning is difficult to be
applied to real users to train dialogue agents from scratch because of the high
cost. User simulators, which choose random user goals for the dialogue agent to
train on, have been considered as an affordable substitute for real users.
However, this random sampling method ignores the law of human learning, making
the learned dialogue policy inefficient and unstable. We propose a novel
framework, Automatic Curriculum Learning-based Deep Q-Network (ACL-DQN), which
replaces the traditional random sampling method with a teacher policy model to
realize the dialogue policy for automatic curriculum learning. The teacher
model arranges a meaningful ordered curriculum and automatically adjusts it by
monitoring the learning progress of the dialogue agent and the over-repetition
penalty without any requirement of prior knowledge. The learning progress of
the dialogue agent reflects the relationship between the dialogue agent's
ability and the sampled goals' difficulty for sample efficiency. The
over-repetition penalty guarantees the sampled diversity. Experiments show that
the ACL-DQN significantly improves the effectiveness and stability of dialogue
tasks with a statistically significant margin. Furthermore, the framework can
be further improved by equipping with different curriculum schedules, which
demonstrates that the framework has strong generalizability
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems
While automatic dialogue tutors hold great potential in making education
personalized and more accessible, research on such systems has been hampered by
a lack of sufficiently large and high-quality datasets. Collecting such
datasets remains challenging, as recording tutoring sessions raises privacy
concerns and crowdsourcing leads to insufficient data quality. To address this,
we propose a framework to generate such dialogues by pairing human teachers
with a Large Language Model (LLM) prompted to represent common student errors.
We describe how we use this framework to collect MathDial, a dataset of 3k
one-to-one teacher-student tutoring dialogues grounded in multi-step math
reasoning problems. While models like GPT-3 are good problem solvers, they fail
at tutoring because they generate factually incorrect feedback or are prone to
revealing solutions to students too early. To overcome this, we let teachers
provide learning opportunities to students by guiding them using various
scaffolding questions according to a taxonomy of teacher moves. We demonstrate
MathDial and its extensive annotations can be used to finetune models to be
more effective tutors (and not just solvers). We confirm this by automatic and
human evaluation, notably in an interactive setting that measures the trade-off
between student solving success and telling solutions. The dataset is released
publicly.Comment: Jakub Macina, Nico Daheim, and Sankalan Pal Chowdhury contributed
equally to this work. Accepted at EMNLP2023 Findings. Code and dataset
available: https://github.com/eth-nlped/mathdia
Exploring Implicit Sentiment Evoked by Fine-grained News Events
We investigate the feasibility of defining sentiment evoked by fine-grained news events. Our research question is based on the premise that methods for detecting implicit sentiment in news can be a key driver of content diversity, which is one way to mitigate the detrimental effects of filter bubbles that recommenders based on collaborative filtering may produce. Our experiments are based on 1,735 news articles from major Flemish newspapers that were manually annotated, with high agreement, for implicit sentiment. While lexical resources prove insufficient for sentiment analysis in this data genre, our results demonstrate that machine learning models based on SVM and BERT are able to automatically infer the implicit sentiment evoked by news events
- …