6 research outputs found
Bridging the Domain Gap for Stance Detection for the Zulu language
Misinformation has become a major concern in recent last years given its
spread across our information sources. In the past years, many NLP tasks have
been introduced in this area, with some systems reaching good results on
English language datasets. Existing AI based approaches for fighting
misinformation in literature suggest automatic stance detection as an integral
first step to success. Our paper aims at utilizing this progress made for
English to transfers that knowledge into other languages, which is a
non-trivial task due to the domain gap between English and the target
languages. We propose a black-box non-intrusive method that utilizes techniques
from Domain Adaptation to reduce the domain gap, without requiring any human
expertise in the target language, by leveraging low-quality data in both a
supervised and unsupervised manner. This allows us to rapidly achieve similar
results for stance detection for the Zulu language, the target language in this
work, as are found for English. We also provide a stance detection dataset in
the Zulu language. Our experimental results show that by leveraging English
datasets and machine translation we can increase performances on both English
data along with other languages.Comment: accepted to Intellisy
Capturing stance dynamics in social media: open challenges and research directions
Social media platforms provide a goldmine for mining public opinion on issues
of wide societal interest and impact. Opinion mining is a problem that can be
operationalised by capturing and aggregating the stance of individual social
media posts as supporting, opposing or being neutral towards the issue at hand.
While most prior work in stance detection has investigated datasets that cover
short periods of time, interest in investigating longitudinal datasets has
recently increased. Evolving dynamics in linguistic and behavioural patterns
observed in new data require adapting stance detection systems to deal with the
changes. In this survey paper, we investigate the intersection between
computational linguistics and the temporal evolution of human communication in
digital media. We perform a critical review of emerging research considering
dynamics, exploring different semantic and pragmatic factors that impact
linguistic data in general, and stance in particular. We further discuss
current directions in capturing stance dynamics in social media. We discuss the
challenges encountered when dealing with stance dynamics, identify open
challenges and discuss future directions in three key dimensions: utterance,
context and influence
Improved Target-specific Stance Detection on Social Media Platforms by Delving into Conversation Threads
Target-specific stance detection on social media, which aims at classifying a
textual data instance such as a post or a comment into a stance class of a
target issue, has become an emerging opinion mining paradigm of importance. An
example application would be to overcome vaccine hesitancy in combating the
coronavirus pandemic. However, existing stance detection strategies rely merely
on the individual instances which cannot always capture the expressed stance of
a given target. In response, we address a new task called conversational stance
detection which is to infer the stance towards a given target (e.g., COVID-19
vaccination) when given a data instance and its corresponding conversation
thread. To tackle the task, we first propose a benchmarking conversational
stance detection (CSD) dataset with annotations of stances and the structures
of conversation threads among the instances based on six major social media
platforms in Hong Kong. To infer the desired stances from both data instances
and conversation threads, we propose a model called Branch-BERT that
incorporates contextual information in conversation threads. Extensive
experiments on our CSD dataset show that our proposed model outperforms all the
baseline models that do not make use of contextual information. Specifically,
it improves the F1 score by 10.3% compared with the state-of-the-art method in
the SemEval-2016 Task 6 competition. This shows the potential of incorporating
rich contextual information on detecting target-specific stances on social
media platforms and implies a more practical way to construct future stance
detection tasks
Adapting to Change: The Temporal Persistence of Text Classifiers in the Context of Longitudinally Evolving Data
This thesis delves into the evolving landscape of NLP, particularly focusing on the temporal persistence of text classifiers amid the dynamic nature of language use. The primary objective is to understand how changes in language patterns over time impact the performance of text classification models and to develop methodologies for maintaining their effectiveness. The research begins by establishing a theoretical foundation for text classification and temporal data analysis, highlighting the challenges posed by the evolving use of language and its implications for NLP models. A detailed exploration of various datasets, including the stance detection and sentiment analysis datasets, sets the stage for examining these dynamics. The characteristics of the datasets, such as linguistic variations and temporal vocabulary growth, are carefully examined to understand their influence on the performance of the text classifier. A series of experiments are conducted to evaluate the performance of text classifiers across different temporal scenarios. The findings reveal a general trend of performance degradation over time, emphasizing the need for classifiers that can adapt to linguistic changes. The experiments assess models' ability to estimate past and future performance based on their current efficacy and linguistic dataset characteristics, leading to valuable insights into the factors influencing model longevity. Innovative solutions are proposed to address the observed performance decline and adapt to temporal changes in language use over time. These include incorporating temporal information into word embeddings and comparing various methods across temporal gaps. The Incremental Temporal Alignment (ITA) method emerges as a significant contributor to enhancing classifier performance in same-period experiments, although it faces challenges in maintaining effectiveness over longer temporal gaps. Furthermore, the exploration of machine learning and statistical methods highlights their potential to maintain classifier accuracy in the face of longitudinally evolving data. The thesis culminates in a shared task evaluation, where participant-submitted models are compared against baseline models to assess their classifiers' temporal persistence. This comparison provides a comprehensive understanding of the short-term, long-term, and overall persistence of their models, providing valuable information to the field. The research identifies several future directions, including interdisciplinary approaches that integrate linguistics and sociology, tracking textual shifts on online platforms, extending the analysis to other classification tasks, and investigating the ethical implications of evolving language in NLP applications. This thesis contributes to the NLP field by highlighting the importance of evaluating text classifiers' temporal persistence and offering methodologies to enhance their sustainability in dynamically evolving language environments. The findings and proposed approaches pave the way for future research, aiming at the development of more robust, reliable, and temporally persistent text classification models
Recommended from our members
Understanding and Reasoning about Implicit Meaning in Language
Enabling machines to interact with humans requires understanding what people mean, even when they do not say it explicitly. For example, machines should understand that ``selfish people oppose gun control'' implies a pro-gun control viewpoint (i.e., is taking a stance in support of gun control) despite the negative tone of the statement. Understanding these types of pragmatic inferences allows humans to grasp meaning (e.g., intentions, relevant facts) beyond what is literally expressed in an utterance. Furthermore, pragmatic inferences conveyed through generalizations (e.g., referring to generic ``selfish people'' rather than specific individuals in order to be more persuasive) support flexible and efficient reasoning. Therefore, in this thesis we focus on improving computational understanding of two inter-related types of pragmatic inferences: stancetaking and linguistic generalizations.
This thesis is divided into two parts. In Part II, we focus on stance detection. One major challenge for stance detection models is the large and continually growing set of stance targets (i.e., topics to take a stance on). Therefore, to address this we define and study zero-shot stance detection (i.e., evaluation on topics for which there is no training data). Our work develops both datasets and models for this task and analyzes the ongoing challenges for future work. This work has stimulated increasing and ongoing research in zero-shot stance detection in NLP.
Then in Part I we study generics --- a specific type of linguistic generalization that does not contain explicit quantifiers (e.g., ``most'', ``some''). These statements can have strong persuasive force and are also related to complex patterns of reasoning. To probe the current understanding capabilities of computational models, we focus on generating generics exemplars --- specific cases when a generic holds true or false. In particular, we propose computational frameworks grounded in linguistic theory to generate the first datasets of exemplars. We then use our datasets to highlight the challenges generics pose for natural language reasoning and the current generic-understanding capabilities of large language models