13 research outputs found
Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Data
A large portion of society united around the same vision and ideas carries
enormous energy. That is precisely what political figures would like to
accumulate for their cause. With this goal in mind, they can sometimes resort
to distorting or hiding the truth, unintentionally or on purpose, which opens
the door for misinformation and disinformation. Tools for automatic detection
of check-worthy claims would be of great help to moderators of debates,
journalists, and fact-checking organizations. While previous work on detecting
check-worthy claims has focused on text, here we explore the utility of the
audio signal as an additional information source. We create a new multimodal
dataset (text and audio in English) containing 48 hours of speech. Our
evaluation results show that the audio modality together with text yields
improvements over text alone in the case of multiple speakers. Moreover, an
audio-only model could outperform a text-only one for a single speaker.Comment: check-worthy claims, fake news, political debates, audi
CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media
While there has been substantial progress in developing systems to automate
fact-checking, they still lack credibility in the eyes of the users. Thus, an
interesting approach has emerged: to perform automatic fact-checking by
verifying whether an input claim has been previously fact-checked by
professional fact-checkers and to return back an article that explains their
decision. This is a sensible approach as people trust manual fact-checking, and
as many claims are repeated multiple times. Yet, a major issue when building
such systems is the small number of known tweet--verifying article pairs
available for training. Here, we aim to bridge this gap by making use of crowd
fact-checking, i.e., mining claims in social media for which users have
responded with a link to a fact-checking article. In particular, we mine a
large-scale collection of 330,000 tweets paired with a corresponding
fact-checking article. We further propose an end-to-end framework to learn from
this noisy data based on modified self-adaptive training, in a distant
supervision scenario. Our experiments on the CLEF'21 CheckThat! test set show
improvements over the state of the art by two points absolute. Our code and
datasets are available at https://github.com/mhardalov/crowdchecked-claimsComment: Accepted to AACL-IJCNLP 2022 (Main Conference
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark
We present bgGLUE (Bulgarian General Language Understanding Evaluation), a
benchmark for evaluating language models on Natural Language Understanding
(NLU) tasks in Bulgarian. Our benchmark includes NLU tasks targeting a variety
of NLP problems (e.g., natural language inference, fact-checking, named entity
recognition, sentiment analysis, question answering, etc.) and machine learning
tasks (sequence labeling, document-level classification, and regression). We
run the first systematic evaluation of pre-trained language models for
Bulgarian, comparing and contrasting results across the nine tasks in the
benchmark. The evaluation results show strong performance on sequence labeling
tasks, but there is a lot of room for improvement for tasks that require more
complex reasoning. We make bgGLUE publicly available together with the
fine-tuning and the evaluation code, as well as a public leaderboard at
https://bgglue.github.io/, and we hope that it will enable further advancements
in developing NLU models for Bulgarian.Comment: Accepted to ACL 2023 (Main Conference
Detecting Abusive Language on Online Platforms: A Critical Analysis
Abusive language on online platforms is a major societal problem, often
leading to important societal problems such as the marginalisation of
underrepresented minorities. There are many different forms of abusive language
such as hate speech, profanity, and cyber-bullying, and online platforms seek
to moderate it in order to limit societal harm, to comply with legislation, and
to create a more inclusive environment for their users. Within the field of
Natural Language Processing, researchers have developed different methods for
automatically detecting abusive language, often focusing on specific
subproblems or on narrow communities, as what is considered abusive language
very much differs by context. We argue that there is currently a dichotomy
between what types of abusive language online platforms seek to curb, and what
research efforts there are to automatically detect abusive language. We thus
survey existing methods as well as content moderation policies by online
platforms in this light, and we suggest directions for future work