1,151 research outputs found
Detecting Online Hate Speech Using Context Aware Models
In the wake of a polarizing election, the cyber world is laden with hate
speech. Context accompanying a hate speech text is useful for identifying hate
speech, which however has been largely overlooked in existing datasets and hate
speech detection models. In this paper, we provide an annotated corpus of hate
speech with context information well kept. Then we propose two types of hate
speech detection models that incorporate context information, a logistic
regression model with context features and a neural network model with learning
components for context. Our evaluation shows that both models outperform a
strong baseline by around 3% to 4% in F1 score and combining these two models
further improve the performance by another 7% in F1 score.Comment: Published in RANLP 201
Challenges for Toxic Comment Classification: An In-Depth Error Analysis
Toxic comment classification has become an active research field with many
recently proposed approaches. However, while these approaches address some of
the task's challenges others still remain unsolved and directions for further
research are needed. To this end, we compare different deep learning and
shallow approaches on a new, large comment dataset and propose an ensemble that
outperforms all individual models. Further, we validate our findings on a
second dataset. The results of the ensemble enable us to perform an extensive
error analysis, which reveals open challenges for state-of-the-art methods and
directions towards pending future research. These challenges include missing
paradigmatic context and inconsistent dataset labels.Comment: ALW2: 2nd Workshop on Abusive Language Online to be held at EMNLP
2018 (Brussels, Belgium), October 31st, 201
Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection
Hate speech detection is a critical, yet challenging problem in Natural
Language Processing (NLP). Despite the existence of numerous studies dedicated
to the development of NLP hate speech detection approaches, the accuracy is
still poor. The central problem is that social media posts are short and noisy,
and most existing hate speech detection solutions take each post as an isolated
input instance, which is likely to yield high false positive and negative
rates. In this paper, we radically improve automated hate speech detection by
presenting a novel model that leverages intra-user and inter-user
representation learning for robust hate speech detection on Twitter. In
addition to the target Tweet, we collect and analyze the user's historical
posts to model intra-user Tweet representations. To suppress the noise in a
single Tweet, we also model the similar Tweets posted by all other users with
reinforced inter-user representation learning techniques. Experimentally, we
show that leveraging these two representations can significantly improve the
f-score of a strong bidirectional LSTM baseline model by 10.1%
Context-Aware Attention for Understanding Twitter Abuse
The original goal of any social media platform is to facilitate users to
indulge in healthy and meaningful conversations. But more often than not, it
has been found that it becomes an avenue for wanton attacks. We want to
alleviate this issue and hence we try to provide a detailed analysis of how
abusive behavior can be monitored in Twitter. The complexity of the natural
language constructs makes this task challenging. We show how applying
contextual attention to Long Short Term Memory networks help us give near state
of art results on multiple benchmarks abuse detection data sets from Twitter.Comment: The full published version of this work is available at:
\url{https://www.aclweb.org/anthology/W19-3508/}. Please use the version
published in the ACL anthology for citation purpose
Towards countering hate speech against journalists on social media
The damaging effects of hate speech on social media are evident during the
last few years, and several organizations, researchers and social media
platforms tried to harness them in various ways. Despite these efforts, social
media users are still affected by hate speech. The problem is even more
apparent to social groups that promote public discourse, such as journalists.
In this work, we focus on countering hate speech that is targeted to
journalistic social media accounts. To accomplish this, a group of journalists
assembled a definition of hate speech, taking into account the journalistic
point of view and the types of hate speech that are usually targeted against
journalists. We then compile a large pool of tweets referring to
journalism-related accounts in multiple languages. In order to annotate the
pool of unlabeled tweets according to the definition, we follow a concise
annotation strategy that involves active learning annotation stages. The
outcome of this paper is a novel, publicly available collection of Twitter
datasets in five different languages. Additionally, we experiment with
state-of-the-art deep learning architectures for hate speech detection and use
our annotated datasets to train and evaluate them. Finally, we propose an
ensemble detection model that outperforms all individual models
Identifying Offensive Posts and Targeted Offense from Twitter
In this paper we present our approach and the system description for Sub-task
A and Sub Task B of SemEval 2019 Task 6: Identifying and Categorizing Offensive
Language in Social Media. Sub-task A involves identifying if a given tweet is
offensive or not, and Sub Task B involves detecting if an offensive tweet is
targeted towards someone (group or an individual). Our models for Sub-task A is
based on an ensemble of Convolutional Neural Network, Bidirectional LSTM with
attention, and Bidirectional LSTM + Bidirectional GRU, whereas for Sub-task B,
we rely on a set of heuristics derived from the training data and manual
observation. We provide detailed analysis of the results obtained using the
trained models. Our team ranked 5th out of 103 participants in Sub-task A,
achieving a macro F1 score of 0.807, and ranked 8th out of 75 participants in
Sub Task B achieving a macro F1 of 0.695
Investigating Deep Learning Approaches for Hate Speech Detection in Social Media
The phenomenal growth on the internet has helped in empowering individual's
expressions, but the misuse of freedom of expression has also led to the
increase of various cyber crimes and anti-social activities. Hate speech is one
such issue that needs to be addressed very seriously as otherwise, this could
pose threats to the integrity of the social fabrics.
In this paper, we proposed deep learning approaches utilizing various
embeddings for detecting various types of hate speeches in social media.
Detecting hate speech from a large volume of text, especially tweets which
contains limited contextual information also poses several practical
challenges.
Moreover, the varieties in user-generated data and the presence of various
forms of hate speech makes it very challenging to identify the degree and
intention of the message. Our experiments on three publicly available datasets
of different domains shows a significant improvement in accuracy and F1-score.Comment: 12 pages, 2 figures, 8 tables. Accepted in CICLing: International
Conference on Computational Linguistics and Intelligent Text Processing,
2019. Modified after reviewer comment
KEIS@JUST at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT
This research presents our team KEIS@JUST participation at SemEval-2020 Task
12 which represents shared task on multilingual offensive language. We
participated in all the provided languages for all subtasks except sub-task-A
for the English language. Two main approaches have been developed the first is
performed to tackle both languages Arabic and English, a weighted ensemble
consists of Bi-GRU and CNN followed by Gaussian noise and global pooling layer
multiplied by weights to improve the overall performance. The second is
performed for other languages, a transfer learning from BERT beside the
recurrent neural networks such as Bi-LSTM and Bi-GRU followed by a global
average pooling layer. Word embedding and contextual embedding have been used
as features, moreover, data augmentation has been used only for the Arabic
language.Comment: 8 pages without references, 4 figures, SemEval 2020 conferenc
An Empirical Evaluation of Text Representation Schemes on Multilingual Social Web to Filter the Textual Aggression
This paper attempt to study the effectiveness of text representation schemes
on two tasks namely: User Aggression and Fact Detection from the social media
contents. In User Aggression detection, The aim is to identify the level of
aggression from the contents generated in the Social media and written in the
English, Devanagari Hindi and Romanized Hindi. Aggression levels are
categorized into three predefined classes namely: `Non-aggressive`, `Overtly
Aggressive`, and `Covertly Aggressive`. During the disaster-related incident,
Social media like, Twitter is flooded with millions of posts. In such emergency
situations, identification of factual posts is important for organizations
involved in the relief operation. We anticipated this problem as a combination
of classification and Ranking problem. This paper presents a comparison of
various text representation scheme based on BoW techniques, distributed
word/sentence representation, transfer learning on classifiers. Weighted
score is used as a primary evaluation metric. Results show that text
representation using BoW performs better than word embedding on machine
learning classifiers. While pre-trained Word embedding techniques perform
better on classifiers based on deep neural net. Recent transfer learning model
like ELMO, ULMFiT are fine-tuned for the Aggression classification task.
However, results are not at par with pre-trained word embedding model. Overall,
word embedding using fastText produce best weighted -score than Word2Vec
and Glove. Results are further improved using pre-trained vector model.
Statistical significance tests are employed to ensure the significance of the
classification results. In the case of lexically different test Dataset, other
than training Dataset, deep neural models are more robust and perform
substantially better than machine learning classifiers.Comment: 21 Page, 2 Figur
Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXEL
Detecting hate speech, especially in low-resource languages, is a non-trivial
challenge. To tackle this, we developed a tailored architecture based on
frozen, pre-trained Transformers to examine cross-lingual zero-shot and
few-shot learning, in addition to uni-lingual learning, on the HatEval
challenge data set. With our novel attention-based classification block AXEL,
we demonstrate highly competitive results on the English and Spanish subsets.
We also re-sample the English subset, enabling additional, meaningful
comparisons in the future
- …