3,706 research outputs found
Detecting and Monitoring Hate Speech in Twitter
Social Media are sensors in the real world that can be used to measure the pulse of societies.
However, the massive and unfiltered feed of messages posted in social media is a phenomenon that
nowadays raises social alarms, especially when these messages contain hate speech targeted to a
specific individual or group. In this context, governments and non-governmental organizations
(NGOs) are concerned about the possible negative impact that these messages can have on individuals
or on the society. In this paper, we present HaterNet, an intelligent system currently being used by
the Spanish National Office Against Hate Crimes of the Spanish State Secretariat for Security that
identifies and monitors the evolution of hate speech in Twitter. The contributions of this research
are many-fold: (1) It introduces the first intelligent system that monitors and visualizes, using social
network analysis techniques, hate speech in Social Media. (2) It introduces a novel public dataset on
hate speech in Spanish consisting of 6000 expert-labeled tweets. (3) It compares several classification
approaches based on different document representation strategies and text classification models. (4)
The best approach consists of a combination of a LTSM+MLP neural network that takes as input the
tweet’s word, emoji, and expression tokens’ embeddings enriched by the tf-idf, and obtains an area
under the curve (AUC) of 0.828 on our dataset, outperforming previous methods presented in the
literatureThe work by Quijano-Sanchez was supported by the Spanish Ministry of Science and Innovation
grant FJCI-2016-28855. The research of Liberatore was supported by the Government of Spain, grant MTM2015-65803-R, and by the European Union’s Horizon 2020 Research and Innovation Programme, under the Marie Sklodowska-Curie grant agreement No. 691161 (GEOSAFE). All the financial support is gratefully acknowledge
Detecting Aggressiveness in Tweets: A Hybrid Model for Detecting Cyberbullying in the Spanish Language
In recent years, the use of social networks has increased exponentially, which has led to a
significant increase in cyberbullying. Currently, in the field of Computer Science, research has been
made on how to detect aggressiveness in texts, which is a prelude to detecting cyberbullying. In this
field, the main work has been done for English language texts, mainly using Machine Learning (ML)
approaches, Lexicon approaches to a lesser extent, and very few works using hybrid approaches.
In these, Lexicons and Machine Learning algorithms are used, such as counting the number of bad
words in a sentence using a Lexicon of bad words, which serves as an input feature for classification
algorithms. This research aims at contributing towards detecting aggressiveness in Spanish language
texts by creating different models that combine the Lexicons and ML approach. Twenty-two models
that combine techniques and algorithms from both approaches are proposed, and for their application,
certain hyperparameters are adjusted in the training datasets of the corpora, to obtain the best results
in the test datasets. Three Spanish language corpora are used in the evaluation: Chilean, Mexican,
and Chilean-Mexican corpora. The results indicate that hybrid models obtain the best results in the
3 corpora, over implemented models that do not use Lexicons. This shows that by mixing approaches,
aggressiveness detection improves. Finally, a web application is developed that gives applicability
to each model by classifying tweets, allowing evaluating the performance of models with external
corpus and receiving feedback on the prediction of each one for future research. In addition, an API
is available that can be integrated into technological tools for parental control, online plugins for
writing analysis in social networks, and educational tools, among others
A Retrospective Analysis of the Fake News Challenge Stance Detection Task
The 2017 Fake News Challenge Stage 1 (FNC-1) shared task addressed a stance
classification task as a crucial first step towards detecting fake news. To
date, there is no in-depth analysis paper to critically discuss FNC-1's
experimental setup, reproduce the results, and draw conclusions for
next-generation stance classification methods. In this paper, we provide such
an in-depth analysis for the three top-performing systems. We first find that
FNC-1's proposed evaluation metric favors the majority class, which can be
easily classified, and thus overestimates the true discriminative power of the
methods. Therefore, we propose a new F1-based metric yielding a changed system
ranking. Next, we compare the features and architectures used, which leads to a
novel feature-rich stacked LSTM model that performs on par with the best
systems, but is superior in predicting minority classes. To understand the
methods' ability to generalize, we derive a new dataset and perform both
in-domain and cross-domain experiments. Our qualitative and quantitative study
helps interpreting the original FNC-1 scores and understand which features help
improving performance and why. Our new dataset and all source code used during
the reproduction study are publicly available for future research
Every team deserves a second chance:an extended study on predicting team performance
Voting among different agents is a powerful tool in problem solving, and it has been widely applied to improve the performance in finding the correct answer to complex problems. We present a novel benefit of voting, that has not been observed before: we can use the voting patterns to assess the performance of a team and predict their final outcome. This prediction can be executed at any moment during problem-solving and it is completely domain independent. Hence, it can be used to identify when a team is failing, allowing an operator to take remedial procedures (such as changing team members, the voting rule, or increasing the allocation of resources). We present three main theoretical results: (1) we show a theoretical explanation of why our prediction method works; (2) contrary to what would be expected based on a simpler explanation using classical voting models, we show that we can make accurate predictions irrespective of the strength (i.e., performance) of the teams, and that in fact, the prediction can work better for diverse teams composed of different agents than uniform teams made of copies of the best agent; (3) we show that the quality of our prediction increases with the size of the action space. We perform extensive experimentation in two different domains: Computer Go and Ensemble Learning. In Computer Go, we obtain high quality predictions about the final outcome of games. We analyze the prediction accuracy for three different teams with different levels of diversity and strength, and show that the prediction works significantly better for a diverse team. Additionally, we show that our method still works well when trained with games against one adversary, but tested with games against another, showing the generality of the learned functions. Moreover, we evaluate four different board sizes, and experimentally confirm better predictions in larger board sizes. We analyze in detail the learned prediction functions, and how they change according to each team and action space size. In order to show that our method is domain independent, we also present results in Ensemble Learning, where we make online predictions about the performance of a team of classifiers, while they are voting to classify sets of items. We study a set of classical classification algorithms from machine learning, in a data-set of hand-written digits, and we are able to make high-quality predictions about the final performance of two different teams. Since our approach is domain independent, it can be easily applied to a variety of other domains
“You Know What to Do”:Proactive Detection of YouTube Videos Targeted by Coordinated Hate Attacks
Video sharing platforms like YouTube are increasingly targeted by aggression
and hate attacks. Prior work has shown how these attacks often take place as a
result of "raids," i.e., organized efforts by ad-hoc mobs coordinating from
third-party communities. Despite the increasing relevance of this phenomenon,
however, online services often lack effective countermeasures to mitigate it.
Unlike well-studied problems like spam and phishing, coordinated aggressive
behavior both targets and is perpetrated by humans, making defense mechanisms
that look for automated activity unsuitable. Therefore, the de-facto solution
is to reactively rely on user reports and human moderation.
In this paper, we propose an automated solution to identify YouTube videos
that are likely to be targeted by coordinated harassers from fringe communities
like 4chan. First, we characterize and model YouTube videos along several axes
(metadata, audio transcripts, thumbnails) based on a ground truth dataset of
videos that were targeted by raids. Then, we use an ensemble of classifiers to
determine the likelihood that a video will be raided with very good results
(AUC up to 94%). Overall, our work provides an important first step towards
deploying proactive systems to detect and mitigate coordinated hate attacks on
platforms like YouTube
Ensemble learning for ranking interesting attributes
Machine learning knowledge representations, such as decision trees; are often incomprehensible to humans. They can also contain errors specific to the representation type and the data used to generate them. By combining larger; less comprehensible decision trees, it is possible to increase their accuracy as an ensemble compared to the best individual tree. The thesis examines an ensemble learning technique and presents a unique knowledge elicitation technique which produces an ordered ranking of attributes by their importance in leading to more desirable classifications. The technique compares full branches of decision trees, finding the set difference of shared attributes. The combination of this information from all ensemble members is used to build an importance table which allows attributes to be ranked ordinally and by relative magnitude. A case study utilizing this method is discussed and its results are presented and summarized
- …