A Weakly Supervised Propagation Model for Rumor Verification and Stance Detection with Multiple Instance Learning

The diffusion of rumors on microblogs generally follows a propagation tree structure, that provides valuable clues on how an original message is transmitted and responded by users over time. Recent studies reveal that rumor detection and stance detection are two different but relevant tasks which can jointly enhance each other, e.g., rumors can be debunked by cross-checking the stances conveyed by their relevant microblog posts, and stances are also conditioned on the nature of the rumor. However, most stance detection methods require enormous post-level stance labels for training, which are labor-intensive given a large number of posts. Enlightened by Multiple Instance Learning (MIL) scheme, we first represent the diffusion of claims with bottom-up and top-down trees, then propose two tree-structured weakly supervised frameworks to jointly classify rumors and stances, where only the bag-level labels concerning claim's veracity are needed. Specifically, we convert the multi-class problem into a multiple MIL-based binary classification problem where each binary model focuses on differentiating a target stance or rumor type and other types. Finally, we propose a hierarchical attention mechanism to aggregate the binary predictions, including (1) a bottom-up or top-down tree attention layer to aggregate binary stances into binary veracity; and (2) a discriminative attention layer to aggregate the binary class into finer-grained classes. Extensive experiments conducted on three Twitter-based datasets demonstrate promising performance of our model on both claim-level rumor detection and post-level stance classification compared with state-of-the-art methods.Comment: Accepted by SIGIR 202

Understanding stance classification of BERT models : an attention-based mechanism

BERT produces state-of-the-art solutions for many natural language processing tasks at the cost of interpretability. As works discuss the value of BERT’s attention weights to this purpose, we contribute with an attention-based interpretability framework to identify the most influential words for stance classification using BERT-based models. Unlike related work, we develop a broader level of interpretability focused on the overall model behavior instead of single instances. We aggregate tokens’ attentions into words’ attention weights that are more meaningful and can be semantically related to the domain. We propose attention metrics to assess words’ influence in the correct classification of stances. We use three case studies related to COVID-19 to assess the proposed framework in a broad experimental setting encompassing six datasets and four BERT pre-trained models for Portuguese and English languages, resulting in sixteen stance classification models. Through establishing five different research questions, we obtained valuable insights on the usefulness of attention weights to interpret stance classification that allowed us to generalize our findings. Our results are independent of a particular pre-trained BERT model and comparable to those obtained using an alternative baseline method. High attention scores improve the probability of finding words that positively impact the model performance and influence the correct classification (up to 82% of identified influential words contribute to correct predictions). The influential words represent the domain and can be used to identify how the model leverages the arguments expressed to predict a stance

Deep Neural Attention for Misinformation and Deception Detection

PhD thesis in Information technologyAt present the influence of social media on society is so much that without it life seems to have no meaning for many. This kind of over-reliance on social media gives an opportunity to the anarchic elements to take undue advantage. Online misinformation and deception are vivid examples of such phenomenon. The misinformation or fake news spreads faster and wider than the true news [32]. The need of the hour is to identify and curb the spread of misinformation and misleading content automatically at the earliest. Several machine learning models have been proposed by the researchers to detect and prevent misinformation and deceptive content. However, these prior works suffer from some limitations: First, they either use feature engineering heavy methods or use intricate deep neural architectures, which are not so transparent in terms of their internal working and decision making. Second, they do not incorporate and learn the available auxiliary and latent cues and patterns, which can be very useful in forming the adequate context for the misinformation. Third, Most of the former methods perform poorly in early detection accuracy measures because of their reliance on features that are usually absent at the initial stage of news or social media posts on social networks. In this dissertation, we propose suitable deep neural attention based solutions to overcome these limitations. For instance, we propose a claim verification model, which learns embddings for the latent aspects such as author and subject of the claim and domain of the external evidence document. This enables the model to learn important additional context other than the textual content. In addition, we also propose an algorithm to extract evidential snippets out of external evidence documents, which serves as explanation of the model’s decisions. Next, we improve this model by using improved claim driven attention mechanism and also generate a topically diverse and non-redundant multi-document fact-checking summary for the claims, which helps to further interpret the model’s decision making. Subsequently, we introduce a novel method to learn influence and affinity relationships among the social media users present on the propagation paths of the news items. By modeling the complex influence relationship among the users, in addition to textual content, we learn the significant patterns pertaining to the diffusion of the news item on social network. The evaluation shows that the proposed model outperforms the other related methods in early detection performance with significant gains. Next, we propose a synthetic headline generation based headline incongruence detection model. Which uses a word-to-word mutual attention based deep semantic matching between original and synthetic news headline to detect incongruence. Further, we investigate and define a new task of incongruence detection in presence of important cardinal values in headline. For this new task, we propose a part-of-speech pattern driven attention based method, which learns requisite context for cardinal values

Classifying Stance in News Articles: Use of Attribution Relations and Source Expertise

The overarching aim behind this research is to automatically detect the stance of the body of a news article relative to the article’s headline. The news headline may not always reflect what is in the news body. The stance of a news body to its headline can be agree, disagree, discuss or unrelated (Pomerleau & Rao 2017). Central to this work is the use of a specific discourse relation, the attribution relation (AR), for detecting the stance of a news article body relative to its headline. An attribution relation is a span of text which links a source to content through a cue. For example, consider The boy said it was a spider. Here, the boy is the source, said is the cue and it was a spider is the content. This thesis also examines how the expertise of sources affects stance detection. The main research question of this work is “Can attribution relations and source expertise be useful in detecting the stance of a news article’s body towards its headline?”. To address this research question, I developed a new attribution detection model that can tag components of attribution relations in news texts. I developed a new stance detection model which uses these tags as input, rather than working on the whole article as a single piece of text, with performance comparable to state-of-the-art. Furthermore, once we add the source expertise information to our stance detection model, this has a positive effect on the F-score for stance detection (increase by 14%). The work is novel in a number of further specific ways. Firstly, it is the first time a single-step deep learning approach has been applied to AR detection and been released as open source code. Second, this is the first time that attribution relations from a news article body have been used as input for a stance detection model instead of the full text of the news article body. As part of this research, I created an extension to the Fake news challenge corpus (Pomerleau & Rao 2017) with addition of source expertise data. Finally, I separately confirmed, through an empirical study, that source expertise is positively correlated with the credibility that readers assign to claims from a source