100 research outputs found
Inconsistent Matters: A Knowledge-guided Dual-consistency Network for Multi-modal Rumor Detection
Rumor spreaders are increasingly utilizing multimedia content to attract the
attention and trust of news consumers. Though quite a few rumor detection
models have exploited the multi-modal data, they seldom consider the
inconsistent semantics between images and texts, and rarely spot the
inconsistency among the post contents and background knowledge. In addition,
they commonly assume the completeness of multiple modalities and thus are
incapable of handling handle missing modalities in real-life scenarios.
Motivated by the intuition that rumors in social media are more likely to have
inconsistent semantics, a novel Knowledge-guided Dual-consistency Network is
proposed to detect rumors with multimedia contents. It uses two consistency
detection subnetworks to capture the inconsistency at the cross-modal level and
the content-knowledge level simultaneously. It also enables robust multi-modal
representation learning under different missing visual modality conditions,
using a special token to discriminate between posts with visual modality and
posts without visual modality. Extensive experiments on three public real-world
multimedia datasets demonstrate that our framework can outperform the
state-of-the-art baselines under both complete and incomplete modality
conditions. Our codes are available at https://github.com/MengzSun/KDCN
TieFake: Title-Text Similarity and Emotion-Aware Fake News Detection
Fake news detection aims to detect fake news widely spreading on social media
platforms, which can negatively influence the public and the government. Many
approaches have been developed to exploit relevant information from news
images, text, or videos. However, these methods may suffer from the following
limitations: (1) ignore the inherent emotional information of the news, which
could be beneficial since it contains the subjective intentions of the authors;
(2) pay little attention to the relation (similarity) between the title and
textual information in news articles, which often use irrelevant title to
attract reader' attention. To this end, we propose a novel Title-Text
similarity and emotion-aware Fake news detection (TieFake) method by jointly
modeling the multi-modal context information and the author sentiment in a
unified framework. Specifically, we respectively employ BERT and ResNeSt to
learn the representations for text and images, and utilize publisher emotion
extractor to capture the author's subjective emotion in the news content. We
also propose a scale-dot product attention mechanism to capture the similarity
between title features and textual features. Experiments are conducted on two
publicly available multi-modal datasets, and the results demonstrate that our
proposed method can significantly improve the performance of fake news
detection. Our code is available at https://github.com/UESTC-GQJ/TieFake.Comment: Appear on IJCNN 202
Deep Multimodal Image-Repurposing Detection
Nefarious actors on social media and other platforms often spread rumors and
falsehoods through images whose metadata (e.g., captions) have been modified to
provide visual substantiation of the rumor/falsehood. This type of modification
is referred to as image repurposing, in which often an unmanipulated image is
published along with incorrect or manipulated metadata to serve the actor's
ulterior motives. We present the Multimodal Entity Image Repurposing (MEIR)
dataset, a substantially challenging dataset over that which has been
previously available to support research into image repurposing detection. The
new dataset includes location, person, and organization manipulations on
real-world data sourced from Flickr. We also present a novel, end-to-end, deep
multimodal learning model for assessing the integrity of an image by combining
information extracted from the image with related information from a knowledge
base. The proposed method is compared against state-of-the-art techniques on
existing datasets as well as MEIR, where it outperforms existing methods across
the board, with AUC improvement up to 0.23.Comment: To be published at ACM Multimeda 2018 (orals
Multi-modal transformer for fake news detection
Fake news has already become a severe problem on social media, with substantially more detrimental impacts on society than previously thought. Research on multi-modal fake news detection has substantial practical significance since online fake news that includes multimedia elements are more likely to mislead users and propagate widely than text-only fake news. However, the existing multi-modal fake news detection methods have the following problems: 1) Existing methods usually use traditional CNN models and their variants to extract image features, which cannot fully extract high-quality visual features. 2) Existing approaches usually adopt a simple concatenate approach to fuse inter-modal features, leading to unsatisfactory detection results. 3) Most fake news has large disparity in feature similarity between images and texts, yet existing models do not fully utilize this aspect. Thus, we propose a novel model (TGA) based on transformers and multi-modal fusion to address the above problems. Specifically, we extract text and image features by different transformers and fuse features by attention mechanisms. In addition, we utilize the degree of feature similarity between texts and images in the classifier to improve the performance of TGA. Experimental results on the public datasets show the effectiveness of TGA*.
* Our code is available at https://github.com/PPEXCEPED/TGA
- …