55 research outputs found
Deep Multimodal Image-Repurposing Detection
Nefarious actors on social media and other platforms often spread rumors and
falsehoods through images whose metadata (e.g., captions) have been modified to
provide visual substantiation of the rumor/falsehood. This type of modification
is referred to as image repurposing, in which often an unmanipulated image is
published along with incorrect or manipulated metadata to serve the actor's
ulterior motives. We present the Multimodal Entity Image Repurposing (MEIR)
dataset, a substantially challenging dataset over that which has been
previously available to support research into image repurposing detection. The
new dataset includes location, person, and organization manipulations on
real-world data sourced from Flickr. We also present a novel, end-to-end, deep
multimodal learning model for assessing the integrity of an image by combining
information extracted from the image with related information from a knowledge
base. The proposed method is compared against state-of-the-art techniques on
existing datasets as well as MEIR, where it outperforms existing methods across
the board, with AUC improvement up to 0.23.Comment: To be published at ACM Multimeda 2018 (orals
Bidirectional Conditional Generative Adversarial Networks
Conditional Generative Adversarial Networks (cGANs) are generative models
that can produce data samples () conditioned on both latent variables ()
and known auxiliary information (). We propose the Bidirectional cGAN
(BiCoGAN), which effectively disentangles and in the generation process
and provides an encoder that learns inverse mappings from to both and
, trained jointly with the generator and the discriminator. We present
crucial techniques for training BiCoGANs, which involve an extrinsic factor
loss along with an associated dynamically-tuned importance weight. As compared
to other encoder-based cGANs, BiCoGANs encode more accurately, and utilize
and more effectively and in a more disentangled way to generate
samples.Comment: To appear in Proceedings of ACCV 201
Do-Operation Guided Causal Representation Learning with Reduced Supervision Strength
Causal representation learning has been proposed to encode relationships
between factors presented in the high dimensional data. However, existing
methods suffer from merely using a large amount of labeled data and ignore the
fact that samples generated by the same causal mechanism follow the same causal
relationships. In this paper, we seek to explore such information by leveraging
do-operation to reduce supervision strength. We propose a framework that
implements do-operation by swapping latent cause and effect factors encoded
from a pair of inputs. Moreover, we also identify the inadequacy of existing
causal representation metrics empirically and theoretically and introduce new
metrics for better evaluation. Experiments conducted on both synthetic and real
datasets demonstrate the superiorities of our method compared with
state-of-the-art methods.Comment: NeurIPS 2022 Workshop CML4Impact Workshop Camera Read
CapsuleGAN: Generative Adversarial Capsule Network
We present Generative Adversarial Capsule Network (CapsuleGAN), a framework
that uses capsule networks (CapsNets) instead of the standard convolutional
neural networks (CNNs) as discriminators within the generative adversarial
network (GAN) setting, while modeling image data. We provide guidelines for
designing CapsNet discriminators and the updated GAN objective function, which
incorporates the CapsNet margin loss, for training CapsuleGAN models. We show
that CapsuleGAN outperforms convolutional-GAN at modeling image data
distribution on MNIST and CIFAR-10 datasets, evaluated on the generative
adversarial metric and at semi-supervised image classification.Comment: To appear in Proceedings of ECCV Workshop on Brain Driven Computer
Vision (BDCV) 201
MONet: Multi-scale Overlap Network for Duplication Detection in Biomedical Images
Manipulation of biomedical images to misrepresent experimental results has
plagued the biomedical community for a while. Recent interest in the problem
led to the curation of a dataset and associated tasks to promote the
development of biomedical forensic methods. Of these, the largest manipulation
detection task focuses on the detection of duplicated regions between images.
Traditional computer-vision based forensic models trained on natural images are
not designed to overcome the challenges presented by biomedical images. We
propose a multi-scale overlap detection model to detect duplicated image
regions. Our model is structured to find duplication hierarchically, so as to
reduce the number of patch operations. It achieves state-of-the-art performance
overall and on multiple biomedical image categories.Comment: To appear at ICIP 202
- …