300 research outputs found
Progressive Poisoned Data Isolation for Training-time Backdoor Defense
Deep Neural Networks (DNN) are susceptible to backdoor attacks where
malicious attackers manipulate the model's predictions via data poisoning. It
is hence imperative to develop a strategy for training a clean model using a
potentially poisoned dataset. Previous training-time defense mechanisms
typically employ an one-time isolation process, often leading to suboptimal
isolation outcomes. In this study, we present a novel and efficacious defense
method, termed Progressive Isolation of Poisoned Data (PIPD), that
progressively isolates poisoned data to enhance the isolation accuracy and
mitigate the risk of benign samples being misclassified as poisoned ones. Once
the poisoned portion of the dataset has been identified, we introduce a
selective training process to train a clean model. Through the implementation
of these techniques, we ensure that the trained model manifests a significantly
diminished attack success rate against the poisoned data. Extensive experiments
on multiple benchmark datasets and DNN models, assessed against nine
state-of-the-art backdoor attacks, demonstrate the superior performance of our
PIPD method for backdoor defense. For instance, our PIPD achieves an average
True Positive Rate (TPR) of 99.95% and an average False Positive Rate (FPR) of
0.06% for diverse attacks over CIFAR-10 dataset, markedly surpassing the
performance of state-of-the-art methods.Comment: Accepted to AAAI202
From Rank Estimation to Rank Approximation: Rank Residual Constraint for Image Restoration
In this paper, we propose a novel approach to the rank minimization problem,
termed rank residual constraint (RRC) model. Different from existing low-rank
based approaches, such as the well-known nuclear norm minimization (NNM) and
the weighted nuclear norm minimization (WNNM), which estimate the underlying
low-rank matrix directly from the corrupted observations, we progressively
approximate the underlying low-rank matrix via minimizing the rank residual.
Through integrating the image nonlocal self-similarity (NSS) prior with the
proposed RRC model, we apply it to image restoration tasks, including image
denoising and image compression artifacts reduction. Towards this end, we first
obtain a good reference of the original image groups by using the image NSS
prior, and then the rank residual of the image groups between this reference
and the degraded image is minimized to achieve a better estimate to the desired
image. In this manner, both the reference and the estimated image are updated
gradually and jointly in each iteration. Based on the group-based sparse
representation model, we further provide a theoretical analysis on the
feasibility of the proposed RRC model. Experimental results demonstrate that
the proposed RRC model outperforms many state-of-the-art schemes in both the
objective and perceptual quality
Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering
Image forgery detection aims to detect and locate forged regions in an image.
Most existing forgery detection algorithms formulate classification problems to
classify pixels into forged or pristine. However, the definition of forged and
pristine pixels is only relative within one single image, e.g., a forged region
in image A is actually a pristine one in its source image B (splicing forgery).
Such a relative definition has been severely overlooked by existing methods,
which unnecessarily mix forged (pristine) regions across different images into
the same category. To resolve this dilemma, we propose the FOrensic ContrAstive
cLustering (FOCAL) method, a novel, simple yet very effective paradigm based on
contrastive learning and unsupervised clustering for the image forgery
detection. Specifically, FOCAL 1) utilizes pixel-level contrastive learning to
supervise the high-level forensic feature extraction in an image-by-image
manner, explicitly reflecting the above relative definition; 2) employs an
on-the-fly unsupervised clustering algorithm (instead of a trained one) to
cluster the learned features into forged/pristine categories, further
suppressing the cross-image influence from training data; and 3) allows to
further boost the detection performance via simple feature-level concatenation
without the need of retraining. Extensive experimental results over six public
testing datasets demonstrate that our proposed FOCAL significantly outperforms
the state-of-the-art competing algorithms by big margins: +24.3% on Coverage,
+18.6% on Columbia, +17.5% on FF++, +14.2% on MISD, +13.5% on CASIA and +10.3%
on NIST in terms of IoU. The paradigm of FOCAL could bring fresh insights and
serve as a novel benchmark for the image forgery detection task. The code is
available at https://github.com/HighwayWu/FOCAL
Generalizable Synthetic Image Detection via Language-guided Contrastive Learning
The heightened realism of AI-generated images can be attributed to the rapid
development of synthetic models, including generative adversarial networks
(GANs) and diffusion models (DMs). The malevolent use of synthetic images, such
as the dissemination of fake news or the creation of fake profiles, however,
raises significant concerns regarding the authenticity of images. Though many
forensic algorithms have been developed for detecting synthetic images, their
performance, especially the generalization capability, is still far from being
adequate to cope with the increasing number of synthetic models. In this work,
we propose a simple yet very effective synthetic image detection method via a
language-guided contrastive learning and a new formulation of the detection
problem. We first augment the training images with carefully-designed textual
labels, enabling us to use a joint image-text contrastive learning for the
forensic feature extraction. In addition, we formulate the synthetic image
detection as an identification problem, which is vastly different from the
traditional classification-based approaches. It is shown that our proposed
LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved
generalizability to unseen image generation models and delivers promising
performance that far exceeds state-of-the-art competitors by +22.66% accuracy
and +15.24% AUC. The code is available at https://github.com/HighwayWu/LASTED
Under-Display Camera Image Restoration with Scattering Effect
The under-display camera (UDC) provides consumers with a full-screen visual
experience without any obstruction due to notches or punched holes. However,
the semi-transparent nature of the display inevitably introduces the severe
degradation into UDC images. In this work, we address the UDC image restoration
problem with the specific consideration of the scattering effect caused by the
display. We explicitly model the scattering effect by treating the display as a
piece of homogeneous scattering medium. With the physical model of the
scattering effect, we improve the image formation pipeline for the image
synthesis to construct a realistic UDC dataset with ground truths. To suppress
the scattering effect for the eventual UDC image recovery, a two-branch
restoration network is designed. More specifically, the scattering branch
leverages global modeling capabilities of the channel-wise self-attention to
estimate parameters of the scattering effect from degraded images. While the
image branch exploits the local representation advantage of CNN to recover
clear scenes, implicitly guided by the scattering branch. Extensive experiments
are conducted on both real-world and synthesized data, demonstrating the
superiority of the proposed method over the state-of-the-art UDC restoration
techniques. The source code and dataset are available at
\url{https://github.com/NamecantbeNULL/SRUDC}.Comment: Accepted to ICCV202
- …