15 research outputs found
Augmentation Backdoors
Data augmentation is used extensively to improve model generalisation.
However, reliance on external libraries to implement augmentation methods
introduces a vulnerability into the machine learning pipeline. It is well known
that backdoors can be inserted into machine learning models through serving a
modified dataset to train on. Augmentation therefore presents a perfect
opportunity to perform this modification without requiring an initially
backdoored dataset. In this paper we present three backdoor attacks that can be
covertly inserted into data augmentation. Our attacks each insert a backdoor
using a different type of computer vision augmentation transform, covering
simple image transforms, GAN-based augmentation, and composition-based
augmentation. By inserting the backdoor using these augmentation transforms, we
make our backdoors difficult to detect, while still supporting arbitrary
backdoor functionality. We evaluate our attacks on a range of computer vision
benchmarks and demonstrate that an attacker is able to introduce backdoors
through just a malicious augmentation routine.Comment: 12 pages, 8 figure
Robust Backdoor Attacks against Deep Neural Networks in Real Physical World
Deep neural networks (DNN) have been widely deployed in various applications.
However, many researches indicated that DNN is vulnerable to backdoor attacks.
The attacker can create a hidden backdoor in target DNN model, and trigger the
malicious behaviors by submitting specific backdoor instance. However, almost
all the existing backdoor works focused on the digital domain, while few
studies investigate the backdoor attacks in real physical world. Restricted to
a variety of physical constraints, the performance of backdoor attacks in the
real physical world will be severely degraded. In this paper, we propose a
robust physical backdoor attack method, PTB (physical transformations for
backdoors), to implement the backdoor attacks against deep learning models in
the real physical world. Specifically, in the training phase, we perform a
series of physical transformations on these injected backdoor instances at each
round of model training, so as to simulate various transformations that a
backdoor may experience in real world, thus improves its physical robustness.
Experimental results on the state-of-the-art face recognition model show that,
compared with the backdoor methods that without PTB, the proposed attack method
can significantly improve the performance of backdoor attacks in real physical
world. Under various complex physical conditions, by injecting only a very
small ratio (0.5%) of backdoor instances, the attack success rate of physical
backdoor attacks with the PTB method on VGGFace is 82%, while the attack
success rate of backdoor attacks without the proposed PTB method is lower than
11%. Meanwhile, the normal performance of the target DNN model has not been
affected
Robust Backdoor Attacks on Object Detection in Real World
Deep learning models are widely deployed in many applications, such as object
detection in various security fields. However, these models are vulnerable to
backdoor attacks. Most backdoor attacks were intensively studied on classified
models, but little on object detection. Previous works mainly focused on the
backdoor attack in the digital world, but neglect the real world. Especially,
the backdoor attack's effect in the real world will be easily influenced by
physical factors like distance and illumination. In this paper, we proposed a
variable-size backdoor trigger to adapt to the different sizes of attacked
objects, overcoming the disturbance caused by the distance between the viewing
point and attacked object. In addition, we proposed a backdoor training named
malicious adversarial training, enabling the backdoor object detector to learn
the feature of the trigger with physical noise. The experiment results show
this robust backdoor attack (RBA) could enhance the attack success rate in the
real world.Comment: 22 pages, 13figure
XMAM:X-raying Models with A Matrix to Reveal Backdoor Attacks for Federated Learning
Federated Learning (FL) has received increasing attention due to its privacy
protection capability. However, the base algorithm FedAvg is vulnerable when it
suffers from so-called backdoor attacks. Former researchers proposed several
robust aggregation methods. Unfortunately, many of these aggregation methods
are unable to defend against backdoor attacks. What's more, the attackers
recently have proposed some hiding methods that further improve backdoor
attacks' stealthiness, making all the existing robust aggregation methods fail.
To tackle the threat of backdoor attacks, we propose a new aggregation
method, X-raying Models with A Matrix (XMAM), to reveal the malicious local
model updates submitted by the backdoor attackers. Since we observe that the
output of the Softmax layer exhibits distinguishable patterns between malicious
and benign updates, we focus on the Softmax layer's output in which the
backdoor attackers are difficult to hide their malicious behavior.
Specifically, like X-ray examinations, we investigate the local model updates
by using a matrix as an input to get their Softmax layer's outputs. Then, we
preclude updates whose outputs are abnormal by clustering. Without any training
dataset in the server, the extensive evaluations show that our XMAM can
effectively distinguish malicious local model updates from benign ones. For
instance, when other methods fail to defend against the backdoor attacks at no
more than 20% malicious clients, our method can tolerate 45% malicious clients
in the black-box mode and about 30% in Projected Gradient Descent (PGD) mode.
Besides, under adaptive attacks, the results demonstrate that XMAM can still
complete the global model training task even when there are 40% malicious
clients. Finally, we analyze our method's screening complexity, and the results
show that XMAM is about 10-10000 times faster than the existing methods.Comment: 23 page