600 research outputs found
A Survey on Federated Learning Poisoning Attacks and Defenses
As one kind of distributed machine learning technique, federated learning
enables multiple clients to build a model across decentralized data
collaboratively without explicitly aggregating the data. Due to its ability to
break data silos, federated learning has received increasing attention in many
fields, including finance, healthcare, and education. However, the invisibility
of clients' training data and the local training process result in some
security issues. Recently, many works have been proposed to research the
security attacks and defenses in federated learning, but there has been no
special survey on poisoning attacks on federated learning and the corresponding
defenses. In this paper, we investigate the most advanced schemes of federated
learning poisoning attacks and defenses and point out the future directions in
these areas
Does Differential Privacy Prevent Backdoor Attacks in Practice?
Differential Privacy (DP) was originally developed to protect privacy.
However, it has recently been utilized to secure machine learning (ML) models
from poisoning attacks, with DP-SGD receiving substantial attention.
Nevertheless, a thorough investigation is required to assess the effectiveness
of different DP techniques in preventing backdoor attacks in practice. In this
paper, we investigate the effectiveness of DP-SGD and, for the first time in
literature, examine PATE in the context of backdoor attacks. We also explore
the role of different components of DP algorithms in defending against backdoor
attacks and will show that PATE is effective against these attacks due to the
bagging structure of the teacher models it employs. Our experiments reveal that
hyperparameters and the number of backdoors in the training dataset impact the
success of DP algorithms. Additionally, we propose Label-DP as a faster and
more accurate alternative to DP-SGD and PATE. We conclude that while Label-DP
algorithms generally offer weaker privacy protection, accurate hyper-parameter
tuning can make them more effective than DP methods in defending against
backdoor attacks while maintaining model accuracy
Mitigating Adversarial Attacks in Deepfake Detection: An Exploration of Perturbation and AI Techniques
Deep learning constitutes a pivotal component within the realm of machine
learning, offering remarkable capabilities in tasks ranging from image
recognition to natural language processing. However, this very strength also
renders deep learning models susceptible to adversarial examples, a phenomenon
pervasive across a diverse array of applications. These adversarial examples
are characterized by subtle perturbations artfully injected into clean images
or videos, thereby causing deep learning algorithms to misclassify or produce
erroneous outputs. This susceptibility extends beyond the confines of digital
domains, as adversarial examples can also be strategically designed to target
human cognition, leading to the creation of deceptive media, such as deepfakes.
Deepfakes, in particular, have emerged as a potent tool to manipulate public
opinion and tarnish the reputations of public figures, underscoring the urgent
need to address the security and ethical implications associated with
adversarial examples. This article delves into the multifaceted world of
adversarial examples, elucidating the underlying principles behind their
capacity to deceive deep learning algorithms. We explore the various
manifestations of this phenomenon, from their insidious role in compromising
model reliability to their impact in shaping the contemporary landscape of
disinformation and misinformation. To illustrate progress in combating
adversarial examples, we showcase the development of a tailored Convolutional
Neural Network (CNN) designed explicitly to detect deepfakes, a pivotal step
towards enhancing model robustness in the face of adversarial threats.
Impressively, this custom CNN has achieved a precision rate of 76.2% on the
DFDC dataset
On Practical Aspects of Aggregation Defenses against Data Poisoning Attacks
The increasing access to data poses both opportunities and risks in deep
learning, as one can manipulate the behaviors of deep learning models with
malicious training samples. Such attacks are known as data poisoning. Recent
advances in defense strategies against data poisoning have highlighted the
effectiveness of aggregation schemes in achieving state-of-the-art results in
certified poisoning robustness. However, the practical implications of these
approaches remain unclear. Here we focus on Deep Partition Aggregation, a
representative aggregation defense, and assess its practical aspects, including
efficiency, performance, and robustness. For evaluations, we use ImageNet
resized to a resolution of 64 by 64 to enable evaluations at a larger scale
than previous ones. Firstly, we demonstrate a simple yet practical approach to
scaling base models, which improves the efficiency of training and inference
for aggregation defenses. Secondly, we provide empirical evidence supporting
the data-to-complexity ratio, i.e. the ratio between the data set size and
sample complexity, as a practical estimation of the maximum number of base
models that can be deployed while preserving accuracy. Last but not least, we
point out how aggregation defenses boost poisoning robustness empirically
through the poisoning overfitting phenomenon, which is the key underlying
mechanism for the empirical poisoning robustness of aggregations. Overall, our
findings provide valuable insights for practical implementations of aggregation
defenses to mitigate the threat of data poisoning.Comment: 15 page
IMPOSITION: Implicit Backdoor Attack through Scenario Injection
This paper presents a novel backdoor attack called IMPlicit BackdOor Attack
through Scenario InjecTION (IMPOSITION) that does not require direct poisoning
of the training data. Instead, the attack leverages a realistic scenario from
the training data as a trigger to manipulate the model's output during
inference. This type of attack is particularly dangerous as it is stealthy and
difficult to detect. The paper focuses on the application of this attack in the
context of Autonomous Driving (AD) systems, specifically targeting the
trajectory prediction module. To implement the attack, we design a trigger
mechanism that mimics a set of cloned behaviors in the driving scene, resulting
in a scenario that triggers the attack. The experimental results demonstrate
that IMPOSITION is effective in attacking trajectory prediction models while
maintaining high performance in untargeted scenarios. Our proposed method
highlights the growing importance of research on the trustworthiness of Deep
Neural Network (DNN) models, particularly in safety-critical applications.
Backdoor attacks pose a significant threat to the safety and reliability of DNN
models, and this paper presents a new perspective on backdooring DNNs. The
proposed IMPOSITION paradigm and the demonstration of its severity in the
context of AD systems are significant contributions of this paper. We highlight
the impact of the proposed attacks via empirical studies showing how IMPOSITION
can easily compromise the safety of AD systems
- …