1,032 research outputs found
Learning the Unlearnable: Adversarial Augmentations Suppress Unlearnable Example Attacks
Unlearnable example attacks are data poisoning techniques that can be used to
safeguard public data against unauthorized use for training deep learning
models. These methods add stealthy perturbations to the original image, thereby
making it difficult for deep learning models to learn from these training data
effectively. Current research suggests that adversarial training can, to a
certain degree, mitigate the impact of unlearnable example attacks, while
common data augmentation methods are not effective against such poisons.
Adversarial training, however, demands considerable computational resources and
can result in non-trivial accuracy loss. In this paper, we introduce the
UEraser method, which outperforms current defenses against different types of
state-of-the-art unlearnable example attacks through a combination of effective
data augmentation policies and loss-maximizing adversarial augmentations. In
stark contrast to the current SOTA adversarial training methods, UEraser uses
adversarial augmentations, which extends beyond the confines of
perturbation budget assumed by current unlearning attacks and defenses. It also
helps to improve the model's generalization ability, thus protecting against
accuracy loss. UEraser wipes out the unlearning effect with error-maximizing
data augmentations, thus restoring trained model accuracies. Interestingly,
UEraser-Lite, a fast variant without adversarial augmentations, is also highly
effective in preserving clean accuracies. On challenging unlearnable CIFAR-10,
CIFAR-100, SVHN, and ImageNet-subset datasets produced with various attacks, it
achieves results that are comparable to those obtained during clean training.
We also demonstrate its efficacy against possible adaptive attacks. Our code is
open source and available to the deep learning community:
https://github.com/lafeat/ueraser.Comment: UEraser introduces adversarial augmentations to suppress unlearnable
example attacks and outperforms current defense
On-Line AdaTron Learning of Unlearnable Rules
We study the on-line AdaTron learning of linearly non-separable rules by a
simple perceptron. Training examples are provided by a perceptron with a
non-monotonic transfer function which reduces to the usual monotonic relation
in a certain limit. We find that, although the on-line AdaTron learning is a
powerful algorithm for the learnable rule, it does not give the best possible
generalization error for unlearnable problems. Optimization of the learning
rate is shown to greatly improve the performance of the AdaTron algorithm,
leading to the best possible generalization error for a wide range of the
parameter which controls the shape of the transfer function.)Comment: RevTeX 17 pages, 8 figures, to appear in Phys.Rev.
On-line learning of non-monotonic rules by simple perceptron
We study the generalization ability of a simple perceptron which learns
unlearnable rules. The rules are presented by a teacher perceptron with a
non-monotonic transfer function. The student is trained in the on-line mode.
The asymptotic behaviour of the generalization error is estimated under various
conditions. Several learning strategies are proposed and improved to obtain the
theoretical lower bound of the generalization error.Comment: LaTeX 20 pages using IOP LaTeX preprint style file, 14 figure
Protein folding using contact maps
We present the development of the idea to use dynamics in the space of
contact maps as a computational approach to the protein folding problem. We
first introduce two important technical ingredients, the reconstruction of a
three dimensional conformation from a contact map and the Monte Carlo dynamics
in contact map space. We then discuss two approximations to the free energy of
the contact maps and a method to derive energy parameters based on perceptron
learning. Finally we present results, first for predictions based on threading
and then for energy minimization of crambin and of a set of 6 immunoglobulins.
The main result is that we proved that the two simple approximations we studied
for the free energy are not suitable for protein folding. Perspectives are
discussed in the last section.Comment: 29 pages, 10 figure
What Can We Learn from Unlearnable Datasets?
In an era of widespread web scraping, unlearnable dataset methods have the
potential to protect data privacy by preventing deep neural networks from
generalizing. But in addition to a number of practical limitations that make
their use unlikely, we make a number of findings that call into question their
ability to safeguard data. First, it is widely believed that neural networks
trained on unlearnable datasets only learn shortcuts, simpler rules that are
not useful for generalization. In contrast, we find that networks actually can
learn useful features that can be reweighed for high test performance,
suggesting that image privacy is not preserved. Unlearnable datasets are also
believed to induce learning shortcuts through linear separability of added
perturbations. We provide a counterexample, demonstrating that linear
separability of perturbations is not a necessary condition. To emphasize why
linearly separable perturbations should not be relied upon, we propose an
orthogonal projection attack which allows learning from unlearnable datasets
published in ICML 2021 and ICLR 2023. Our proposed attack is significantly less
complex than recently proposed techniques.Comment: 17 pages, 9 figure
- …