55 research outputs found
Guarantees on learning depth-2 neural networks under a data-poisoning attack
In recent times many state-of-the-art machine learning models have been shown
to be fragile to adversarial attacks. In this work we attempt to build our
theoretical understanding of adversarially robust learning with neural nets. We
demonstrate a specific class of neural networks of finite size and a
non-gradient stochastic algorithm which tries to recover the weights of the net
generating the realizable true labels in the presence of an oracle doing a
bounded amount of malicious additive distortion to the labels. We prove (nearly
optimal) trade-offs among the magnitude of the adversarial attack, the accuracy
and the confidence achieved by the proposed algorithm.Comment: 11 page
Are You Tampering With My Data?
We propose a novel approach towards adversarial attacks on neural networks
(NN), focusing on tampering the data used for training instead of generating
attacks on trained models. Our network-agnostic method creates a backdoor
during training which can be exploited at test time to force a neural network
to exhibit abnormal behaviour. We demonstrate on two widely used datasets
(CIFAR-10 and SVHN) that a universal modification of just one pixel per image
for all the images of a class in the training set is enough to corrupt the
training procedure of several state-of-the-art deep neural networks causing the
networks to misclassify any images to which the modification is applied. Our
aim is to bring to the attention of the machine learning community, the
possibility that even learning-based methods that are personally trained on
public datasets can be subject to attacks by a skillful adversary.Comment: 18 page
- …