2,185 research outputs found
EMShepherd: Detecting Adversarial Samples via Side-channel Leakage
Deep Neural Networks (DNN) are vulnerable to adversarial perturbations-small
changes crafted deliberately on the input to mislead the model for wrong
predictions. Adversarial attacks have disastrous consequences for deep
learning-empowered critical applications. Existing defense and detection
techniques both require extensive knowledge of the model, testing inputs, and
even execution details. They are not viable for general deep learning
implementations where the model internal is unknown, a common 'black-box'
scenario for model users. Inspired by the fact that electromagnetic (EM)
emanations of a model inference are dependent on both operations and data and
may contain footprints of different input classes, we propose a framework,
EMShepherd, to capture EM traces of model execution, perform processing on
traces and exploit them for adversarial detection. Only benign samples and
their EM traces are used to train the adversarial detector: a set of EM
classifiers and class-specific unsupervised anomaly detectors. When the victim
model system is under attack by an adversarial example, the model execution
will be different from executions for the known classes, and the EM trace will
be different. We demonstrate that our air-gapped EMShepherd can effectively
detect different adversarial attacks on a commonly used FPGA deep learning
accelerator for both Fashion MNIST and CIFAR-10 datasets. It achieves a 100%
detection rate on most types of adversarial samples, which is comparable to the
state-of-the-art 'white-box' software-based detectors
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
Learning-based pattern classifiers, including deep networks, have shown
impressive performance in several application domains, ranging from computer
vision to cybersecurity. However, it has also been shown that adversarial input
perturbations carefully crafted either at training or at test time can easily
subvert their predictions. The vulnerability of machine learning to such wild
patterns (also referred to as adversarial examples), along with the design of
suitable countermeasures, have been investigated in the research field of
adversarial machine learning. In this work, we provide a thorough overview of
the evolution of this research area over the last ten years and beyond,
starting from pioneering, earlier work on the security of non-deep learning
algorithms up to more recent work aimed to understand the security properties
of deep learning algorithms, in the context of computer vision and
cybersecurity tasks. We report interesting connections between these
apparently-different lines of work, highlighting common misconceptions related
to the security evaluation of machine-learning algorithms. We review the main
threat models and attacks defined to this end, and discuss the main limitations
of current work, along with the corresponding future challenges towards the
design of more secure learning algorithms.Comment: Accepted for publication on Pattern Recognition, 201
- …