3 research outputs found
Perception Visualization: Seeing Through the Eyes of a DNN
Artificial intelligence (AI) systems power the world we live in. Deep neural
networks (DNNs) are able to solve tasks in an ever-expanding landscape of
scenarios, but our eagerness to apply these powerful models leads us to focus
on their performance and deprioritises our ability to understand them. Current
research in the field of explainable AI tries to bridge this gap by developing
various perturbation or gradient-based explanation techniques. For images,
these techniques fail to fully capture and convey the semantic information
needed to elucidate why the model makes the predictions it does. In this work,
we develop a new form of explanation that is radically different in nature from
current explanation methods, such as Grad-CAM. Perception visualization
provides a visual representation of what the DNN perceives in the input image
by depicting what visual patterns the latent representation corresponds to.
Visualizations are obtained through a reconstruction model that inverts the
encoded features, such that the parameters and predictions of the original
models are not modified. Results of our user study demonstrate that humans can
better understand and predict the system's decisions when perception
visualizations are available, thus easing the debugging and deployment of deep
models as trusted systems.Comment: Accepted paper at BMVC 2021 (Proceedings not available yet
Adversarial Scratches: Deployable Attacks to CNN Classifiers
A growing body of work has shown that deep neural networks are susceptible to
adversarial examples. These take the form of small perturbations applied to the
model's input which lead to incorrect predictions. Unfortunately, most
literature focuses on visually imperceivable perturbations to be applied to
digital images that often are, by design, impossible to be deployed to physical
targets. We present Adversarial Scratches: a novel L0 black-box attack, which
takes the form of scratches in images, and which possesses much greater
deployability than other state-of-the-art attacks. Adversarial Scratches
leverage B\'ezier Curves to reduce the dimension of the search space and
possibly constrain the attack to a specific location. We test Adversarial
Scratches in several scenarios, including a publicly available API and images
of traffic signs. Results show that, often, our attack achieves higher fooling
rate than other deployable state-of-the-art methods, while requiring
significantly fewer queries and modifying very few pixels.Comment: This paper stems from 'Scratch that! An Evolution-based Adversarial
Attack against Neural Networks' for which an arXiv preprint is available at
arXiv:1912.02316. Further studies led to a complete overhaul of the work,
resulting in this paper. This work was submitted for review in Pattern
Recognition (Elsevier
Adversarial scratches: Deployable attacks to CNN classifiers
A growing body of work has shown that deep neural networks are susceptible to adversarial examples. These take the form of small perturbations applied to the model’s input which lead to incorrect predictions. Unfortunately, most literature focuses on visually imperceivable perturbations to be applied to digital images that often are, by design, impossible to be deployed to physical targets.
We present Adversarial Scratches: a novel L0 black-box attack, which takes the form of scratches in images, and which possesses much greater deployability than other state-of-the-art attacks. Adversarial Scratches leverage BĂ©zier Curves to reduce the dimension of the search space and possibly constrain the attack to a specific location.
We test Adversarial Scratches in several scenarios, including a publicly available API and images of traffic signs. Results show that our attack achieves higher fooling rate than other deployable state-of-the-art methods, while requiring significantly fewer queries and modifying very few pixels