Attention-Based Real-Time Defenses for Physical Adversarial Attacks in Vision Applications

Deep neural networks exhibit excellent performance in computer vision tasks, but their vulnerability to real-world adversarial attacks, achieved through physical objects that can corrupt their predictions, raises serious security concerns for their application in safety-critical domains. Existing defense methods focus on single-frame analysis and are characterized by high computational costs that limit their applicability in multi-frame scenarios, where real-time decisions are crucial. To address this problem, this paper proposes an efficient attention-based defense mechanism that exploits adversarial channel-attention to quickly identify and track malicious objects in shallow network layers and mask their adversarial effects in a multi-frame setting. This work advances the state of the art by enhancing existing over-activation techniques for real-world adversarial attacks to make them usable in real-time applications. It also introduces an efficient multi-frame defense framework, validating its efficacy through extensive experiments aimed at evaluating both defense performance and computational cost

Increasing the Confidence of Deep Neural Networks by Coverage Analysis

The great performance of machine learning algorithms and deep neural networks in several perception and control tasks is pushing the industry to adopt such technologies in safety-critical applications, as autonomous robots and self-driving vehicles. At present, however, several issues need to be solved to make deep learning methods more trustworthy, predictable, safe, and secure against adversarial attacks. Although several methods have been proposed to improve the trustworthiness of deep neural networks, most of them are tailored for specific classes of adversarial examples, hence failing to detect other corner cases or unsafe inputs that heavily deviate from the training samples. This paper presents a lightweight monitoring architecture based on coverage paradigms to enhance the model robustness against different unsafe inputs. In particular, four coverage analysis methods are proposed and tested in the architecture for evaluating multiple detection logics. Experimental results show that the proposed approach is effective in detecting both powerful adversarial examples and out-of-distribution inputs, introducing limited extra-execution time and memory requirements

Coverage-driven Safety Monitoring of Deep Neural Networks

In recent years, artificial intelligence (AI) made enormous progress thanks to the evolution of deep neural networks (DNNs), which reached human-level performance in several tasks. However, the behaviors of Deep Learning (DL) methods remain unclear and unpredictable in various situations. One of the most known threats for DNNs are adversarial examples (i.e., particular inputs that cause a model to make a false prediction). To prevent these problems, coverage techniques have been conceived for DNN to drive certification and testing algorithms. Nevertheless, even when reaching a high coverage value, several networks can still exhibit faulty behaviors during operation that were not detected during testing. This project aims at ensuring safety requirements for DNN during their operational phase by introducing a Coverage-Driven Mechanism to monitor the state of the network at inference time. The proposed tool is an extension of the Caffe framework, which provides a series of versatile mechanisms to speed up the integration and the deployment of novel algorithms. In this regard, three Runtime Safety Algorithms are integrated into the tool and tested. They aim to detect adversarial examples runtime using a new interpretation of Neuron Coverage Criteria for DNN. The experimental results show the capability of the algorithms to detect up to 99.7% adversarial examples on MNIST and 87.2% on CIFAR-10 datasets, using the LeNet and ConvNet networks, respectively

Defending from Physically-Realizable Adversarial Attacks through Internal Over-Activation Analysis

This work presents Z-Mask, a robust and effective strategy to improve the adversarial robustness of convolutional networks against physically-realizable adversarial attacks. The presented defense relies on specific Z-score analysis performed on the internal network features to detect and mask the pixels corresponding to adversarial objects in the input image. To this end, spatially contiguous activations are examined in shallow and deep layers to suggest potential adversarial regions. Such proposals are then aggregated through a multi-thresholding mechanism. The effectiveness of Z-Mask is evaluated with an extensive set of experiments carried out on models for both semantic segmentation and object detection. The evaluation is performed with both digital patches added to the input images and printed patches positioned in the real world. The obtained results confirm that Z-Mask outperforms the state-of-the-art methods in terms of both detection accuracy and overall performance of the networks under attack. Additional experiments showed that Z-Mask is also robust against possible defense-aware attacks

On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving

: The existence of real-world adversarial examples (RWAEs) (commonly in the form of patches) poses a serious threat for the use of deep learning models in safety-critical computer vision tasks such as visual perception in autonomous driving. This article presents an extensive evaluation of the robustness of semantic segmentation (SS) models when attacked with different types of adversarial patches, including digital, simulated, and physical ones. A novel loss function is proposed to improve the capabilities of attackers in inducing a misclassification of pixels. Also, a novel attack strategy is presented to improve the expectation over transformation (EOT) method for placing a patch in the scene. Finally, a state-of-the-art method for detecting adversarial patch is first extended to cope with SS models, then improved to obtain real-time performance, and eventually evaluated in real-world scenarios. Experimental results reveal that even though the adversarial effect is visible with both digital and real-world attacks, its impact is often spatially confined to areas of the image around the patch. This opens to further questions about the spatial robustness of real-time SS models