Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey

Adversarial attacks and defenses in machine learning and deep neural network have been gaining significant attention due to the rapidly growing applications of deep learning in the Internet and relevant scenarios. This survey provides a comprehensive overview of the recent advancements in the field of adversarial attack and defense techniques, with a focus on deep neural network-based classification models. Specifically, we conduct a comprehensive classification of recent adversarial attack methods and state-of-the-art adversarial defense techniques based on attack principles, and present them in visually appealing tables and tree diagrams. This is based on a rigorous evaluation of the existing works, including an analysis of their strengths and limitations. We also categorize the methods into counter-attack detection and robustness enhancement, with a specific focus on regularization-based methods for enhancing robustness. New avenues of attack are also explored, including search-based, decision-based, drop-based, and physical-world attacks, and a hierarchical classification of the latest defense methods is provided, highlighting the challenges of balancing training costs with performance, maintaining clean accuracy, overcoming the effect of gradient masking, and ensuring method transferability. At last, the lessons learned and open challenges are summarized with future research opportunities recommended.Comment: 46 pages, 21 figure

Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks

Tight estimation of the Lipschitz constant for deep neural networks (DNNs) is useful in many applications ranging from robustness certification of classifiers to stability analysis of closed-loop systems with reinforcement learning controllers. Existing methods in the literature for estimating the Lipschitz constant suffer from either lack of accuracy or poor scalability. In this paper, we present a convex optimization framework to compute guaranteed upper bounds on the Lipschitz constant of DNNs both accurately and efficiently. Our main idea is to interpret activation functions as gradients of convex potential functions. Hence, they satisfy certain properties that can be described by quadratic constraints. This particular description allows us to pose the Lipschitz constant estimation problem as a semidefinite program (SDP). The resulting SDP can be adapted to increase either the estimation accuracy (by capturing the interaction between activation functions of different layers) or scalability (by decomposition and parallel implementation). We illustrate the utility of our approach with a variety of experiments on randomly generated networks and on classifiers trained on the MNIST and Iris datasets. In particular, we experimentally demonstrate that our Lipschitz bounds are the most accurate compared to those in the literature. We also study the impact of adversarial training methods on the Lipschitz bounds of the resulting classifiers and show that our bounds can be used to efficiently provide robustness guarantees

Neuron Sensitivity Guided Test Case Selection for Deep Learning Testing

Deep Neural Networks~(DNNs) have been widely deployed in software to address various tasks~(e.g., autonomous driving, medical diagnosis). However, they could also produce incorrect behaviors that result in financial losses and even threaten human safety. To reveal the incorrect behaviors in DNN and repair them, DNN developers often collect rich unlabeled datasets from the natural world and label them to test the DNN models. However, properly labeling a large number of unlabeled datasets is a highly expensive and time-consuming task. To address the above-mentioned problem, we propose NSS, Neuron Sensitivity guided test case Selection, which can reduce the labeling time by selecting valuable test cases from unlabeled datasets. NSS leverages the internal neuron's information induced by test cases to select valuable test cases, which have high confidence in causing the model to behave incorrectly. We evaluate NSS with four widely used datasets and four well-designed DNN models compared to SOTA baseline methods. The results show that NSS performs well in assessing the test cases' probability of fault triggering and model improvement capabilities. Specifically, compared with baseline approaches, NSS obtains a higher fault detection rate~(e.g., when selecting 5\% test case from the unlabeled dataset in MNIST \& LeNet1 experiment, NSS can obtain 81.8\% fault detection rate, 20\% higher than baselines)

DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessment

Quantization is one of the most applied Deep Neural Network (DNN) compression strategies, when deploying a trained DNN model on an embedded system or a cell phone. This is owing to its simplicity and adaptability to a wide range of applications and circumstances, as opposed to specific Artificial Intelligence (AI) accelerators and compilers that are often designed only for certain specific hardware (e.g., Google Coral Edge TPU). With the growing demand for quantization, ensuring the reliability of this strategy is becoming a critical challenge. Traditional testing methods, which gather more and more genuine data for better assessment, are often not practical because of the large size of the input space and the high similarity between the original DNN and its quantized counterpart. As a result, advanced assessment strategies have become of paramount importance. In this paper, we present DiverGet, a search-based testing framework for quantization assessment. DiverGet defines a space of metamorphic relations that simulate naturally-occurring distortions on the inputs. Then, it optimally explores these relations to reveal the disagreements among DNNs of different arithmetic precision. We evaluate the performance of DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing images. We chose the remote sensing DNNs as they're being increasingly deployed at the edge (e.g., high-lift drones) in critical domains like climate change research and astronomy. Our results show that DiverGet successfully challenges the robustness of established quantization techniques against naturally-occurring shifted data, and outperforms its most recent concurrent, DiffChaser, with a success rate that is (on average) four times higher.Comment: Accepted for publication in The Empirical Software Engineering Journal (EMSE

Deep Learning Approach For Sign Language Recognition

Sign language is a method of communication that uses hand movements between fellow people with hearing loss. Problems occur when communication between normal people with hearing disorders, because not everyone understands sign language, so the model is needed for sign language recognition. This study aims to make the model of the introduction of hand sign language using a deep learning approach. The model used is Convolutional Neural Network (CNN). This model is tested using the ASL alphabet database consisting of 27 categories, where each category consists of 3000 images or a total of 87,000 images of 200 x 200 pixels of hand signals. First is the process of resizing the image input to 32 x 32 pixels. Furthermore, separating the dataset for training and validation respectively 75% and 25%. The test results indicate that the proposed model has good performance with a value of 99% accuracy. Experiment results show that preprocessing images using background correction can improve model performance

FMT: Removing Backdoor Feature Maps via Feature Map Testing in Deep Neural Networks

Deep neural networks have been widely used in many critical applications, such as autonomous vehicles and medical diagnosis. However, their security is threatened by backdoor attack, which is achieved by adding artificial patterns to specific training data. Existing defense strategies primarily focus on using reverse engineering to reproduce the backdoor trigger generated by attackers and subsequently repair the DNN model by adding the trigger into inputs and fine-tuning the model with ground-truth labels. However, once the trigger generated by the attackers is complex and invisible, the defender can not successfully reproduce the trigger. Consequently, the DNN model will not be repaired since the trigger is not effectively removed. In this work, we propose Feature Map Testing~(FMT). Different from existing defense strategies, which focus on reproducing backdoor triggers, FMT tries to detect the backdoor feature maps, which are trained to extract backdoor information from the inputs. After detecting these backdoor feature maps, FMT will erase them and then fine-tune the model with a secure subset of training data. Our experiments demonstrate that, compared to existing defense strategies, FMT can effectively reduce the Attack Success Rate (ASR) even against the most complex and invisible attack triggers. Second, unlike conventional defense methods that tend to exhibit low Robust Accuracy (i.e., the model's accuracy on the poisoned data), FMT achieves higher RA, indicating its superiority in maintaining model performance while mitigating the effects of backdoor attacks~(e.g., FMT obtains 87.40\% RA in CIFAR10). Third, compared to existing feature map pruning techniques, FMT can cover more backdoor feature maps~(e.g., FMT removes 83.33\% of backdoor feature maps from the model in the CIFAR10 \& BadNet scenario).Comment: 12 pages, 4 figure

Enhancing Security in Internet of Healthcare Application using Secure Convolutional Neural Network

The ubiquity of Internet of Things (IoT) devices has completely changed the healthcare industry by presenting previously unheard-of potential for remote patient monitoring and individualized care. In this regard, we suggest a unique method that makes use of Secure Convolutional Neural Networks (SCNNs) to improve security in Internet-of-Healthcare (IoH) applications. IoT-enabled healthcare has advanced as a result of the integration of IoT technologies, giving it impressive data processing powers and large data storage capacity. This synergy has led to the development of an intelligent healthcare system that is intended to remotely monitor a patient's medical well-being via a wearable device as a result of the ongoing advancement of the Industrial Internet of Things (IIoT). This paper focuses on safeguarding user privacy and easing data analysis. Sensitive data is carefully separated from user-generated data before being gathered. Convolutional neural network (CNN) technology is used to analyse health-related data thoroughly in the cloud while scrupulously protecting the privacy of the consumers.The paper provide a secure access control module that functions using user attributes within the IoT-Healthcare system to strengthen security. This module strengthens the system's overall security and privacy by ensuring that only authorised personnel may access and interact with the sensitive health data. The IoT-enabled healthcare system gets the capacity to offer seamless remote monitoring while ensuring the confidentiality and integrity of user information thanks to this integrated architecture

Towards Robust Deep Neural Networks

Deep neural networks (DNNs) enable state-of-the-art performance for most machine learning tasks. Unfortunately, they are vulnerable to attacks, such as Trojans during training and Adversarial Examples at test time. Adversarial Examples are inputs with carefully crafted perturbations added to benign samples. In the Computer Vision domain, while the perturbations being imperceptible to humans, Adversarial Examples can successfully misguide or fool DNNs. Meanwhile, Trojan or backdoor attacks involve attackers tampering with the training process, for example, to inject poisoned training data to embed a backdoor into the network that can be activated during model deployment when the Trojan triggers (known only to the attackers) appear in the model’s inputs. This dissertation investigates methods of building robust DNNs against these training-time and test-time threats. Recognising the threat of Adversarial Examples in the malware domain, this research considers the problem of realising a robust DNN-based malware detector against Adversarial Example attacks by developing a Bayesian adversarial learning algorithm. In contrast to vision tasks, adversarial learning in a domain without a differentiable or invertible mapping function from the problemspace (such as software code inputs) to the feature space is hard. The study proposes an alternative; performing adversarial learning in the feature space and proving the projection of perturbed yet, valid malware, in the problem space into the feature space will be a subset of feature-space adversarial attacks. The Bayesian approach improves benign performance, provably bounds the difference between adversarial risk and empirical risk and improves robustness against increasingly large attack budgets not employed during training. To investigate the problem of improving the robustness of DNNs against Adversarial Examples–carefully crafted perturbation added to inputs—in the Computer Vision domain, the research considers the problem of developing a Bayesian learning algorithm to realise a robust DNN against Adversarial Examples in the CV domain. Accordingly, a novel Bayesian learning method is designed that conceptualises an information gain objective to measure and force the information learned from both benign and Adversarial Examples to be similar. This method proves that minimising this information gain objective further tightens the bound of the difference between adversarial risk and empirical risk to move towards a basis for a principled method of adversarially training BNNs. Recognising the threat from backdoor or Trojan attacks against DNNs, the research considers the problem of finding a robust defence method that is effective against Trojan attacks. The research explores a new idea in the domain; sanitisation of inputs and proposes Februus to neutralise highly potent and insidious Trojan attacks on DNN systems at run-time. In Trojan attacks, an adversary activates a backdoor crafted in a deep neural network model using a secret trigger, a Trojan, applied to any input to alter the model’s decision to a target prediction—a target determined by and only known to the attacker. Februus sanitises the incoming input by surgically removing the potential trigger artifacts and restoring the input for the classification task. Februus enables effective Trojan mitigation by sanitising inputs with no loss of performance for sanitised inputs, trojaned or benign. This method is highly effective at defending against advanced Trojan attack variants as well as challenging, adaptive attacks where attackers have full knowledge of the defence method. Investigating the connections between Trojan attacks and spatially constrained Adversarial Examples or so-called Adversarial Patches in the input space, the research exposes an emerging threat; an attack exploiting the vulnerability of a DNN to generate naturalistic adversarial patches as universal triggers. For the first time, a method based on Generative Adversarial Networks is developed to exploit a GAN’s latent space to search for universal naturalistic adversarial patches. The proposed attack’s advantage is its ability to exert a high level of control, enabling attackers to craft naturalistic adversarial patches that are highly effective, robust against state-of-the-art DNNs, and deployable in the physical world without needing to interfere with the model building process or risking discovery. Until now, this has only been demonstrably possible using Trojan attack methods.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202