Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning

Learning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the main limitations of current work, along with the corresponding future challenges towards the design of more secure learning algorithms.Comment: Accepted for publication on Pattern Recognition, 201

Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks

Malware still constitutes a major threat in the cybersecurity landscape, also due to the widespread use of infection vectors such as documents. These infection vectors hide embedded malicious code to the victim users, facilitating the use of social engineering techniques to infect their machines. Research showed that machine-learning algorithms provide effective detection mechanisms against such threats, but the existence of an arms race in adversarial settings has recently challenged such systems. In this work, we focus on malware embedded in PDF files as a representative case of such an arms race. We start by providing a comprehensive taxonomy of the different approaches used to generate PDF malware, and of the corresponding learning-based detection systems. We then categorize threats specifically targeted against learning-based PDF malware detectors, using a well-established framework in the field of adversarial machine learning. This framework allows us to categorize known vulnerabilities of learning-based PDF malware detectors and to identify novel attacks that may threaten such systems, along with the potential defense mechanisms that can mitigate the impact of such threats. We conclude the paper by discussing how such findings highlight promising research directions towards tackling the more general challenge of designing robust malware detectors in adversarial settings

Anomaly Detection by Recombining Gated Unsupervised Experts

Inspired by mixture-of-experts models and the analysis of the hidden activations of neural networks, we introduce a novel unsupervised anomaly detection method called ARGUE. Current anomaly detection methods struggle when the training data does contain multiple notions of normal. We designed ARGUE as a combination of multiple expert networks, which specialise on parts of the input data. For its final decision, ARGUE fuses the distributed knowledge across the expert systems using a gated mixture-of-experts architecture. ARGUE achieves superior detection performance across several domains in a purely data-driven way and is more robust to noisy data sets than other state-of-the-art anomaly detection methods

ARTIFICIAL IMMUNE SYSTEMS FOR INFORMATION FILTERING: FOCUSING ON PROFILE ADAPTATION

The human immune system has characteristics such as self-organisation, robustness and adaptivity that may be useful in the development of adaptive systems. One suitable application area for adaptive systems is Information Filtering (IF). Within the context of IF, learning and adapting user profiles is an important research area. In an individual profile, an IF system has to rely on the ability of the user profile to maintain a satisfactory level of filtering accuracy for as long as it is being used. This thesis explores a possible way to enable Artificial Immune Systems (AIS) to filter information in the context of profile adaptation. Previous work has investigated this issue from the perspective of self-organisation based on Autopoetic Theory. In contrast, this current work approaches the problem from the perspective of diversity inspired by the concept of dynamic clonal selection and gene library to maintain sufficient diversity. An immune inspired IF for profile adaptation is proposed and developed. This algorithm is demonstrated to work in detecting relevant documents by using a single profile to recognize a user’s interests and to adapt to changes in them. We employed a virtual user tested on a web document corpus to test the profile on learning of an emerging new topic of interest and forgetting uninteresting topics. The results clearly indicate the profile’s ability to adapt to frequent variations and radical changes in user interest. This work has focused on textual information, but it may have the potential to be applied in other media such as audio and images in which adaptivity to dynamic environments is crucial. These are all interesting future directions in which this work might develop

Securing future decentralised industrial IoT infrastructures: challenges and free open source solutions

peer-reviewedThe next industrial revolution is said to be paved by the use of novel Internet of Things (IoT) technology. One important aspect of the modern IoT infrastructures is decentralised communication, often called Peer-to-Peer (P2P). In the context of industrial communication, P2P contributes to resilience and improved stability for industrial components. Current industrial facilities, however, still rely on centralised networking schemes which are considered to be mandatory to comply with security standards. In order to succeed, introduced industrial P2P technology must maintain the current level of protection and also consider possible new threats. The presented work starts with a short analysis of well-established industrial communication infrastructures and how these could benefit from decentralised structures. Subsequently, previously undefined Information Technology (IT) security requirements are derived from the new cloud based decentralised industrial automation model architecture presented in this paper. To meet those requirements, state-of-the-art communication schemes and their open source implementations are presented and assessed for their usability in the context of industrial IoT. Finally, derived building blocks for industrial IoT P2P security are presented which are qualified to comply with the stated industrial IoT security requirements