Applications of Machine Learning to Threat Intelligence, Intrusion Detection and Malware

Artificial Intelligence (AI) and Machine Learning (ML) are emerging technologies with applications to many fields. This paper is a survey of use cases of ML for threat intelligence, intrusion detection, and malware analysis and detection. Threat intelligence, especially attack attribution, can benefit from the use of ML classification. False positives from rule-based intrusion detection systems can be reduced with the use of ML models. Malware analysis and classification can be made easier by developing ML frameworks to distill similarities between the malicious programs. Adversarial machine learning will also be discussed, because while ML can be used to solve problems or reduce analyst workload, it also introduces new attack surfaces

Constructing multi-layered boundary to defend against intrusive anomalies: an autonomic detection coordinator

© 2005 IEEE.An autonomic detection coordinator is developed in this paper, which constructs a multi-layered boundary to defend against host-based intrusive anomalies by correlating several observation-specific anomaly detectors. Two key observations facilitate the model formulation: First, different anomaly detectors have different detection coverage and blind spots; Second, diverse operating environments provide different kinds of information to reveal anomalies. After formulating the cooperation between basic detectors as a partially observable Markov decision process, a policy-gradient reinforcement learning algorithm is applied to search in an optimal cooperation manner, with the objective to achieve broader detection coverage and fewer false alerts. Furthermore, the coordinator’s behavior can be adjusted easily by setting a reward signal to meet the diverse demands of changing system situations. A preliminary experiment is implemented, together with some comparative studies, to demonstrate the coordinator’s performance in terms of admitted criteria.Zonghua Zhang, Hong She

Artificial intelligence in the cyber domain: Offense and defense

Artificial intelligence techniques have grown rapidly in recent years, and their applications in practice can be seen in many fields, ranging from facial recognition to image analysis. In the cybersecurity domain, AI-based techniques can provide better cyber defense tools and help adversaries improve methods of attack. However, malicious actors are aware of the new prospects too and will probably attempt to use them for nefarious purposes. This survey paper aims at providing an overview of how artificial intelligence can be used in the context of cybersecurity in both offense and defense.Web of Science123art. no. 41

Intrusion Detection in Containerized Environments

In this paper, we present the results of using Hidden Markov Models for learning the behavior of Docker containers. This is for use in anomaly-detection based intrusion detection system. Containers provide isolation between the host system and the containerized environment by efficiently packaging applications along with their dependencies. This way, containers become a portable software environment for applications to run and scale. Unlike virtual machines, containers share the same kernel as the host operating system. This is leveraged to monitor the system calls of the container from the host system for anomaly detection. Thus, the monitoring system is not required to have any knowledge about the container nature, neither does the host system or the container being monitored need to be modified

Experiments on Adaptive Techniques for Host-Based Intrusion Detection

This research explores four experiments of adaptive host-based intrusion detection (ID) techniques in an attempt to develop systems that can detect novel exploits. The technique considered to have the most potential is adaptive critic designs (ACDs) because of their utilization of reinforcement learning, which allows learning exploits that are difficult to pinpoint in sensor data. Preliminary results of ID using an ACD, an Elman recurrent neural network, and a statistical anomaly detection technique demonstrate an ability to learn to distinguish between clean and exploit data. We used the Solaris Basic Security Module (BSM) as a data source and performed considerable preprocessing on the raw data. A detection approach called generalized signature-based ID is recommended as a middle ground between signature-based ID, which has an inability to detect novel exploits, and anomaly detection, which detects too many events including events that are not exploits. The primary results of the ID experiments demonstrate the use of custom data for generalized signature-based intrusion detection and the ability of neural network-based systems to learn in this application environment

Enhancing Intrusion Detection Systems with a Hybrid Deep Learning Model and Optimized Feature Composition

Systems for detecting intrusions (IDS) are essential for protecting network infrastructures from hostile activity. Advanced methods are required since traditional IDS techniques frequently fail to properly identify sophisticated and developing assaults. In this article, we suggest a novel method for improving IDS performance through the use of a hybrid deep learning model and feature composition optimization. RNN and CNN has strengths that the proposed hybrid deep learning model leverages to efficiently capture both spatial and temporal correlations in network traffic data. The model can extract useful features from unprocessed network packets using CNNs and RNNs, giving a thorough picture of network behaviour. To increase the IDS's ability to discriminate, we also offer feature optimization strategies. We uncover the most pertinent and instructive features that support precise intrusion detection through a methodical feature selection and engineering process. In order to reduce the computational load and improve the model's efficiency without compromising detection accuracy, we also use dimensionality reduction approaches. We carried out extensive experiments using a benchmark dataset that is frequently utilized in intrusion detection research to assess the suggested approach. The outcomes show that the hybrid deep learning model performs better than conventional IDS methods, obtaining noticeably greater detection rates and lower false positive rates. The performance of model is further improved by the optimized feature composition, which offers a more accurate depiction of network traffic patterns