Anomaly-Based Intrusion Detection by Modeling Probability Distributions of Flow Characteristics

In recent years, with the increased use of network communication, the risk of compromising the information has grown immensely. Intrusions have evolved and become more sophisticated. Hence, classical detection systems show poor performance in detecting novel attacks. Although much research has been devoted to improving the performance of intrusion detection systems, few methods can achieve consistently efficient results with the constant changes in network communications. This thesis proposes an intrusion detection system based on modeling distributions of network flow statistics in order to achieve a high detection rate for known and stealthy attacks. The proposed model aggregates the traffic at the IP subnetwork level using a hierarchical heavy hitters algorithm. This aggregated traffic is used to build the distribution of network statistics for the most frequent IPv4 addresses encountered as destination. The obtained probability density functions are learned by the Extreme Learning Machine method which is a single-hidden layer feedforward neural network. In this thesis, different sequential and batch learning strategies are proposed in order to analyze the efficiency of this proposed approach. The performance of the model is evaluated on the ISCX-IDS 2012 dataset consisting of injection attacks, HTTP flooding, DDoS and brute force intrusions. The experimental results of the thesis indicate that the presented method achieves an average detection rate of 91% while having a low misclassification rate of 9%, which is on par with the state-of-the-art approaches using this dataset. In addition, the proposed method can be utilized as a network behavior analysis tool specifically for DDoS mitigation, since it can isolate aggregated IPv4 addresses from the rest of the network traffic, thus supporting filtering out DDoS attacks

Classification hardness for supervised learners on 20 years of intrusion detection data

This article consolidates analysis of established (NSL-KDD) and new intrusion detection datasets (ISCXIDS2012, CICIDS2017, CICIDS2018) through the use of supervised machine learning (ML) algorithms. The uniformity in analysis procedure opens up the option to compare the obtained results. It also provides a stronger foundation for the conclusions about the efficacy of supervised learners on the main classification task in network security. This research is motivated in part to address the lack of adoption of these modern datasets. Starting with a broad scope that includes classification by algorithms from different families on both established and new datasets has been done to expand the existing foundation and reveal the most opportune avenues for further inquiry. After obtaining baseline results, the classification task was increased in difficulty, by reducing the available data to learn from, both horizontally and vertically. The data reduction has been included as a stress-test to verify if the very high baseline results hold up under increasingly harsh constraints. Ultimately, this work contains the most comprehensive set of results on the topic of intrusion detection through supervised machine learning. Researchers working on algorithmic improvements can compare their results to this collection, knowing that all results reported here were gathered through a uniform framework. This work's main contributions are the outstanding classification results on the current state of the art datasets for intrusion detection and the conclusion that these methods show remarkable resilience in classification performance even when aggressively reducing the amount of data to learn from

Data analytics for modeling and visualizing attack behaviors: A case study on SSH brute force attacks

In this research, we explore a data analytics based approach for modeling and visualizing attack behaviors. To this end, we employ Self-Organizing Map and Association Rule Mining algorithms to analyze and interpret the behaviors of SSH brute force attacks and SSH normal traffic as a case study. The experimental results based on four different data sets show that the patterns extracted and interpreted from the SSH brute force attack data sets are similar to each other but significantly different from those extracted from the SSH normal traffic data sets. The analysis of the attack traffic provides insight into behavior modeling for brute force SSH attacks. Furthermore, this sheds light into how data analytics could help in modeling and visualizing attack behaviors in general in terms of data acquisition and feature extraction

In-depth comparative evaluation of supervised machine learning approaches for detection of cybersecurity threats

This paper describes the process and results of analyzing CICIDS2017, a modern, labeled data set for testing intrusion detection systems. The data set is divided into several days, each pertaining to different attack classes (Dos, DDoS, infiltration, botnet, etc.). A pipeline has been created that includes nine supervised learning algorithms. The goal was binary classification of benign versus attack traffic. Cross-validated parameter optimization, using a voting mechanism that includes five classification metrics, was employed to select optimal parameters. These results were interpreted to discover whether certain parameter choices were dominant for most (or all) of the attack classes. Ultimately, every algorithm was retested with optimal parameters to obtain the final classification scores. During the review of these results, execution time, both on consumerand corporate-grade equipment, was taken into account as an additional requirement. The work detailed in this paper establishes a novel supervised machine learning performance baseline for CICIDS2017

Utilizing Machine Learning Classifiers to Identify SSH Brute Force Attacks

SSH brute force attacks are a type of network attack in which an attacker tries to guess the username and password of a user on the Secure Shell protocol. This kind of attack is simple to perform, and the results from a successfully compromised system can lead to a number of destructive outcomes. Because of its simplicity and potential payout, large networks experience many instances of these attacks in their traffic, and current prevention methods rely heavily on per-machine logs that, in aggregate, take up a large amount of space. This paper explores the usage of machine learning algorithms in detecting and preventing these kinds of attacks as an alternative to the firewall techniques used today. We use three different classifiers - naïve Bayes, K-nearest neighbors, and decision trees - on a publicly available dataset of labeled network flows to try and classify unknown network flows into benign and SSH brute force categories. Our results show that machine learning is very well suited for this task, with all of our classifiers having accuracy scores of over 85% in the classification of our test data