In-depth comparative evaluation of supervised machine learning approaches for detection of cybersecurity threats

This paper describes the process and results of analyzing CICIDS2017, a modern, labeled data set for testing intrusion detection systems. The data set is divided into several days, each pertaining to different attack classes (Dos, DDoS, infiltration, botnet, etc.). A pipeline has been created that includes nine supervised learning algorithms. The goal was binary classification of benign versus attack traffic. Cross-validated parameter optimization, using a voting mechanism that includes five classification metrics, was employed to select optimal parameters. These results were interpreted to discover whether certain parameter choices were dominant for most (or all) of the attack classes. Ultimately, every algorithm was retested with optimal parameters to obtain the final classification scores. During the review of these results, execution time, both on consumerand corporate-grade equipment, was taken into account as an additional requirement. The work detailed in this paper establishes a novel supervised machine learning performance baseline for CICIDS2017

Review of Detection Denial of Service Attacks using Machine Learning through Ensemble Learning

Today's network hacking is more resource-intensive because the goal is to prohibit the user from using the network's resources when the target is either offensive or for financial gain, especially in businesses and organizations. That relies on the Internet like Amazon Due to this, several techniques, such as artificial intelligence algorithms like machine learning (ML) and deep learning (DL), have been developed to identify intrusion and network infiltration and discriminate between legitimate and unauthorized users. Application of machine learning and ensemble learning algorithms to various datasets, consideration of homogeneous ensembles using a single algorithm type or heterogeneous ensembles using several algorithm types, and evaluation of the discovery outcomes in terms of accuracy or discovery error for detecting attacks. The survey literature provides an overview of the many approaches and approaches of one or more machine-learning algorithms used in various datasets to identify denial of service attacks. It has also been shown that employing the hybrid approach is the most common and produces better attack detection outcomes than using the sole approaches. Numerous machine learning techniques, including support vector machines (SVM), K-Nearest Neighbors (KNN), and ensemble learning like random forest (RF), bagging, and boosting, are illustrated in this work (DT). That is employed in several articles to identify different denial of service (DoS) assaults, including the trojan horse, teardrop, land, smurf, flooding, and worm. That attacks network traffic and resources to deny users access to the resources or to steal confidential information from the company without damaging the system and employs several algorithms to obtain high attack detection accuracy and low false alarm rates

Detection and Explanation of Distributed Denial of Service (DDoS) Attack Through Interpretable Machine Learning

Distributed denial of service (DDoS) is a network-based attack where the aim of the attacker is to overwhelm the victim server. The attacker floods the server by sending enormous amount of network packets in a distributed manner beyond the servers capacity and thus causing the disruption of its normal service. In this dissertation, we focus to build intelligent detectors that can learn by themselves with less human interactions and detect DDoS attacks accurately. Machine learning (ML) has promising outcomes throughout the technologies including cybersecurity and provides us with intelligence when applied on Intrusion Detection Systems (IDSs). In addition, from the state-of-the-art ML-based IDSs, the Ensemble classifier (combination of classifiers) outperforms single classifier. Therefore, we have implemented both supervised and unsupervised ensemble frameworks to build IDSs for better DDoS detection accuracy with lower false alarms compared to the existing ones. Our experimentation, done with the most popular and benchmark datasets such as NSL-KDD, UNSW-NB15, and CICIDS2017, have achieved at most detection accuracy of 99.1% with the lowest false positive rate of 0.01%. As feature selection is one of the mandatory preprocessing phases in ML classification, we have designed several feature selection techniques for better performances in terms of DDoS detection accuracy, false positive alarms, and training times. Initially, we have implemented an ensemble framework for feature selection (FS) methods which combines almost all well-known FS methods and yields better outcomes compared to any single FS method.The goal of my dissertation is not only to detect DDoS attacks precisely but also to demonstrate explanations for these detections. Interpretable machine learning (IML) technique is used to explain a detected DDoS attack with the help of the effectiveness of the corresponding features. We also have implemented a novel feature selection approach based on IML which helps to find optimum features that are used further to retrain our models. The retrained model gives better performances than general feature selection process. Moreover, we have developed an explainer model using IML that identifies detected DDoS attacks with proper explanations based on effectiveness of the features. The contribution of this dissertation is five-folded with the ultimate goal of detecting the most frequent DDoS attacks in cyber security. In order to detect DDoS attacks, we first used ensemble machine learning classification with both supervised and unsupervised classifiers. For better performance, we then implemented and applied two feature selection approaches, such as ensemble feature selection framework and IML based feature selection approach, both individually and in a combination with supervised ensemble framework. Furthermore, we exclusively added explanations for the detected DDoS attacks with the help of explainer models that are built using LIME and SHAP IML methods. To build trustworthy explainer models, a detailed survey has been conducted on interpretable machine learning methods and on their associated tools. We applied the designed framework in various domains, like smart grid and NLP-based IDS to verify its efficacy and ability of performing as a generic model

Classification hardness for supervised learners on 20 years of intrusion detection data

This article consolidates analysis of established (NSL-KDD) and new intrusion detection datasets (ISCXIDS2012, CICIDS2017, CICIDS2018) through the use of supervised machine learning (ML) algorithms. The uniformity in analysis procedure opens up the option to compare the obtained results. It also provides a stronger foundation for the conclusions about the efficacy of supervised learners on the main classification task in network security. This research is motivated in part to address the lack of adoption of these modern datasets. Starting with a broad scope that includes classification by algorithms from different families on both established and new datasets has been done to expand the existing foundation and reveal the most opportune avenues for further inquiry. After obtaining baseline results, the classification task was increased in difficulty, by reducing the available data to learn from, both horizontally and vertically. The data reduction has been included as a stress-test to verify if the very high baseline results hold up under increasingly harsh constraints. Ultimately, this work contains the most comprehensive set of results on the topic of intrusion detection through supervised machine learning. Researchers working on algorithmic improvements can compare their results to this collection, knowing that all results reported here were gathered through a uniform framework. This work's main contributions are the outstanding classification results on the current state of the art datasets for intrusion detection and the conclusion that these methods show remarkable resilience in classification performance even when aggressively reducing the amount of data to learn from

Towards Effective Detection of Botnet Attacks using BoT-IoT Dataset

In the world of cybersecurity, intrusion detection systems (IDS) have leveraged the power of artificial intelligence for the efficient detection of attacks. This is done by applying supervised machine learning (ML) techniques on labeled datasets. A growing body of literature has been devoted to the use of BoT-IoT dataset for IDS based ML frameworks. A few number of related works have recognized the need for a balanced dataset and applied techniques to alleviate the issue of imbalance. However, a significant amount of related research works failed to treat the imbalance in the BoT-IoT dataset. A lack of unanimity was observed in the literature towards the definition of taxonomy for balancing techniques. The study presented here seeks to explore the degree to which the imbalance of the dataset has been treated and to determine the taxonomy of techniques used. In this thesis, a comparison analysis is performed by using a small subset of an entire dataset to determine the threshold sample limit at which the model achieves the highest accuracy. In addition to this analysis, a study was conducted to determine the extent to which each feature of the dataset has an impact on the threshold performance. The study is implemented on the BoT-IoT dataset using three supervised ML classifiers: K-nearest Neighbor, Random Forest, and Logistic Regression. The four principal findings of this thesis are: existing taxonomies are not understood and imbalance of the dataset is not treated; high performance across all metrics is achieved on a highly imbalanced dataset; model is able to achieve the threshold performance using a small subset of samples; certain features had varying impact on the threshold value using different techniques

A Novel Approach for Detection of DoS / DDoS Attack in Network Environment using Ensemble Machine Learning Model

One of the most  serious threat to network security is Denial of service (DOS) attacks. Internet and computer networks are now important parts of our businesses and daily lives. Malicious actions have become more common as our reliance on computers and communication networks has grown. Network threats are a big problem in the way people communicate today. To make sure that the networks work well and that users' information is safe, the network data must be watched and analysed to find malicious activities and attacks. Flooding may be the simplest DDoS assault. Computer networks and services are vulnerable to DoS and DDoS attacks. These assaults flood target systems with malicious traffic, making them unreachable to genuine users. The work aims to enhance the resilience of network infrastructures against these attacks and ensure uninterrupted service delivery. This research develops and evaluates enhanced DoS/DDoS detection methods. DoS attacks usually stop or slow down legal computer or network use. Denial-of-service (DoS) attacks prevent genuine users from accessing and using information systems and resources. The OSI model's layers make up the computer network. Different types of DDoS strikes target different layers. The Network Layer can be broken by using ICMP Floods or Smurf Attacks. The Transport layer can be attacked using UDP Floods, TCP Connection Exhaustion, and SYN Floods. HTTP-encrypted attacks can be used to get through to the application layer. DoS/DDoS attacks are malicious attacks. Protect network data from harm. Computer network services are increasingly threatened by DoS/DDoS attacks. Machine learning may detect prior DoS/DDoS attacks. DoS/DDoS attacks proliferate online and via social media. Network security is IT's top priority. DoS and DDoS assaults include ICMP, UDP, and the more prevalent TCP flood attacks. These strikes must be identified and stopped immediately. In this work, a stacking ensemble method is suggested for detecting DoS/DDoS attacks so that our networked data doesn't get any worse. This paper used a method called "Ensemble of classifiers," in which each class uses a different way to learn. In proposed  methodology Experiment#1 , I used the Home Wifi Network Traffic Collected and generated own Dataset named it as MywifiNetwork.csv, whereas in proposed methodology Experiment#2, I used the kaggle repository “NSL-KDD benchmark dataset” to perform experiments in order to find detection accuracy of dos attack detection using python language in jupyter notebook. The system detects attack-type or legitimate-type of network traffic during detection ML classification methods are used to compare how well the suggested system works. The results show that when the ensembled stacking learning model is used, 99% of the time it is able to find the problem. In proposed methodology two Experiments are implemented for comparing detection accuracy with the existing techniques. Compared to other measuring methods, we get a big step forward in finding attacks. So, our model gives a lot of faith in securing these networks. This paper will analyse the behaviour of network traffics

Ensemble Method for Anomaly Detection On the Internet of Things

 The internet of things generates various types of data traffic with a very large amount of data traffic which has an impact on security issues, one of which is an attack on the Internet of Things network. In the IoT data traffic flow, which contains various data, it turns out that the portion of attack data traffic is usually smaller than normal traffic. Therefore, the attack detection method must be able to recognize the type of attack on a very large data traffic flow and unbalanced data. High data dimensions and unbalanced data are one of the challenges in detecting attacks. To overcome the large data dimensions, Chi-square was chosen as a feature selection technique. In this study, the ensemble method is proposed to improve the ability to detect anomalies in unbalanced data. To produce an ideal detection method, a combination of several classification algorithms such as Bayes Network, Naive Bayes, REPtree and J48 is used. The CICIDS-2017 dataset is used as experimental data because it has a high data dimension which contains unbalanced data. The test results show that the proposed Ensemble method can improve the performance of anomaly detection for high-dimensional data containing unbalanced dat