Comparing a Hybrid Multi-layered Machine Learning Intrusion Detection System to Single-layered and Deep Learning Models

Advancements in computing technology have created additional network attack surface, allowed the development of new attack types, and increased the impact caused by an attack. Researchers agree, current intrusion detection systems (IDSs) are not able to adapt to detect these new attack forms, so alternative IDS methods have been proposed. Among these methods are machine learning-based intrusion detection systems. This research explores the current relevant studies related to intrusion detection systems and machine learning models and proposes a new hybrid machine learning IDS model consisting of the Principal Component Analysis (PCA) and Support Vector Machine (SVM) learning algorithms. The NSL-KDD Dataset, benchmark dataset for IDSs, is used for comparing the models’ performance. The performance accuracy and false-positive rate of the hybrid model are compared to the results of the model’s individual algorithmic components to determine which components most impact attack prediction performance. The performance metrics of the hybrid model are also compared to two deep learning Autoencoder Neuro Network models and the results found that the complexity of the model does not add to the performance accuracy. The research showed that pre-processing and feature selection impact the predictive accuracy across models. Future research recommendations were to implement the proposed hybrid IDS model into a live network for testing and analysis, and to focus research into the pre-processing algorithms that improve performance accuracy, and lower false-positive rate. This research indicated that pre-processing and feature selection/feature extraction can increase model performance accuracy and decrease false-positive rate helping businesses to improve network security

CLASSIFICATION OF CYBERSECURITY INCIDENTS IN NIGERIA USING MACHINE LEARNING METHODS

Cybercrime has become more likely as a result of technological advancements and increased use of the internet and computer systems. As a result, there is an urgent need to develop effective methods of dealing with these cyber threats or incidents to identify and combat the associated cybercrimes in Nigerian cyberspace adequately. It is therefore desirable to build models that will enable the Nigeria Computer Emergency Response Team (ngCERT) and law enforcement agencies to gain valuable knowledge of insights from the available data to detect, identify and efficiently classify the most prevalent cyber incidents within Nigeria cyberspace, and predict future threats. This study applied machine learning methods to study and understand cybercrime incidents or threats recorded by ngCERT to build models that will characterize cybercrime incidents in Nigeria and classify cybersecurity incidents by mode of attacks and identify the most prevalent incidents within Nigerian cyberspace. Seven different machine learning methods were used to build the classification and prediction models. The Logistic Regression (LR), Naïve Bayes (NB), Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), K-Nearest Neighbor (KNN), Decision Tree (CART) and Random Forest (RF) Algorithms were used to discover the relationship between the relevant attributes of the datasets then classify the threats into several categories. The RF, CART, and KNN models were shown to be the most effective in classifying our data with accuracy score of 99%  each while others has accuracy scores of 98% for SVM, 89% for NB, 88% for LR, and 88% for LDA. Therefore, the result of our classification will help organizations in Nigeria to be able to understand the threats that could affect their assets

CLASSIFICATION OF CYBERSECURITY INCIDENTS IN NIGERIA USING MACHINE LEARNING METHODS

Cybercrime has become more likely as a result of technological advancements and increased use of the internet and computer systems. As a result, there is an urgent need to develop effective methods of dealing with these cyber threats or incidents to identify and combat the associated cybercrimes in Nigerian cyberspace adequately. It is therefore desirable to build models that will enable the Nigeria Computer Emergency Response Team (ngCERT) and law enforcement agencies to gain valuable knowledge of insights from the available data to detect, identify and efficiently classify the most prevalent cyber incidents within Nigeria cyberspace, and predict future threats. This study applied machine learning methods to study and understand cybercrime incidents or threats recorded by ngCERT to build models that will characterize cybercrime incidents in Nigeria and classify cybersecurity incidents by mode of attacks and identify the most prevalent incidents within Nigerian cyberspace. Seven different machine learning methods were used to build the classification and prediction models. The Logistic Regression (LR), Naïve Bayes (NB), Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), K-Nearest Neighbor (KNN), Decision Tree (CART) and Random Forest (RF) Algorithms were used to discover the relationship between the relevant attributes of the datasets then classify the threats into several categories. The RF, CART, and KNN models were shown to be the most effective in classifying our data with accuracy score of 99%  each while others has accuracy scores of 98% for SVM, 89% for NB, 88% for LR, and 88% for LDA. Therefore, the result of our classification will help organizations in Nigeria to be able to understand the threats that could affect their assets

Web server load prediction and anomaly detection from hypertext transfer protocol logs

As network traffic increases and new intrusions occur, anomaly detection solutions based on machine learning are necessary to detect previously unknown intrusion patterns. Most of the developed models require a labelled dataset, which can be challenging owing to a shortage of publicly available datasets. These datasets are often too small to effectively train machine learning models, which further motivates the use of real unlabeled traffic. By using real traffic, it is possible to more accurately simulate the types of anomalies that might occur in a real-world network and improve the performance of the detection model. We present a method able to predict and categorize anomalies without the aid of a labelled dataset, demonstrating the model’s usability while also gathering a dataset from real noisy network traffic. The proposed long short-term memory (LTSM) based intrusion detection system was tested in a real-world setting of an antivirus company and was successful in detecting various intrusions using 5-minute windowing over both the predicted and real update curves thereby demonstrating its usefulness. Our contribution was the development of a robust model generally applicable to any hypertext transfer protocol (HTTP) traffic with almost real-time anomaly detection, while also outperforming earlier studies in terms of prediction accuracy

Intrusion detection on computer networks using anomaly detection approach

Bilgisayar ağlarına yapılan saldırılar günden güne artarken ve saldırıların nitelikleri de sürekli olarak değişmektedir. Ağ saldırıları, bilgisayar ağlarına zarar vererek bilgi güvenliğini ortadan kaldırmaktadır. Bu durum kişiler, şirketler, kurumlar ve hatta devletler için büyük bir risk oluşturmaktadır. Ağ trafiğinin analizi ve böylece saldırıların ortaya çıkarılabilmesi için Saldırı Tespit Sistemlerinden yararlanılmaktadır. Saldırı türlerini tanıyacak şekilde oluşturulan bu sistemlerin gelişimleri de artan saldırı tiplerine göre sürekli devam etmektedir. Bu çalışmada makine öğrenmesi teknikleri yardımıyla anormallik tabanlı bir saldırı tespit sistemi oluşturulması amaçlanmıştır. Çalışma sürecinde; Yinelemeli Özellik Elemesi, İleri Yönelimli Seçim, Rastgele Orman, Karar Ağaçları, Naive Bayes, Lojistik Regresyon ve Ekstrem Gradyan Artırma gibi algoritmalardan yararlanılmış ve Doğruluk, Kesinlik, Duyarlılık ve F1 gibi metrikler ile değerlendirmeler yapılmıştır. Ayrıca model değerlendirme için ROC eğrilerinden yararlanılmıştır. Bahsi geçen bu algoritmalardan elde edilen sonuçlar karşılaştırılarak en etkili modelin bulunması için CICIDS 2017 veri seti kullanılmıştır. Çalışma kapsamında Yinelemeli Özellik Elemesi ve İleri Yönelimli Seçim teknikleriyle özellik seçimi yapılmış ve en iyi sınıflandırma sonuçları Rasgele Orman ve Ekstrem Gradyan Artırma algoritmalarından elde edilmiştir.Attacks on computer networks are increasing day by day and characteristics of them are changing continuously. Network attacks destroy information security by damaging computer network systems. This situation poses a great risk for individuals, companies, institutions and even governments. To prevent or minimize the damages of network attacks, Intrusion Detection Systems are used. The development of these systems, which are created according to attack characteristics, continues parallelly to increasing attack types. In this study, it is aimed to create an intrusion detection system based on machine learning principles with anomaly detection. Recursive Feature Elimination, Forward Feature Selection, Random Forest, Decision Tree, Naive Bayes, Logistic Regression and Extreme Gradient Boosting algorithms are used during the study and evaluations are made by Accuracy, Precision, Recall and F1 Score metrics. Also, Cross Validation and ROC Curve methods are used for the evaluation. CICIDS2017 data set is used to find the most effective model by comparing the results obtained from the mentioned algorithms. As the result of this study, it is determined that the Intrusion Detection System models, which are created by classifying the features obtained the methods of Forward Feature Selection and Recursive Feature Elimination with Random Forest and Extreme Gradient Boosting algorithms, are successful

Network Traffic Analysis Framework For Cyber Threat Detection

The growing sophistication of attacks and newly emerging cyber threats requires advanced cyber threat detection systems. Although there are several cyber threat detection tools in use, cyber threats and data breaches continue to rise. This research is intended to improve the cyber threat detection approach by developing a cyber threat detection framework using two complementary technologies, search engine and machine learning, combining artificial intelligence and classical technologies. In this design science research, several artifacts such as a custom search engine library, a machine learning-based engine and different algorithms have been developed to build a new cyber threat detection framework based on self-learning search and machine learning engines. Apache Lucene.Net search engine library was customized in order to function as a cyber threat detector, and Microsoft ML.NET was used to work with and train the customized search engine. This research proves that a custom search engine can function as a cyber threat detection system. Using both search and machine learning engines in the newly developed framework provides improved cyber threat detection capabilities such as self-learning and predicting attack details. When the two engines run together, the search engine is continuously trained by the machine learning engine and grow smarter to predict yet unknown threats with greater accuracy. While customizing the search engine to function as a cyber threat detector, this research also identified and proved the best algorithms for the search engine based cyber threat detection model. For example, the best scoring algorithm was found to be the Manhattan distance. The validation case study also shows that not every network traffic feature makes an equal contribution to determine the status of the traffic, and thus the variable-dimension Vector Space Model (VSM) achieves better detection accuracy than n-dimensional VSM. Although the use of different technologies and approaches improved detection results, this research is primarily focused on developing techniques rather than building a complete threat detection system. Additional components such as those that can track and investigate the impact of network traffic on the destination devices make the newly developed framework robust enough to build a comprehensive cyber threat detection appliance

Towards IP traceback based defense against DDoS attacks.

Lau Nga Sin.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 101-110).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Research Motivation --- p.2Chapter 1.2 --- Problem Statement --- p.3Chapter 1.3 --- Research Objectives --- p.4Chapter 1.4 --- Structure of the Thesis --- p.6Chapter 2 --- Background Study on DDoS Attacks --- p.8Chapter 2.1 --- Distributed Denial of Service Attacks --- p.8Chapter 2.1.1 --- DDoS Attack Architecture --- p.9Chapter 2.1.2 --- DDoS Attack Taxonomy --- p.11Chapter 2.1.3 --- DDoS Tools --- p.19Chapter 2.1.4 --- DDoS Detection --- p.21Chapter 2.2 --- DDoS Countermeasure: Attack Source Traceback --- p.23Chapter 2.2.1 --- Link Testing --- p.23Chapter 2.2.2 --- Logging --- p.24Chapter 2.2.3 --- ICMP-based traceback --- p.26Chapter 2.2.4 --- Packet marking --- p.28Chapter 2.2.5 --- Comparison of various IP Traceback Schemes --- p.31Chapter 2.3 --- DDoS Countermeasure: Packet Filtering --- p.33Chapter 2.3.1 --- Ingress Filtering --- p.33Chapter 2.3.2 --- Egress Filtering --- p.34Chapter 2.3.3 --- Route-based Packet Filtering --- p.35Chapter 2.3.4 --- IP Traceback-based Packet Filtering --- p.36Chapter 2.3.5 --- Router-based Pushback --- p.37Chapter 3 --- Domain-based IP Traceback Scheme --- p.40Chapter 3.1 --- Overview of our IP Traceback Scheme --- p.41Chapter 3.2 --- Assumptions --- p.44Chapter 3.3 --- Proposed Packet Marking Scheme --- p.45Chapter 3.3.1 --- IP Markings with Edge Sampling --- p.46Chapter 3.3.2 --- Domain-based Design Motivation --- p.48Chapter 3.3.3 --- Mathematical Principle --- p.49Chapter 3.3.4 --- Marking Mechanism --- p.51Chapter 3.3.5 --- Storage Space of the Marking Fields --- p.56Chapter 3.3.6 --- Packet Marking Integrity --- p.57Chapter 3.3.7 --- Path Reconstruction --- p.58Chapter 4 --- Route-based Packet Filtering Scheme --- p.62Chapter 4.1 --- Placement of Filters --- p.63Chapter 4.1.1 --- At Sources' Networks --- p.64Chapter 4.1.2 --- At Victim's Network --- p.64Chapter 4.2 --- Proposed Packet Filtering Scheme --- p.65Chapter 4.2.1 --- Classification of Packets --- p.66Chapter 4.2.2 --- Filtering Mechanism --- p.67Chapter 5 --- Performance Evaluation --- p.70Chapter 5.1 --- Simulation Setup --- p.70Chapter 5.2 --- Experiments on IP Traceback Scheme --- p.72Chapter 5.2.1 --- Performance Metrics --- p.72Chapter 5.2.2 --- Choice of Marking Probabilities --- p.73Chapter 5.2.3 --- Experimental Results --- p.75Chapter 5.3 --- Experiments on Packet Filtering Scheme --- p.82Chapter 5.3.1 --- Performance Metrics --- p.82Chapter 5.3.2 --- Choices of Filtering Probabilities --- p.84Chapter 5.3.3 --- Experimental Results --- p.85Chapter 5.4 --- Deployment Issues --- p.91Chapter 5.4.1 --- Backward Compatibility --- p.91Chapter 5.4.2 --- Processing Overheads to the Routers and Network --- p.93Chapter 5.5 --- Evaluations --- p.95Chapter 6 --- Conclusion --- p.96Chapter 6.1 --- Contributions --- p.96Chapter 6.2 --- Discussions and future work --- p.99Bibliography --- p.11

Internet of Things Applications, Security Challenges, Attacks, Intrusion Detection, and Future Visions : A Systematic Review

Internet of Things (IoT) technology is prospering and entering every part of our lives, be it education, home, vehicles, or healthcare. With the increase in the number of connected devices, several challenges are also coming up with IoT technology: heterogeneity, scalability, quality of service, security requirements, and many more. Security management takes a back seat in IoT because of cost, size, and power. It poses a significant risk as lack of security makes users skeptical towards using IoT devices. This, in turn, makes IoT vulnerable to security attacks, ultimately causing enormous financial and reputational losses. It makes up for an urgent need to assess present security risks and discuss the upcoming challenges to be ready to face the same. The undertaken study is a multi-fold survey of different security issues present in IoT layers: perception layer, network layer, support layer, application layer, with further focus on Distributed Denial of Service (DDoS) attacks. DDoS attacks are significant threats for the cyber world because of their potential to bring down the victims. Different types of DDoS attacks, DDoS attacks in IoT devices, impacts of DDoS attacks, and solutions for mitigation are discussed in detail. The presented review work compares Intrusion Detection and Prevention models for mitigating DDoS attacks and focuses on Intrusion Detection models. Furthermore, the classification of Intrusion Detection Systems, different anomaly detection techniques, different Intrusion Detection System models based on datasets, various machine learning and deep learning techniques for data pre-processing and malware detection has been discussed. In the end, a broader perspective has been envisioned while discussing research challenges, its proposed solutions, and future visions