Automatic Dataset Labelling and Feature Selection for Intrusion Detection Systems

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Correctly labelled datasets are commonly required. Three particular scenarios are highlighted, which showcase this need. When using supervised Intrusion Detection Systems (IDSs), these systems need labelled datasets to be trained. Also, the real nature of the analysed datasets must be known when evaluating the efficiency of the IDSs when detecting intrusions. Another scenario is the use of feature selection that works only if the processed datasets are labelled. In normal conditions, collecting labelled datasets from real networks is impossible. Currently, datasets are mainly labelled by implementing off-line forensic analysis, which is impractical because it does not allow real-time implementation. We have developed a novel approach to automatically generate labelled network traffic datasets using an unsupervised anomaly based IDS. The resulting labelled datasets are subsets of the original unlabelled datasets. The labelled dataset is then processed using a Genetic Algorithm (GA) based approach, which performs the task of feature selection. The GA has been implemented to automatically provide the set of metrics that generate the most appropriate intrusion detection results

Empirical study of automatic dataset labelling

Correctly labelled dataseis are commonly required. Three particular scenarios are highlighted, which showcase this need. One of these scenarios is when using supervised Intrusion Detection Systems (TDSs). These systems need labelled datasets for their training process. Also, the real nature of analysed datasets must be known when evaluating the efficiency of IDSs detecting intrusions. The third scenario is the use of feature selection that works only if the processed datasets are labelled. In normal conditions, collecting labelled datasets from real communication networks is impossible. In a previous work we developed a novel approach to automatically generate labelled network traffic datasets using an unsupervised anomaly based IDS. The approach was empirically proven to be an efficient unsupervised labelling approach. It was evaluated using a single dataset. This paper extends our previous work by using a greater number of datasets, gathered from a real IEEE 802.11 network testbed. The datasets are comprised of different wireless-specific attacks. This paper also proposes a new and more precise method to calculate the boundary threshold, used in the labelling process

A Comparative Study on Performance Evaluation of Intrusion Detection System through Feature Reduction for High Speed Networks

Abstract- The rapid growth in the usage of the internet had led to many serious security issues in the network The intrusion detection system IDS is one of the sophisticated defensive systems used to detect the malicious activities happening in the network services across the world Hence more advanced IDS are been developed in past few years To improve the performance of the IDS the system has to be trained effectively to increase the efficiency and decrease the false alarm rate To train the system the attributes selection plays the major role This paper evaluates and compares the performance of the intrusion detection systems for different feature reduction techniques in high speed network

A New Deep Learning Approach for Anomaly Base IDS using Memetic Classifier

A model of an intrusion-detection system capable of detecting attack in computer networks is described. The model is based on deep learning approach to learn best features of network connections and Memetic algorithm as final classifier for detection of abnormal traffic.One of the problems in intrusion detection systems is large scale of features. Which makes typical methods data mining method were ineffective in this area. Deep learning algorithms succeed in image and video mining which has high dimensionality of features. It seems to use them to solve the large scale of features problem of intrusion detection systems is possible. The model is offered in this paper which tries to use deep learning for detecting best features.An evaluation algorithm is used for produce final classifier that work well in multi density environments.We use NSL-KDD and Kdd99 dataset to evaluate our model, our findings showed 98.11 detection rate. NSL-KDD estimation shows the proposed model has succeeded to classify 92.72% R2L attack group

Machine Learning Models for Network Intrusion Detection and Authentication of Smart Phone Users

A thesis presented to the faculty of the Elmer R. Smith College of Business and Technology at Morehead State University in partial fulfillment of the requirements for the Degree of Master of Science by S. Sareh Ahmadi on November 18, 2019

Enhanced Prediction of Network Attacks Using Incomplete Data

For years, intrusion detection has been considered a key component of many organizations’ network defense capabilities. Although a number of approaches to intrusion detection have been tried, few have been capable of providing security personnel responsible for the protection of a network with sufficient information to make adjustments and respond to attacks in real-time. Because intrusion detection systems rarely have complete information, false negatives and false positives are extremely common, and thus valuable resources are wasted responding to irrelevant events. In order to provide better actionable information for security personnel, a mechanism for quantifying the confidence level in predictions is needed. This work presents an approach which seeks to combine a primary prediction model with a novel secondary confidence level model which provides a measurement of the confidence in a given attack prediction being made. The ability to accurately identify an attack and quantify the confidence level in the prediction could serve as the basis for a new generation of intrusion detection devices, devices that provide earlier and better alerts for administrators and allow more proactive response to events as they are occurring

Detector Design Considerations in High-Dimensional Artificial Immune Systems

This research lays the groundwork for a network intrusion detection system that can operate with only knowledge of normal network traffic, using a process known as anomaly detection. Real-valued negative selection (RNS) is a specific anomaly detection algorithm that can be used to perform two-class classification when only one class is available for training. Researchers have shown fundamental problems with the most common detector shape, hyperspheres, in high-dimensional space. The research contained herein shows that the second most common detector type, hypercubes, can also cause problems due to biasing certain features in high dimensions. To address these problems, a new detector shape, the hypersteinmetz solid, is proposed, the goal of which is to provide a tradeoff between the problems plaguing hyperspheres and hypercubes. In order to investigate the potential benefits of the hypersteinmetz solid, an effective RNS detector size range is determined. Then, the relationship between content coverage of a dataset and classification accuracy is investigated. Subsequently, this research shows the tradeoffs that take place in high-dimensional data when hypersteinmetzes are chosen over hyperspheres or hypercubes. The experimental results show that detector shape is the dominant factor toward classification accuracy in high-dimensional RNS

A structured approach to malware detection and analysis in digital forensics investigation

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirement for the degree of PhDWithin the World Wide Web (WWW), malware is considered one of the most serious threats to system security with complex system issues caused by malware and spam. Networks and systems can be accessed and compromised by various types of malware, such as viruses, worms, Trojans, botnet and rootkits, which compromise systems through coordinated attacks. Malware often uses anti-forensic techniques to avoid detection and investigation. Moreover, the results of investigating such attacks are often ineffective and can create barriers for obtaining clear evidence due to the lack of sufficient tools and the immaturity of forensics methodology. This research addressed various complexities faced by investigators in the detection and analysis of malware. In this thesis, the author identified the need for a new approach towards malware detection that focuses on a robust framework, and proposed a solution based on an extensive literature review and market research analysis. The literature review focussed on the different trials and techniques in malware detection to identify the parameters for developing a solution design, while market research was carried out to understand the precise nature of the current problem. The author termed the new approaches and development of the new framework the triple-tier centralised online real-time environment (tri-CORE) malware analysis (TCMA). The tiers come from three distinctive phases of detection and analysis where the entire research pattern is divided into three different domains. The tiers are the malware acquisition function, detection and analysis, and the database operational function. This framework design will contribute to the field of computer forensics by making the investigative process more effective and efficient. By integrating a hybrid method for malware detection, associated limitations with both static and dynamic methods are eliminated. This aids forensics experts with carrying out quick, investigatory processes to detect the behaviour of the malware and its related elements. The proposed framework will help to ensure system confidentiality, integrity, availability and accountability. The current research also focussed on a prototype (artefact) that was developed in favour of a different approach in digital forensics and malware detection methods. As such, a new Toolkit was designed and implemented, which is based on a simple architectural structure and built from open source software that can help investigators develop the skills to critically respond to current cyber incidents and analyses