Feature Selection in UNSW-NB15 and KDDCUP’99 datasets

Machine learning and data mining techniques have been widely used in order to improve network intrusion detection in recent years. These techniques make it possible to automate anomaly detection in network traffics. One of the major problems that researchers are facing is the lack of published data available for research purposes. The KDD’99 dataset was used by researchers for over a decade even though this dataset was suffering from some reported shortcomings and it was criticized by few researchers. In 2009, Tavallaee M. et al. proposed a new dataset (NSL-KDD) extracted from the KDD’99 dataset in order to improve the dataset where it can be used for carrying out research in anomaly detection. The UNSW-NB15 dataset is the latest published dataset which was created in 2015 for research purposes in intrusion detection. This research is analysing the features included in the UNSW-NB15 dataset by employing machine learning techniques and exploring significant features (curse of high dimensionality) by which intrusion detection can be improved in network systems. Therefore, the existing irrelevant and redundant features are omitted from the dataset resulting not only faster training and testing process but also less resource consumption while maintaining high detection rates. A subset of features is proposed in this study and the findings are compared with the previous work in relation to features selection in the KDD’99 dataset

Implementation and Analysis of Combined Machine Learning Method for Intrusion Detection System

As one of the security components in Network Security Monitoring System, Intrusion Detection System (IDS) is implemented by many organizations in their networks to detect and address the impact of network attacks. There are many machine-learning methods that have been widely developed and applied in the IDS. Selection of appropriate methods is necessary to improve the detection accuracy in the application of machine-learning in IDS. In this research we proposed an IDS that we developed based on machine learning approach. We use 28 features subset without content features of  Knowledge Data Discovery (KDD) dataset to build machine learning model. From our analysis and experiment we get 28 features subset of KDD dataset that are most likely to be applied for the IDS in the real network. The machine learning model based on this 28 features subset obtained 99.9% accuracy for both two-class and multiclass classification. From our experiments using the IDS we have developed show good performance in detecting attacks on real networks

A Real-Time Sequential Deep Extreme Learning Machine Cybersecurity Intrusion Detection System

ABSTRACT: In recent years, cybersecurity has attracted significant interest due to the rapid growth of the Internet of Things (IoT) and the widespread development of computer infrastructure and systems. It is thus becoming particularly necessary to identify cyber-attacks or irregularities in the system and develop an efficient intru- sion detection framework that is integral to security. Researchers have worked on developing intrusion detection models that depend on machine learning (ML) methods to address these security problems. An intelligent intrusion detection device powered by data can exploit artificial intelligence (AI), and especially ML, techniques. Accordingly, we propose in this article an intrusion detection model based on a Real-Time Sequential Deep Extreme Learning Machine Cyber- security Intrusion Detection System (RTS-DELM-CSIDS) security model. The proposed model initially determines the rating of security aspects contributing to their significance and then develops a comprehensive intrusion detection frame- work focused on the essential characteristics. Furthermore, we investigated the feasibility of our proposed RTS-DELM-CSIDS framework by performing dataset evaluations and calculating accuracy parameters to validate. The experimental findings demonstrate that the RTS-DELM-CSIDS framework outperforms con- ventional algorithms. Furthermore, the proposed approach has not only research significance but also practical significance

Makine öğrenmesi teknikleriyle saldırı tespiti: Karşılaştırmalı analiz

İnternet, günlük hayatımızın vazgeçilmez bir parçasıdır. Artan web uygulamaları ve kullanıcı sayısı, veri güvenliği açısından bazı riskleri de beraberinde getirmiştir. Ağ güvenliği için önemli araçlardan biri olan saldırı tespit sistemleri, güvenli iç ağlara yapılan saldırıları ve beklenmeyen erişim taleplerini tespit etmede başarılı bir şekilde kullanılmaktadır. Günümüzde, pek çok araştırmacı, daha etkin saldırı tespit sistemi gerçekleştirilmesi amacıyla çalışma yapmaktadır. Bu amaçla literatürde farklı makine öğrenme teknikleri ile gerçekleştirilmiş pek çok saldırı tespit sistemi vardır. Yapılan bu çalışmada, saldırı tespit sistemlerinde sıklıkla kullanılan makine öğrenme teknikleri araştırılmış, kullandıkları sınıflandırıcılar, veri setleri ve elde edilen başarılar değerlendirilmiştir. Bu amaçla 2007-2013 yılları arasında SCI, SCI Expanded ve EBSCO indekslerince taranan ulusal ve uluslararası dergilerde yayınlanmış 65 makale incelenmiş, sonuçlar, karşılaştırılmalı bir şekilde sunulmuştur. Böylece, gelecekte yapılacak makine öğrenme teknikleri ile saldırı tespiti çalışmalarına bir bakış açısı kazandırılması amaçlanmıştır

CHID : conditional hybrid intrusion detection system for reducing false positives and resource consumption on malicous datasets

Inspecting packets to detect intrusions faces challenges when coping with a high volume of network traffic. Packet-based detection processes every payload on the wire, which degrades the performance of network intrusion detection system (NIDS). This issue requires an introduction of a flow-based NIDS that reduces the amount of data to be processed by examining aggregated information of related packets. However, flow-based detection still suffers from the generation of the false positive alerts due to incomplete data input. This study proposed a Conditional Hybrid Intrusion Detection (CHID) by combining the flow-based with packet-based detection. In addition, it is also aimed to improve the resource consumption of the packet-based detection approach. CHID applied attribute wrapper features evaluation algorithms that marked malicious flows for further analysis by the packet-based detection. Input Framework approach was employed for triggering packet flows between the packetbased and flow-based detections. A controlled testbed experiment was conducted to evaluate the performance of detection mechanism’s CHID using datasets obtained from on different traffic rates. The result of the evaluation showed that CHID gains a significant performance improvement in terms of resource consumption and packet drop rate, compared to the default packet-based detection implementation. At a 200 Mbps, CHID in IRC-bot scenario, can reduce 50.6% of memory usage and decreases 18.1% of the CPU utilization without packets drop. CHID approach can mitigate the false positive rate of flow-based detection and reduce the resource consumption of packet-based detection while preserving detection accuracy. CHID approach can be considered as generic system to be applied for monitoring of intrusion detection systems

Uma Revisão Sobre as Publicações de Sistemas de Detecção de Intrusão

O crescente registro de incidentes de segurança em redes de computadores tem motivado o desenvolvimento de estudos em detecção de intrusão, as principais técnicas de identificação de uma intrusão são baseadas em anomalias e assinaturas. Atualmente, a comunidade acadêmica explora preferencialmente pesquisas em redes baseadas em anomalias, entretanto, não existe um modelo comum de desenvolvimento destas propostas de modo que muitos autores descrevem, implementam e validam seus sistemas do modo heterogêneo. Neste artigo foi realizado uma pesquisa que investigou a produção científica de 112 publicações relacionadas a sistemas de detecção de intrusão. Alguns dos critérios utilizados para avaliação destes artigos foram fator de impacto, características de detecção utilizadas e a base de dados implementado. Os resultados obtidos demonstram que ocorreu um aumento da compreensão deste tema, entretanto futuros estudos serão necessários para explorar a validade dos novos métodos de avaliação em detecção de intrusão.

Machine Learning-Enabled IoT Security: Open Issues and Challenges Under Advanced Persistent Threats

Despite its technological benefits, Internet of Things (IoT) has cyber weaknesses due to the vulnerabilities in the wireless medium. Machine learning (ML)-based methods are widely used against cyber threats in IoT networks with promising performance. Advanced persistent threat (APT) is prominent for cybercriminals to compromise networks, and it is crucial to long-term and harmful characteristics. However, it is difficult to apply ML-based approaches to identify APT attacks to obtain a promising detection performance due to an extremely small percentage among normal traffic. There are limited surveys to fully investigate APT attacks in IoT networks due to the lack of public datasets with all types of APT attacks. It is worth to bridge the state-of-the-art in network attack detection with APT attack detection in a comprehensive review article. This survey article reviews the security challenges in IoT networks and presents the well-known attacks, APT attacks, and threat models in IoT systems. Meanwhile, signature-based, anomaly-based, and hybrid intrusion detection systems are summarized for IoT networks. The article highlights statistical insights regarding frequently applied ML-based methods against network intrusion alongside the number of attacks types detected. Finally, open issues and challenges for common network intrusion and APT attacks are presented for future research.Comment: ACM Computing Surveys, 2022, 35 pages, 10 Figures, 8 Table

Anomaly detection for IoT networks using machine learning

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe Internet of Things (IoT) is considered one of the trending technologies today. IoT affects various industries, including logistics tracking, healthcare, automotive and smart cities. A rising number of cyber-attacks and breaches are rapidly targeting networks equipped with IoT devices. This thesis aims to improve security in IoT networks by enhancing anomaly detection using machine learning. This thesis identified the challenges and gaps related to securing the Internet of Things networks. The challenges are network size, the number of devices, the human factor, and the complexity of IoT networks. The gaps identified include the lack of research on signature-based intrusion detection systems used for anomaly detection, in addition to the lack of modelling input parameters required for anomaly detection in IoT networks. Furthermore, there is a lack of comparison of the performance of machine learning algorithms on standard and real IoT datasets. This thesis creates a dataset to test the anomaly binary classification performance of the Neural Networks, Gaussian Naive Bayes, Support Vector Machine, and Decision Trees machine learning algorithms and compares their results with the KDDCUP99 dataset. The results show that Support Vector Machine and Gaussian Naive Bayes perform lower than the other models on the created IoT dataset. This thesis reduces the number of features required by machine learning algorithms for anomaly detection in the IoT networks to five features only, which resulted in reduced execution time by an average of 58%. This thesis tests CNNwGFC, which is an enhanced Convolutional Neural Network model, in detecting and classifying anomalies in IoT networks. This model achieves an increase of 15.34% in the accuracy for IoT anomaly classification in the UNSW-NB15 compared to the classic Convolutional Neural Network. The CNNwGFC multi-classification accuracy (96.24%) is higher by 7.16 than the highest from the literature