Accurate shellcode recognition from network traffic data using artificial neural nets

This paper presents an approach to shellcode recognition directly from network traffic data using a multi-layer perceptron with back-propagation learning algorithm. Using raw network data composed of a mixture of shellcode, image files, and DLL-Dynamic Link Library files, our proposed design was able to classify the three types of data with high accuracy and high precision with neither false positives nor false negatives. The proposed method comprises simple and fast pre-processing of raw data of a fixed length for each network data package and yields perfect results with 100\% accuracy for the three data types considered. The research is significant in the context of network security and intrusion detection systems. Work is under way for real time recognition and fine-tuning the differentiation between various shellcodes

Performance Metrics for Network Intrusion Systems

Intrusion systems have been the subject of considerable research during the past 33 years, since the original work of Anderson. Much has been published attempting to improve their performance using advanced data processing techniques including neural nets, statistical pattern recognition and genetic algorithms. Whilst some significant improvements have been achieved they are often the result of assumptions that are difficult to justify and comparing performance between different research groups is difficult. The thesis develops a new approach to defining performance focussed on comparing intrusion systems and technologies. A new taxonomy is proposed in which the type of output and the data scale over which an intrusion system operates is used for classification. The inconsistencies and inadequacies of existing definitions of detection are examined and five new intrusion levels are proposed from analogy with other detection-based technologies. These levels are known as detection, recognition, identification, confirmation and prosecution, each representing an increase in the information output from, and functionality of, the intrusion system. These levels are contrasted over four physical data scales, from application/host through to enterprise networks, introducing and developing the concept of a footprint as a pictorial representation of the scope of an intrusion system. An intrusion is now defined as “an activity that leads to the violation of the security policy of a computer system”. Five different intrusion technologies are illustrated using the footprint with current challenges also shown to stimulate further research. Integrity in the presence of mixed trust data streams at the highest intrusion level is identified as particularly challenging. Two metrics new to intrusion systems are defined to quantify performance and further aid comparison. Sensitivity is introduced to define basic detectability of an attack in terms of a single parameter, rather than the usual four currently in use. Selectivity is used to describe the ability of an intrusion system to discriminate between attack types. These metrics are quantified experimentally for network intrusion using the DARPA 1999 dataset and SNORT. Only nine of the 58 attack types present were detected with sensitivities in excess of 12dB indicating that detection performance of the attack types present in this dataset remains a challenge. The measured selectivity was also poor indicting that only three of the attack types could be confidently distinguished. The highest value of selectivity was 3.52, significantly lower than the theoretical limit of 5.83 for the evaluated system. Options for improving selectivity and sensitivity through additional measurements are examined.Stochastic Systems Lt

Enabling Quantum Cybersecurity Analytics in Botnet Detection: Stable Architecture and Speed-up through Tree Algorithms

For the first time, we enable the execution of hybrid machine learning methods on real quantum computers, with 100 data samples, and also with real-device-based simulations, with 5,000 data samples and thereby outperforming the current state of research of Suryotrisongko and Musashi from the year 2022 who were dealing with 1,000 data samples and not with simulations on quantum real devices but on quantum simulators (i.e. pure software-based emulators) only. Additionally, we beat their reported accuracy of 76.8% by an average accuracy of 89.0%, all of this in a total computation time of 382 seconds only. They did not report the execution time. We gain this significant progress by a two-fold strategy: First, we provide a stabilized quantum architecture that enables us to execute HQML algorithms on real quantum devices. Second, we design a new form of hybrid quantum binary classification algorithms that are based on Hoeffding decision tree algorithms. These algorithms lead to the mentioned speed-up through their batch-wise execution in order to drastically reduce the number of shots needed for the real quantum device compared to standard loop-based optimizers. Their incremental nature serves the purpose of big data online streaming for DGA botnet detection. These two steps allow us to apply hybrid quantum machine learning to the field of cybersecurity analytics on the example of DGA botnet detection and how quantum-enhanced SIEM and, thereby, quantum cybersecurity analytics is made possible. We conduct experiments using the library Qiskit with quantum simulator Aer as well as on three different real quantum devices from MS Azure Quantum, naming IonQ, Rigetti and Quantinuum. It is the first time that these tools have been combined.Comment: 33 pages, 6 figures, 6 table

A model for multi-attack classification to improve intrusion detection performance using deep learning approaches

This proposed model introduces novel deep learning methodologies. The objective here is to create a reliable intrusion detection mechanism to help identify malicious attacks. Deep learning based solution framework is developed consisting of three approaches. The first approach is Long-Short Term Memory Recurrent Neural Network (LSTM-RNN) with seven optimizer functions such as adamax, SGD, adagrad, adam, RMSprop, nadam and adadelta. The model is evaluated on NSL-KDD dataset and classified multi attack classification. The model has outperformed with adamax optimizer in terms of accuracy, detection rate and low false alarm rate. The results of LSTM-RNN with adamax optimizer is compared with existing shallow machine and deep learning models in terms of accuracy, detection rate and low false alarm rate. The multi model methodology consisting of Recurrent Neural Network (RNN), Long-Short Term Memory Recurrent Neural Network (LSTM-RNN), and Deep Neural Network (DNN). The multi models are evaluated on bench mark datasets such as KDD99, NSL-KDD, and UNSWNB15 datasets. The models self-learnt the features and classifies the attack classes as multi-attack classification. The models RNN, and LSTM-RNN provide considerable performance compared to other existing methods on KDD99 and NSL-KDD datase

Reduction of False Positives in Intrusion Detection Based on Extreme Learning Machine with Situation Awareness

Protecting computer networks from intrusions is more important than ever for our privacy, economy, and national security. Seemingly a month does not pass without news of a major data breach involving sensitive personal identity, financial, medical, trade secret, or national security data. Democratic processes can now be potentially compromised through breaches of electronic voting systems. As ever more devices, including medical machines, automobiles, and control systems for critical infrastructure are increasingly networked, human life is also more at risk from cyber-attacks. Research into Intrusion Detection Systems (IDSs) began several decades ago and IDSs are still a mainstay of computer and network protection and continue to evolve. However, detecting previously unseen, or zero-day, threats is still an elusive goal. Many commercial IDS deployments still use misuse detection based on known threat signatures. Systems utilizing anomaly detection have shown great promise to detect previously unseen threats in academic research. But their success has been limited in large part due to the excessive number of false positives that they produce. This research demonstrates that false positives can be better minimized, while maintaining detection accuracy, by combining Extreme Learning Machine (ELM) and Hidden Markov Models (HMM) as classifiers within the context of a situation awareness framework. This research was performed using the University of New South Wales - Network Based 2015 (UNSW-NB15) data set which is more representative of contemporary cyber-attack and normal network traffic than older data sets typically used in IDS research. It is shown that this approach provides better results than either HMM or ELM alone and with a lower False Positive Rate (FPR) than other comparable approaches that also used the UNSW-NB15 data set

Network Intrusion Detection System:A systematic study of Machine Learning and Deep Learning approaches

The rapid advances in the internet and communication fields have resulted in ahuge increase in the network size and the corresponding data. As a result, manynovel attacks are being generated and have posed challenges for network secu-rity to accurately detect intrusions. Furthermore, the presence of the intruderswiththeaimtolaunchvariousattackswithinthenetworkcannotbeignored.Anintrusion detection system (IDS) is one such tool that prevents the network frompossible intrusions by inspecting the network traffic, to ensure its confidential-ity, integrity, and availability. Despite enormous efforts by the researchers, IDSstillfaceschallengesinimprovingdetectionaccuracywhilereducingfalsealarmrates and in detecting novel intrusions. Recently, machine learning (ML) anddeep learning (DL)-based IDS systems are being deployed as potential solutionsto detect intrusions across the network in an efficient manner. This article firstclarifiestheconceptofIDSandthenprovidesthetaxonomybasedonthenotableML and DL techniques adopted in designing network-based IDS (NIDS) sys-tems. A comprehensive review of the recent NIDS-based articles is provided bydiscussing the strengths and limitations of the proposed solutions. Then, recenttrends and advancements of ML and DL-based NIDS are provided in terms ofthe proposed methodology, evaluation metrics, and dataset selection. Using theshortcomings of the proposed methods, we highlighted various research chal-lenges and provided the future scope for the research in improving ML andDL-based NIDS

Exploring Artificial Intelligence (AI) Techniques for Forecasting Network Traffic: Network QoS and Security Perspectives

This thesis identifies the research gaps in the field of network intrusion detection and network QoS prediction, and proposes novel solutions to address these challenges. Our first topic presents a novel network intrusion detection system using a stacking ensemble technique using UNSW-15 and CICIDS-2017 datasets. In contrast to earlier research, our proposed novel network intrusion detection techniques not only determine if the network traffic is benign or normal, but also reveal the type of assault in the flow. Our proposed stacking ensemble model provides a more effective detection capability than the existing works. Our proposed stacking ensemble technique can detect 90.4% and 98.7% cyberattacks with an f1-score of 90.0% and 98.5%, respectively. Our second topic proposes a novel QoS prediction model tested in a live 5G network environment. Compared to the existing work in this domain, our study is the first approach to conduct a large-scale field test in a 5G network to measure and forecast the network QoS metrics. More than 50 days of continuous data have been collected, cleaned, and used for training the deep sequence models to predict the 5G network QoS metrics such as throughput, latency, jitter, and packet loss. Our experiments demonstrate the effectiveness of predicting the QoS metrics using LSTM and LSTM Encoder-Decoder models, providing lower prediction errors of 14.57% and 13.75%, respectively

Looking deeper: Using deep learning to identify internet communications traffic

Recent years have shown an unprecedented reliance on the internet to provide services essential for business, education, and personal use. Due to this reliance, coupled with the exponential growth of the internet traffic being generated, there has never been a greater necessity for effective network management techniques. Network traffic classification is one key component of this network management which aims to identify the types and quantity of traffic flowing through a network. Previous traffic classification techniques are limited by the use of non-standardised port numbers and the encryption of traffic contents. To tackle these challenges, we propose using deep learning techniques for network traffic classification. This paper investigates the viability of using deep learning for traffic classification with a focus on both network management applications and detecting malicious traffic. Our preliminary results thus far show that a highly accurate classifier can be created using the first 50 bytes of a traffic flow.Daniel Smit, Kyle Millar, Clinton Page, Adriel Cheng, Hong-Gunn Chew and Cheng-Chew Li