Anomaly detection for IoT networks using machine learning

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe Internet of Things (IoT) is considered one of the trending technologies today. IoT affects various industries, including logistics tracking, healthcare, automotive and smart cities. A rising number of cyber-attacks and breaches are rapidly targeting networks equipped with IoT devices. This thesis aims to improve security in IoT networks by enhancing anomaly detection using machine learning. This thesis identified the challenges and gaps related to securing the Internet of Things networks. The challenges are network size, the number of devices, the human factor, and the complexity of IoT networks. The gaps identified include the lack of research on signature-based intrusion detection systems used for anomaly detection, in addition to the lack of modelling input parameters required for anomaly detection in IoT networks. Furthermore, there is a lack of comparison of the performance of machine learning algorithms on standard and real IoT datasets. This thesis creates a dataset to test the anomaly binary classification performance of the Neural Networks, Gaussian Naive Bayes, Support Vector Machine, and Decision Trees machine learning algorithms and compares their results with the KDDCUP99 dataset. The results show that Support Vector Machine and Gaussian Naive Bayes perform lower than the other models on the created IoT dataset. This thesis reduces the number of features required by machine learning algorithms for anomaly detection in the IoT networks to five features only, which resulted in reduced execution time by an average of 58%. This thesis tests CNNwGFC, which is an enhanced Convolutional Neural Network model, in detecting and classifying anomalies in IoT networks. This model achieves an increase of 15.34% in the accuracy for IoT anomaly classification in the UNSW-NB15 compared to the classic Convolutional Neural Network. The CNNwGFC multi-classification accuracy (96.24%) is higher by 7.16 than the highest from the literature

Engineering the application of machine learning in an IDS based on IoT traffic flow

Internet of Things (IoT) devices are now widely used, enabling intelligent services that, in association with new communication technologies like the 5G and broadband internet, boost smart-city environments. Despite their limited resources, IoT devices collect and share large amounts of data and are connected to the internet, becoming an attractive target for malicious actors. This work uses machine learning combined with an Intrusion Detection System (IDS) to detect possible attacks. Due to the limitations of IoT devices and low latency services, the IDS must have a specialized architecture. Furthermore, although machine learning-based solutions have high potential, there are still challenges related to training and generalization, which may impose constraints on the architecture. Our proposal is an IDS with a distributed architecture that relies on Fog computing to run specialized modules and use deep neural networks to identify malicious traffic inside IoT data flows. We compare our IoT-Flow IDS with three other architectures. We assess model generalization using test data from different datasets and evaluate their performance in terms of Recall, Precision, and F1-Score. Results confirm the feasibility of flowbased anomaly detection and the importance of network traffic segmentation and specialized models in the AI-based IDS for IoT.info:eu-repo/semantics/publishedVersio

A taxonomy of network threats and the effect of current datasets on intrusion detection systems

As the world moves towards being increasingly dependent on computers and automation, building secure applications, systems and networks are some of the main challenges faced in the current decade. The number of threats that individuals and businesses face is rising exponentially due to the increasing complexity of networks and services of modern networks. To alleviate the impact of these threats, researchers have proposed numerous solutions for anomaly detection; however, current tools often fail to adapt to ever-changing architectures, associated threats and zero-day attacks. This manuscript aims to pinpoint research gaps and shortcomings of current datasets, their impact on building Network Intrusion Detection Systems (NIDS) and the growing number of sophisticated threats. To this end, this manuscript provides researchers with two key pieces of information; a survey of prominent datasets, analyzing their use and impact on the development of the past decade’s Intrusion Detection Systems (IDS) and a taxonomy of network threats and associated tools to carry out these attacks. The manuscript highlights that current IDS research covers only 33.3% of our threat taxonomy. Current datasets demonstrate a clear lack of real-network threats, attack representation and include a large number of deprecated threats, which together limit the detection accuracy of current machine learning IDS approaches. The unique combination of the taxonomy and the analysis of the datasets provided in this manuscript aims to improve the creation of datasets and the collection of real-world data. As a result, this will improve the efficiency of the next generation IDS and reflect network threats more accurately within new datasets

A taxonomy of network threats and the effect of current datasets on intrusion detection systems

As the world moves towards being increasingly dependent on computers and automation, building secure applications, systems and networks are some of the main challenges faced in the current decade. The number of threats that individuals and businesses face is rising exponentially due to the increasing complexity of networks and services of modern networks. To alleviate the impact of these threats, researchers have proposed numerous solutions for anomaly detection; however, current tools often fail to adapt to ever-changing architectures, associated threats and zero-day attacks. This manuscript aims to pinpoint research gaps and shortcomings of current datasets, their impact on building Network Intrusion Detection Systems (NIDS) and the growing number of sophisticated threats. To this end, this manuscript provides researchers with two key pieces of information; a survey of prominent datasets, analyzing their use and impact on the development of the past decade's Intrusion Detection Systems (IDS) and a taxonomy of network threats and associated tools to carry out these attacks. The manuscript highlights that current IDS research covers only 33.3% of our threat taxonomy. Current datasets demonstrate a clear lack of real-network threats, attack representation and include a large number of deprecated threats, which together limit the detection accuracy of current machine learning IDS approaches. The unique combination of the taxonomy and the analysis of the datasets provided in this manuscript aims to improve the creation of datasets and the collection of real-world data. As a result, this will improve the efficiency of the next generation IDS and reflect network threats more accurately within new datasets

Evolving attackers against wireless sensor networks using genetic programming

Recent hardware developments have made it possible for the Internet of Things (IoT) to be built. A wide variety of industry sectors, including manufacturing, utilities, agriculture, transportation, and healthcare are actively seeking to incorporate IoT technologies in their operations. The increased connectivity and data sharing that give IoT systems their advantages also increase their vulnerability to attack. In this study, the authors explore the automated generation of attacks using genetic programming (GP), so that defences can be tested objectively in advance of deployment. In the authors' system, the GP-generated attackers targeted publish-subscribe communications within a wireless sensor networks that was protected by an artificial immune intrusion detection system (IDS) taken from the literature. The GP attackers successfully suppressed more legitimate messages than the hand-coded attack used originally to test the IDS, whilst reducing the likelihood of detection. Based on the results, it was possible to reconfigure the IDS to improve its performance. Whilst the experiments were focussed on establishing a proof-of-principle rather than a turnkey solution, they indicate that GP-generated attackers have the potential to improve the protection of systems with large attack surfaces, in a way that is complementary to traditional testing and certification