Transferability Evaluation in Wi-Fi Intrusion Detection Systems Through Machine Learning and Deep Learning Approaches

Intrusion Detection System (IDS) plays a pivotal role in safeguarding network security. The efficacy of these systems is rigorously assessed through established metrics including precision, recall, F1 score, and AUC score. When subjected to rigorous testing on well-known datasets like AWID and AWID3, individual IDS models consistently deliver exceptional performances, boasting F1 scores ranging from 0.98 to 1 and AUC scores spanning 0.97 to 0.99. However, the true challenge surfaces when the objective is to extend the transferability of these high-performing models to entirely novel, unseen datasets. This endeavor unravels a diverse performance landscape, demonstrating that the outstanding performance observed on a particular dataset doesn’t guarantee the transferability of features across dissimilar datasets nestled within different network environments. In order to evaluate the feature transferability, we turn to AWID and AWID3 datasets as the main distinction between AWID (potentially referring to AWID2) and AWID3 lies in their specific focuses and contexts within the field of Wi-Fi intrusion detection. Although both datasets are centered on the general goal of detecting Wi-Fi intrusions, AWID3 has been carefully designed to meet the specific needs of corporate Wi-Fi applications. A comprehensive evaluation involving Multilayer Perceptron(MLP), and Convolutional Neural Networks (CNN) models has been executed, uncovering that CNN conspicuously outshines the MLP model

A comparative study of deep learning-based network intrusion detection system with explainable artificial intelligence

In the rapidly evolving landscape of cybersecurity, robust network intrusion detection systems (NIDS) are crucial to countering increasingly sophisticated cyber threats, including zero-day attacks. Deep learning approaches in NIDS offer promising improvements in intrusion detection rates and reduction of false positives. However, the inherent opacity of deep learning models presents significant challenges, hindering the understanding and trust in their decision-making processes. This study explores the efficacy of explainable artificial intelligence (XAI) techniques, specifically Shapley additive explanations (SHAP) and local interpretable model-agnostic explanations (LIME), in enhancing the transparency and trustworthiness of NIDS systems. With the implementation of TabNet architecture on the AWID3 dataset, it is able to achieve a remarkable accuracy of 99.99%. Despite this high performance, concerns regarding the interpretability of the TabNet model's decisions persist. By employing SHAP and LIME, this study aims to elucidate the intricacies of model interpretability, focusing on both global and local aspects of the TabNet model's decision-making processes. Ultimately, this study underscores the pivotal role of XAI in improving understanding and fostering trust in deep learning -based NIDS systems. The robustness of the model is also being tested by adding the signal-to-noise ratio (SNR) to the datasets

An Investigation of Feature Reduction, Transferability, and Generalization in AWID Datasets for Secure Wi-Fi Networks

The widespread use of wireless networks to transfer an enormous amount of sensitive information has caused a plethora of vulnerabilities and privacy issues. The management frames, particularly authentication and association frames, are vulnerable to cyberattacks and it is a significant concern. Existing research in Wi-Fi attack detection focused on obtaining high detection accuracy while neglecting modern traffic and attack scenarios such as key reinstallation or unauthorized decryption attacks. This study proposed a novel approach using the AWID 3 dataset for cyberattack detection. The retained features were analyzed to assess their transferability, creating a lightweight and cost-effective model. A decision tree with a recursive feature elimination method was implemented for the extraction of the reduced features subset, and an additional feature wlan_radio.signal_dbm was used in combination with the extracted feature subset. Several deep learning and machine learning models were implemented, where DT and CNN achieved promising classification results. Further, feature transferability and generalizability were evaluated, and their detection performance was analyzed across different network versions where CNN outperformed other classification models. The practical implications of this research are crucial for the secure automation of wireless intrusion detection frameworks and tools in personal and enterprise paradigms

Enhanced Anomaly Detection in Wireless 5G Networks With Hybrid Learning Technique Using AWID3 Dataset

In recent years, the expansion of the Internet of Things and 5G networks has significantly increased wireless traffic, heightening the risk of cyberattacks. Intrusion detection systems have become essential for safeguarding wireless networks by providing real-time threat detection and response. This study presents a comprehensive review and implementation of machine learning-based techniques for detecting various types of wireless attacks, with a focus on improving detection accuracy through ensemble learning. The AWID3 dataset, based on the IEEE 802.11 standard, was used for experimentation. The study was conducted in multiple phases: (1) evaluating six machine learning algorithms (random forest, J48, naïve Bayes, logistic regression, decision tree, and deep neural networks) using three feature selection methods (information gain, gain ratio, and chi-squared); (2) developing a hybrid ensemble model by integrating the strengths of deep neural network, random forest, XGBoost, and LightGBM, with logistic regression as a meta-classifier; and (3) validating performance using key metrics: accuracy, precision, recall, and F1-score. The proposed hybrid model achieved a peak accuracy of 99.75%, outperforming benchmark models in the literature. These results demonstrate the superior performance and robustness of the proposed hybrid approach. By addressing multiple network layers and leveraging ensemble learning, this research highlights the critical role of hybrid models in achieving reliable and accurate intrusion detection for wireless environments

An Investigation of Feature Reduction, Transferability, and Generalization in AWID Datasets for Secure Wi-Fi Networks

The widespread use of wireless networks to transfer an enormous amount of sensitive information has caused a plethora of vulnerabilities and privacy issues. The management frames, particularly authentication and association frames, are vulnerable to cyberattacks and it is a significant concern. Existing research in Wi-Fi attack detection focused on obtaining high detection accuracy while neglecting modern traffic and attack scenarios such as key reinstallation or unauthorized decryption attacks. This study proposed a novel approach using the AWID 3 dataset for cyberattack detection. The retained features were analyzed to assess their transferability, creating a lightweight and cost-effective model. A decision tree with a recursive feature elimination method was implemented for the extraction of the reduced features subset, and an additional feature wlan_radio.signal_dbm was used in combination with the extracted feature subset. Several deep learning and machine learning models were implemented, where DT and CNN achieved promising classification results. Further, feature transferability and generalizability were evaluated, and their detection performance was analyzed across different network versions where CNN outperformed other classification models. The practical implications of this research are crucial for the secure automation of wireless intrusion detection frameworks and tools in personal and enterprise paradigms

Rule-based with machine learning IDS for DDoS attack detection in cyber-physical production systems (CPPS)

Recent advancements in communication technology have transformed the way the industrial system works. This digitalization has improved the way of communication between different actors involved in cyber physical production systems (CPPS), such as users, suppliers, and manufacturers, thus making the whole process transparent. The utilization of emerging new technologies in CPPS can cause vulnerable spots that can be exploited by attackers to launch sophisticated distributed denial of service (DDoS) attacks, hence threatening the availability of the production systems. Existing machine learning based intrusion detection systems (IDS) often rely on unrealistic datasets for training and validation, thus missing the crucial testing phase with real-time scenarios. The results generated by the ML models are based on predictions at each flow level and cannot provide summarized information about malicious entities. To address this limitation, this study proposed an efficient IDS system that uses both rule-based detection and ML-based approaches to detect DDoS attacks damaging the infrastructure of CPPS. For training and validation of the system, we use real-time network traffic extracted from a real industrial scenario, referred to as Farm-to-Fork (F2F) supply chain system. Both, attacks and normal traffic were captured, and bidirectional features were extracted through CIC-FLOWMETER. We make use of 8 ML supervised and unsupervised approaches to detect the malicious flows; and then a rule-based detection mechanism is used to calculate the frequency of the malicious flows and to assign different severity levels based on the computed frequency. The overall results show that supervised models outperform unsupervised approaches and achieve an accuracy 99.97% and TPR 99.96%. Overall, the weighted accuracy when tested and deployed in a real-time scenario is around 98.71%. The results prove that the system works better when considering real-time scenarios and provides comprehensive information about the detected results that can be used to take different mitigation actions.This work was supported in part by European Union’s Horizon Europe (PHOENi2X) under Grant 101070586, in part by the Spanish Ministry of Science and Innovation funded by MCIN/AEI/10.13039/501100011033 under Grant PID2021-124463OB-I00, in part by ERDF a way of making Europe, and in part by the Catalan Government under Contract 2021 SGR 00326.Peer ReviewedPostprint (published version

Adversarial attack detection framework based on optimized weighted conditional stepwise adversarial network

Artificial Intelligence (AI)-based IDS systems are susceptible to adversarial attacks and face challenges such as complex evaluation methods, elevated false positive rates, absence of effective validation, and time-intensive processes. This study proposes a WCSAN-PSO framework to detect adversarial attacks in IDS based on a weighted conditional stepwise adversarial network (WCSAN) with a particle swarm optimization (PSO) algorithm and SVC (support vector classifier) for classification. The Principal component analysis (PCA) and the least absolute shrinkage and selection operator (LASSO) are used for feature selection and extraction. The PSO algorithm optimizes the parameters of the generator and discriminator in WCSAN to improve the adversarial training of IDS. The study presented three distinct scenarios with quantitative evaluation, and the proposed framework is evaluated with adversarial training in balanced and imbalanced data. Compared with existing studies, the proposed framework accomplished an accuracy of 99.36% in normal and 98.55% in malicious traffic in adversarial attacks. This study presents a comprehensive overview for researchers interested in adversarial attacks and their significance in computer security.publishedVersio

CheckShake: Passively Detecting Anomaly in Wi-Fi Security Handshake using Gradient Boosting based Ensemble Learning

Recently, a number of attacks have been demonstrated (like key reinstallation attack, called KRACK) on WPA2 protocol suite in Wi-Fi WLAN. As the firmware of the WLAN devices in the context of IoT, industrial systems, and medical devices is often not patched, detecting and preventing such attacks is challenging. In this paper, we design and implement a system, called CheckShake, to passively detect anomalies in the handshake of Wi-Fi security protocols, in particular WPA2, between a client and an access point using COTS radios. Our proposed system works without decrypting any traffic. It passively monitors multiple wireless channels in parallel in the neighborhood and uses a state machine model to characterize and detect the attacks. In particular, we develop a state machine model for grouping Wi-Fi handshake packets and then perform deep packet inspection to identify the symptoms of the anomaly in specific stages of a handshake session. Our implementation of CheckShake does not require any modification to the firmware of the client or the access point or the COTS devices, it only requires to be physically placed within the range of the access point and its clients. We use both the publicly available dataset and our own data set for performance analysis of CheckShake. Using gradient boosting-based supervised machine learning models, we show that an accuracy around 93.39% and a false positive rate of 5.08% can be achieved using CheckShak

Advancing network security: a comparative research of machine learning techniques for intrusion detection

In the current digital era, the advancement of network-based technologies has brought a surge in security vulnerabilities, necessitating complex and dynamic defense mechanisms. This paper explores the integration of machine learning techniques within intrusion detection systems (IDS) to tackle the intricacies of modern network threats. A detailed comparative analysis of various algorithms, including k-nearest neighbors (KNN), logistic regression, and perceptron neural networks, is conducted to evaluate their efficiency in detecting and classifying different types of network intrusions such as denial of service (DoS), probe, user to root (U2R), and remote to local (R2L). Utilizing the national software laboratory knowledge discovery and data mining (NSL-KDD) dataset, a standard in the field, the study examines the algorithms’ ability to identify complex patterns and anomalies indicative of security breaches. Principal component analysis is utilized to streamline the dataset into 20 principal components for data processing efficiency. Results indicate that the neural network model is particularly effective, demonstrating exceptional performance metrics across accuracy, precision, and recall in both training and testing phases, affirming its reliability and utility in IDS. The potential for hybrid models combining different machine learning (ML) strategies is also discussed, highlighting a path towards more robust and adaptable IDS solutions

A Review of Existence Intrusions Detection-Based Machine Learning Datasets of Future Generation Networks

"Innovative technologies of future generation networks such as Cyber-Physical System (CPS), Mobile Ad Hoc Network (MANET), Vehicular Ad-Hoc Network (VANET), Internet of Things (IOT), and Wireless network commonly known as Wi-Fi have emerged, which require a distinguished understanding of the main challenges and constraints that face the design and implementation of an Intrusion Detection Systems (IDS) for such type of networks. Moreover, a dramatic increase in the rate of cyber-attacks has increased, and new cases of intrusions, bugs, novel attacking tactics, and vulnerabilities are evolving daily. Intrusion Detection Systems (IDS) are one of the solutions against these attacks. Thus, IDS needs to improve its performance in terms of its ability to detect new attacks and respond to threats. Getting suitable datasets for evaluating various research designs in IDS design domains is a significant challenge"“. The machine learning (ML) design approach can quickly identify trends and patterns of intrusions, bugs, tactics, and cyber vulnerabilities with minimum human intervention. This paper reviews datasets for the research community. Furthermore, it explores the challenges of Dataset for intrusion detection based on Machine learning. It glances through a period of 6 years of intrusion detection datasets, explores what is currently applicable, outlines criteria for selecting the best Dataset, and explores future directions for creating relevant datasets