Statistical analysis driven optimized deep learning system for intrusion detection

Attackers have developed ever more sophisticated and intelligent ways to hack information and communication technology systems. The extent of damage an individual hacker can carry out upon infiltrating a system is well understood. A potentially catastrophic scenario can be envisaged where a nation-state intercepting encrypted financial data gets hacked. Thus, intelligent cybersecurity systems have become inevitably important for improved protection against malicious threats. However, as malware attacks continue to dramatically increase in volume and complexity, it has become ever more challenging for traditional analytic tools to detect and mitigate threat. Furthermore, a huge amount of data produced by large networks has made the recognition task even more complicated and challenging. In this work, we propose an innovative statistical analysis driven optimized deep learning system for intrusion detection. The proposed intrusion detection system (IDS) extracts optimized and more correlated features using big data visualization and statistical analysis methods (human-in-the-loop), followed by a deep autoencoder for potential threat detection. Specifically, a pre-processing module eliminates the outliers and converts categorical variables into one-hot-encoded vectors. The feature extraction module discard features with null values and selects the most significant features as input to the deep autoencoder model (trained in a greedy-wise manner). The NSL-KDD dataset from the Canadian Institute for Cybersecurity is used as a benchmark to evaluate the feasibility and effectiveness of the proposed architecture. Simulation results demonstrate the potential of our proposed system and its outperformance as compared to existing state-of-the-art methods and recently published novel approaches. Ongoing work includes further optimization and real-time evaluation of our proposed IDS.Comment: To appear in the 9th International Conference on Brain Inspired Cognitive Systems (BICS 2018

Autoencoder-Based Representation Learning to Predict Anomalies in Computer Networks

With the recent advances in Internet-of-thing devices (IoT), cloud-based services, and diversity in the network data, there has been a growing need for sophisticated anomaly detection algorithms within the network intrusion detection system (NIDS) that can tackle advanced network threats. Advances in Deep and Machine learning (ML) has been garnering considerable interest among researchers since it has the capacity to provide a solution to advanced threats such as the zero-day attack. An Intrusion Detection System (IDS) is the first line of defense against network-based attacks compared to other traditional technologies, such as firewall systems. This report adds to the existing approaches by proposing a novel strategy to incorporate both supervised and unsupervised learning to Intrusion Detection Systems (IDS). Specifically, the study will utilize deep Autoencoder (DAE) as a dimensionality reduction tool and Support Vector Machine (SVM) as a classifier to perform anomaly-based classification. The study diverts from other similar studies by performing a thorough analysis of using deep autoencoders as a valid non-linear dimensionality tool by comparing it against Principal Component Analysis (PCA) and tuning hyperparameters that optimizes for \u27F-1 Micro\u27 score and \u27Balanced Accuracy\u27 since we are dealing with a dataset with imbalanced classes. The study employs robust analysis tools such as Precision-Recall Curves, Average-Precision score, Train-Test Times, t-SNE, Grid Search, and L1/L2 regularization. Our model will be trained and tested on a publicly available datasets KDDTrain+ and KDDTest+

A Deep Learning Approach Combining Auto-encoder with One-class SVM for DDoS Attack Detection in SDNs

Software Defined Networking (SDN) provides us with the capability of collecting network traffic information and managing networks proactively. Therefore, SDN facilitates the promotion of more robust and secure networks. Recently, several Machine Learning (ML)/Deep Learning (DL) intrusion detection approaches have been proposed to secure SDN networks. Currently, most of the proposed ML/DL intrusion detection approaches are based on supervised learning approach that required labelled and well-balanced datasets for training. However, this is time intensive and require significant human expertise to curate these datasets. These approaches cannot deal well with imbalanced and unlabeled datasets. In this paper, we propose a hybrid unsupervised DL approach using the stack autoencoder and One-class Support Vector Machine (SAE-1SVM) for Distributed Denial of Service (DDoS) attack detection. The experimental results show that the proposed algorithm can achieve an average accuracy of 99.35 % with a small set of flow features. The SAE-1SVM shows that it can reduce the processing time significantly while maintaining a high detection rate. In summary, the SAE-1SVM can work well with imbalanced and unlabeled datasets and yield a high detection accuracy

DEMISe: interpretable deep extraction and mutual information selection techniques for IoT intrusion detection

Recent studies have proposed that traditional security technology – involving pattern-matching algorithms that check predefined pattern sets of intrusion signatures – should be replaced with sophisticated adaptive approaches that combine machine learning and behavioural analytics. However, machine learning is performance driven, and the high computational cost is incompatible with the limited computing power, memory capacity and energy resources of portable IoT-enabled devices. The convoluted nature of deep-structured machine learning means that such models also lack transparency and interpretability. The knowledge obtained by interpretable learners is critical in security software design. We therefore propose two novel models featuring a common Deep Extraction and Mutual Information Selection (DEMISe) element which extracts features using a deep-structured stacked autoencoder, prior to feature selection based on the amount of mutual information (MI) shared between each feature and the class label. An entropy-based tree wrapper is used to optimise the feature subsets identified by the DEMISe element, yielding the DEMISe with Tree Evaluation and Regression Detection (DETEReD) model. This affords ‘white box’ insight, and achieves a time to build of 603 seconds, a 99.07% detection rate, and 98.04% model accuracy. When tested against AWID, the best-referenced intrusion detection dataset, the new models achieved a test error comparable to or better than state-of-the-art machine-learning models, with a lower computational cost and higher levels of transparency and interpretability