Payload-based anomaly detection in HTTP traffic

University of Technology, Sydney. Faculty of Engineering and Information Technology.Internet provides quality and convenience to human life but at the same time it provides a platform for network hackers and criminals. Intrusion Detection Systems (IDSs) have been proven to be powerful methods for detecting anomalies in the network. Traditional IDSs based on signatures are unable to detect new (zero days) attacks. Anomaly-based systems are alternative to signature based systems. However, present anomaly detection systems suffer from three major setbacks: (a) Large number of false alarms, (b) Very high volume of network traffic due to high data rates (Gbps), and (c) Inefficiency in operation. In this thesis, we address above issues and develop efficient intrusion detection frameworks and models which can be used in detecting a wide variety of attacks including web-based attacks. Our proposed methods are designed to have very few false alarms. We also address Intrusion Detection as a Pattern Recognition problem and discuss all aspects that are important in realizing an anomaly-based IDS. We present three payload-based anomaly detectors, including Geometrical Structure Anomaly Detection (GSAD), Two-Tier Intrusion Detection system using Linear Discriminant Analysis (LDA), and Real-time Payload-based Intrusion Detection System (RePIDS), for intrusion detection. These detectors perform deep-packet analysis and examine payload content using n-gram text categorization and Mahalanobis Distance Map (MDM) techniques. An MDM extracts hidden correlations between the features within each payload and among packet payloads. GSAD generates model of normal network payload as geometrical structure using MDMs in a fully automatic and unsupervised manner. We have implemented the GSAD model in HTTP environment for web-based applications. For efficient operation of IDSs, the detection speed is a key point. Current IDSs examine a large number of data features to detect intrusions and misuse patterns. Hence, for quickly and accurately identifying anomalies of Internet traffic, feature reduction becomes mandatory. We have proposed two models to address this issue, namely two-tier intrusion detection model and RePIDS. Two-tier intrusion detection model uses Linear Discriminant Analysis approach for feature reduction and optimal feature selection. It uses MDM technique to create a model of normal network payload using an extracted feature set. RePIDS uses a 3-tier Iterative Feature Selection Engine (IFSEng) to reduce dimensionality of the raw dataset using Principal Component Analysis (PCA) technique. IFSEng extracts the most significant features from the original feature set and uses mathematical and graphical methods for optimal feature subset selection. Like two-tier intrusion detection model, RePIDS then uses MDM technique to generate a model of normal network payload using extracted features. We test the proposed IDSs on two publicly available datasets of attacks and normal traffic. Experimental results confirm the effectiveness and validation of our proposed solutions in terms of detection rate, false alarm rate and computational complexity

CLASSIFICATION OF CYBERSECURITY INCIDENTS IN NIGERIA USING MACHINE LEARNING METHODS

Cybercrime has become more likely as a result of technological advancements and increased use of the internet and computer systems. As a result, there is an urgent need to develop effective methods of dealing with these cyber threats or incidents to identify and combat the associated cybercrimes in Nigerian cyberspace adequately. It is therefore desirable to build models that will enable the Nigeria Computer Emergency Response Team (ngCERT) and law enforcement agencies to gain valuable knowledge of insights from the available data to detect, identify and efficiently classify the most prevalent cyber incidents within Nigeria cyberspace, and predict future threats. This study applied machine learning methods to study and understand cybercrime incidents or threats recorded by ngCERT to build models that will characterize cybercrime incidents in Nigeria and classify cybersecurity incidents by mode of attacks and identify the most prevalent incidents within Nigerian cyberspace. Seven different machine learning methods were used to build the classification and prediction models. The Logistic Regression (LR), Naïve Bayes (NB), Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), K-Nearest Neighbor (KNN), Decision Tree (CART) and Random Forest (RF) Algorithms were used to discover the relationship between the relevant attributes of the datasets then classify the threats into several categories. The RF, CART, and KNN models were shown to be the most effective in classifying our data with accuracy score of 99%  each while others has accuracy scores of 98% for SVM, 89% for NB, 88% for LR, and 88% for LDA. Therefore, the result of our classification will help organizations in Nigeria to be able to understand the threats that could affect their assets

CLASSIFICATION OF CYBERSECURITY INCIDENTS IN NIGERIA USING MACHINE LEARNING METHODS

Cybercrime has become more likely as a result of technological advancements and increased use of the internet and computer systems. As a result, there is an urgent need to develop effective methods of dealing with these cyber threats or incidents to identify and combat the associated cybercrimes in Nigerian cyberspace adequately. It is therefore desirable to build models that will enable the Nigeria Computer Emergency Response Team (ngCERT) and law enforcement agencies to gain valuable knowledge of insights from the available data to detect, identify and efficiently classify the most prevalent cyber incidents within Nigeria cyberspace, and predict future threats. This study applied machine learning methods to study and understand cybercrime incidents or threats recorded by ngCERT to build models that will characterize cybercrime incidents in Nigeria and classify cybersecurity incidents by mode of attacks and identify the most prevalent incidents within Nigerian cyberspace. Seven different machine learning methods were used to build the classification and prediction models. The Logistic Regression (LR), Naïve Bayes (NB), Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), K-Nearest Neighbor (KNN), Decision Tree (CART) and Random Forest (RF) Algorithms were used to discover the relationship between the relevant attributes of the datasets then classify the threats into several categories. The RF, CART, and KNN models were shown to be the most effective in classifying our data with accuracy score of 99%  each while others has accuracy scores of 98% for SVM, 89% for NB, 88% for LR, and 88% for LDA. Therefore, the result of our classification will help organizations in Nigeria to be able to understand the threats that could affect their assets

A two-layer dimension reduction and two-tier classification model for anomaly-based intrusion detection in IoT backbone networks

With increasing reliance on Internet of Things (IoT) devices and services, the capability to detect intrusions and malicious activities within IoT networks is critical for resilience of the network infrastructure. In this paper, we present a novel model for intrusion detection based on two-layer dimension reduction and two-tier classification module, designed to detect malicious activities such as User to Root (U2R) and Remote to Local (R2L) attacks. The proposed model is using component analysis and linear discriminate analysis of dimension reduction module to spate the high dimensional dataset to a lower one with lesser features. We then apply a two-tier classification module utilizing Naïve Bayes and Certainty Factor version of K-Nearest Neighbor to identify suspicious behaviors. The experiment results using NSL-KDD dataset shows that our model outperforms previous models designed to detect U2R and R2L attacks

TSE-IDS: A Two-Stage Classifier Ensemble for Intelligent Anomaly-based Intrusion Detection System

Intrusion detection systems (IDS) play a pivotal role in computer security by discovering and repealing malicious activities in computer networks. Anomaly-based IDS, in particular, rely on classification models trained using historical data to discover such malicious activities. In this paper, an improved IDS based on hybrid feature selection and two-level classifier ensembles is proposed. An hybrid feature selection technique comprising three methods, i.e. particle swarm optimization, ant colony algorithm, and genetic algorithm, is utilized to reduce the feature size of the training datasets (NSL-KDD and UNSW-NB15 are considered in this paper). Features are selected based on the classification performance of a reduced error pruning tree (REPT) classifier. Then, a two-level classifier ensembles based on two meta learners, i.e., rotation forest and bagging, is proposed. On the NSL-KDD dataset, the proposed classifier shows 85.8% accuracy, 86.8% sensitivity, and 88.0% detection rate, which remarkably outperform other classification techniques recently proposed in the literature. Results regarding the UNSW-NB15 dataset also improve the ones achieved by several state of the art techniques. Finally, to verify the results, a two-step statistical significance test is conducted. This is not usually considered by IDS research thus far and, therefore, adds value to the experimental results achieved by the proposed classifier

Detection of denial-of-service attacks based on computer vision techniques

University of Technology, Sydney. Faculty of Engineering and Information Technology.A Denial-of-Service (DoS) attack is an intrusive attempt, which aims to force a designated resource (e.g., network bandwidth, processor time or memory) to be unavailable to its intended users. This attack is launched either by deliberately exploiting system vulnerabilities of a victim (e.g., a host, a router, or an entire network) or by flooding a victim with large volume of useless network traffic. Since 1990s, DoS attacks have emerged as a type of the most severe network intrusive behaviours and have posed serious threats to the infrastructures of computer networks and various network-based services. This thesis aims to provide an intelligent and effective solution for DoS attack detection. Unlike the related works based on machine learning and statistical analysis, this thesis suggests to treat network traffic records as images and to redefine the DoS attack detection problem as a computer vision task. To achieve the aforementioned objectives, this thesis first conducts a detailed literature review on the state of the art in DoS attack detection. Then, it analyses and chooses the most appropriate mechanisms for DoS attack detection. Afterwards, it designs a general system framework for DoS attack detection with respect to the chosen mechanisms. Furthermore, two Multivariate Correlation Analysis (MCA) approaches are proposed based on two techniques, namely Euclidean distance and triangle area. These two proposed MCA approaches provide accurate description for network traffic records and facilitate conversion of network traffic into the respective images. In addition, this thesis proposes a DoS attack detection system, in which the images of network traffic are served as the observed objects and the task of DoS attack detection is reformulated as a computer vision problem, namely image retrieval. This proposed DoS attack detection system applies a widely used dissimilarity measure, namely the Earth Mover’s Distance (EMD), to object classification. The EMD takes cross-bin matching into account and provides a more accurate evaluation on the dissimilarity between distributions than some other well-known dissimilarity measures, such as Minkowski-form distance Lp and X² statistics. The merits of the EMD facilitate the capability of our proposed system with effective detection. Last but not least, our intelligent and effective solutions, including the two proposed MCA approaches and the EMD-based DoS attack detection system, are evaluated using the KDD Cup 99 dataset. The evaluation results illustrate that our proposed MCA approaches provide accurate characterisation for network traffic, and the proposed detection system can detect unknown DoS attacks and outperforms two state-of-the-art approaches

Detecting Abnormal Social Robot Behavior through Emotion Recognition

Sharing characteristics with both the Internet of Things and the Cyber Physical Systems categories, a new type of device has arrived to claim a third category and raise its very own privacy concerns. Social robots are in the market asking consumers to become part of their daily routine and interactions. Ranging in the level and method of communication with the users, all social robots are able to collect, share and analyze a great variety and large volume of personal data.In this thesis, we focus the community’s attention to this emerging area of interest for privacy and security research. We discuss the likely privacy issues, comment on current defense mechanisms that are applicable to this new category of devices, outline new forms of attack that are made possible through social robots, highlight paths that research on consumer perceptions could follow, and propose a system for detecting abnormal social robot behavior based on emotion detection

Comparison study of machine learning classifiers to detect anomalies

In this era of Internet ensuring the confidentiality, authentication and integrity of any resource exchanged over the net is the imperative. Presence of intrusion prevention techniques like strong password, firewalls etc. are not sufficient to monitor such voluminous network traffic as they can be breached easily. Existing signature based detection techniques like antivirus only offers protection against known attacks whose signatures are stored in the database.Thus, the need for real-time detection of aberrations is observed. Existing signature based detection techniques like antivirus only offers protection against known attacks whose signatures are stored in the database. Machine learning classifiers are implemented here to learn how the values of various fields like source bytes, destination bytes etc. in a network packet decides if the packet is compromised or not . Finally the accuracy of their detection is compared to choose the best suited classifier for this purpose. The outcome thus produced may be useful to offer real time detection while exchanging sensitive information such as credit card details

Multivariate correlation analysis technique based on Euclidean distance map for network traffic characterization

The quality of feature has significant impact on the performance of detection techniques used for Denial-of-Service (DoS) attack. The features that fail to provide accurate characterization for network traffic records make the techniques suffer from low accuracy in detection. Although researches have been conducted and attempted to overcome this problem, there are some constraints in these works. In this paper, we propose a technique based on Euclidean Distance Map (EDM) for optimal feature extraction. The proposed technique runs analysis on original feature space (first-order statistics) and extracts the multivariate correlations between the first-order statistics. The extracted multivariate correlations, namely second-order statistics, preserve significant discriminative information for accurate characterizations of network traffic records, and these multivariate correlations can be the high-quality potential features for DoS attack detection. The effectiveness of the proposed technique is evaluated using KDD CUP 99 dataset and experimental analysis shows encouraging results. © 2011 Springer-Verlag