Payload-based anomaly detection in HTTP traffic

University of Technology, Sydney. Faculty of Engineering and Information Technology.Internet provides quality and convenience to human life but at the same time it provides a platform for network hackers and criminals. Intrusion Detection Systems (IDSs) have been proven to be powerful methods for detecting anomalies in the network. Traditional IDSs based on signatures are unable to detect new (zero days) attacks. Anomaly-based systems are alternative to signature based systems. However, present anomaly detection systems suffer from three major setbacks: (a) Large number of false alarms, (b) Very high volume of network traffic due to high data rates (Gbps), and (c) Inefficiency in operation. In this thesis, we address above issues and develop efficient intrusion detection frameworks and models which can be used in detecting a wide variety of attacks including web-based attacks. Our proposed methods are designed to have very few false alarms. We also address Intrusion Detection as a Pattern Recognition problem and discuss all aspects that are important in realizing an anomaly-based IDS. We present three payload-based anomaly detectors, including Geometrical Structure Anomaly Detection (GSAD), Two-Tier Intrusion Detection system using Linear Discriminant Analysis (LDA), and Real-time Payload-based Intrusion Detection System (RePIDS), for intrusion detection. These detectors perform deep-packet analysis and examine payload content using n-gram text categorization and Mahalanobis Distance Map (MDM) techniques. An MDM extracts hidden correlations between the features within each payload and among packet payloads. GSAD generates model of normal network payload as geometrical structure using MDMs in a fully automatic and unsupervised manner. We have implemented the GSAD model in HTTP environment for web-based applications. For efficient operation of IDSs, the detection speed is a key point. Current IDSs examine a large number of data features to detect intrusions and misuse patterns. Hence, for quickly and accurately identifying anomalies of Internet traffic, feature reduction becomes mandatory. We have proposed two models to address this issue, namely two-tier intrusion detection model and RePIDS. Two-tier intrusion detection model uses Linear Discriminant Analysis approach for feature reduction and optimal feature selection. It uses MDM technique to create a model of normal network payload using an extracted feature set. RePIDS uses a 3-tier Iterative Feature Selection Engine (IFSEng) to reduce dimensionality of the raw dataset using Principal Component Analysis (PCA) technique. IFSEng extracts the most significant features from the original feature set and uses mathematical and graphical methods for optimal feature subset selection. Like two-tier intrusion detection model, RePIDS then uses MDM technique to generate a model of normal network payload using extracted features. We test the proposed IDSs on two publicly available datasets of attacks and normal traffic. Experimental results confirm the effectiveness and validation of our proposed solutions in terms of detection rate, false alarm rate and computational complexity

Network intrusion detection based on LDA for payload feature selection

Anomaly Intrusion Detection System (IDS) is a statistical based network IDS which can detect attack variants and novel attacks without a priori knowledge. Current anomaly IDSs are inefficient for real-time detection because of their complex computation. This paper proposes a novel approach to reduce the heavy computational cost of an anomaly IDS. Linear Discriminant Analysis (LDA) and difference distance map are used for selection of significant features. This approach is able to transform high-dimensional feature vectors into a low-dimensional domain. The similarity between new incoming packets and a normal profile is determined using Euclidean distance on the simple, low-dimensional feature domain. The final decision will be made according to a pre-calculated threshold to differentiate normal and abnormal network packets. The proposed approach is evaluated using DARPA 1999 IDS dataset. ©2010 IEEE

A Novel Hybrid Authentication Model for Geo Location Oriented Routing in Dynamic Wireless Mesh Networks

Authentication is an essential part of any network and plays a pivotal role in ensuring the security of a network by preventing unauthorised devices/users access to the network. As dynamic wireless mesh networks are evolving and being accepted in various fields, there is a strong need to improve the security of the network. It’s features like self-organizing and self-healing make it great but get undermined when rigid authentication schemes are used. We propose a hybrid authentication scheme for such dynamic mesh networks under three specified scenarios; full authentication, quick authentication and new node authentication. The proposed schemes are applied on our previous works on dynamic mesh routing protocol, Geo location Oriented Routing Protocol (GLOR Simulation results show our proposed scheme is efficient in terms of resource utilization as well as defending against security threats

Secure-GLOR: An adaptive secure routing protocol for dynamic wireless mesh networks

© 2017 IEEE. With the dawn of a new era, digital security has become one of the most essential part of any network. Be it a physical network, virtual network or social network, the demand for secure data transmission is ever increasing. Wireless mesh networks also stand the same test of security as the legacy networks. This paper presents a secure version of the Geo-Location Oriented Routing (GLOR) protocol for wireless mesh networks, incorporating a multilevel security framework. It implements authentication using the new features of the network model and enables encryption throughout the network to provide high levels of security

Multivariate correlation analysis technique based on Euclidean distance map for network traffic characterization

The quality of feature has significant impact on the performance of detection techniques used for Denial-of-Service (DoS) attack. The features that fail to provide accurate characterization for network traffic records make the techniques suffer from low accuracy in detection. Although researches have been conducted and attempted to overcome this problem, there are some constraints in these works. In this paper, we propose a technique based on Euclidean Distance Map (EDM) for optimal feature extraction. The proposed technique runs analysis on original feature space (first-order statistics) and extracts the multivariate correlations between the first-order statistics. The extracted multivariate correlations, namely second-order statistics, preserve significant discriminative information for accurate characterizations of network traffic records, and these multivariate correlations can be the high-quality potential features for DoS attack detection. The effectiveness of the proposed technique is evaluated using KDD CUP 99 dataset and experimental analysis shows encouraging results. © 2011 Springer-Verlag

Denial-of-service attack detection based on multivariate correlation analysis

The reliability and availability of network services are being threatened by the growing number of Denial-of-Service (DoS) attacks. Effective mechanisms for DoS attack detection are demanded. Therefore, we propose a multivariate correlation analysis approach to investigate and extract second-order statistics from the observed network traffic records. These second-order statistics extracted by the proposed analysis approach can provide important correlative information hiding among the features. By making use of this hidden information, the detection accuracy can be significantly enhanced. The effectiveness of the proposed multivariate correlation analysis approach is evaluated on the KDD CUP 99 dataset. The evaluation shows encouraging results with average 99.96% detection rate and 2.08% false positive rate. Comparisons also show that our multivariate correlation analysis based detection approach outperforms some other current researches in detecting DoS attacks. © 2011 Springer-Verlag

Intrusion detection using geometrical structure

We propose a statistical model, namely Geometrical Structure Anomaly Detection (GSAD) to detect intrusion using the packet payload in the network. GSAD takes into account the correlations among the packet payload features arranged in a geometrical structure. The representation is based on statistical analysis of Mahalanobis distances among payload features, which calculate the similarity of new data against precomputed profile. It calculates weight factor to determine anomaly in the payload. In the 1999 DARPA intrusion detection evaluation data set, we conduct several tests for limited attacks on port 80 and port 25. Our approach establishes and identifies the correlation among packet payloads in a network. © 2009 IEEE

Pattern Recognition Approach for Anomaly Detection of Web-based Attacks

The universal use of the Internet has made it more difficult to achieve high security. Attackers target web applications instead of Telnet ports. Cyber-attacks and breaches of information security are increasing in frequency. The goal of Intrusion Detection Systems (IDSs) is to monitor network traffic and detect web-based attacks. Common IDSs are either signature based or anomaly based. Signature based IDS is unable to detect novel attack (Le., zero-day) or polymorphic attacks, until the signature database is updated. On the other hand, an anomaly-based IDS can detect new attacks and polymorphic attacks. However, anomaly based system has a relatively high number of false positives

A Nonlinear Correlation Measure for Intrusion Detection

The popularity of the Internet supplies attackers with a new means to violate any organizations and individuals. This raises the concerns of the Internet users and research community. One of the effective solutions of addressing this issue is Intrusion Detection System (IDS), which is defined as a type of security tools used to detect any malicious behaviors on computer networks. However, IDSs are commonly prone to high false positive rates. In order to solve this technical challenge, this paper proposes an effective Nonlinear Correlation Coefficient (NCC) based measure, which can accurately extract both linear and nonlinear correlations between network traffic records, for intrusion detection. Then, we demonstrate the effectiveness of our proposed NCC-based measure in extracting correlations by comparing against the Pearsonâs Correlation Coefficient (PCC) based measure. The demonstration is conducted on KDD Cup 99 data set, and the experimental results show that our proposed NCC-based measure not only helps reduce false alarm rate, but also helps distinguish normal and abnormal behaviors efficiently

An intrusion detection system based on polynomial feature correlation analysis

© 2017 IEEE. This paper proposes an anomaly-based Intrusion Detection System (IDS), which flags anomalous network traffic with a distance-based classifier. A polynomial approach was designed and applied in this work to extract hidden correlations from traffic related statistics in order to provide distinguishing features for detection. The proposed IDS was evaluated using the well-known KDD Cup 99 data set. Evaluation results show that the proposed system achieved better detection rates on KDD Cup 99 data set in comparison with another two state-of-the-art detection schemes. Moreover, the computational complexity of the system has been analysed in this paper and shows similar to the two state-of-the-art schemes