Anomaly detection using network traffic characterization

Thesis (Master)--Izmir Institute of Technology, Computer Engineering, Izmir, 2009Includes bibliographical references (leaves: 63-66)Text in English Abstract: Turkish and Englishix, 80 leavesDetecting suspicious traffic and anomaly sources are a general tendency about approaching the traffic analyzing. Since the necessity of detecting anomalies, different approaches are developed with their software candidates. Either event based or signature based anomaly detection mechanism can be applied to analyze network traffic. Signature based approaches require the detected signatures of the past anomalies though event based approaches propose a more flexible approach that is defining application level abnormal anomalies is possible. Both approach focus on the implementing and defining abnormal traffic. The problem about anomaly is that there is not a common definition of anomaly for all protocols or malicious attacks. In this thesis it is aimed to define the non-malicious traffic and extract it, so that the rest is marked as suspicious traffic for further traffic. To achieve this approach, a method and its software application to identify IP sessions, based on statistical metrics of the packet flows are presented. An adaptive network flow knowledge-base is derived. The knowledge-base is constructed using calculated flows attributes. A method to define known traffic is displayed by using the derived flow attributes. By using the attributes, analyzed flow is categorized as a known application level protocol. It is also explained a mathematical model to analyze the undefined traffic to display network traffic anomalies. The mathematical model is based on principle component analysis which is applied on the origindestination pair flows. By using metric based traffic characterization and principle component analysis it is observed that network traffic can be analyzed and some anomalies can be detected

Distributed Network Anomaly Detection on an Event Processing Framework

Network Intrusion Detection Systems (NIDS) are an integral part of modern data centres to ensure high availability and compliance with Service Level Agreements (SLAs). Currently, NIDS are deployed on high-performance, high-cost middleboxes that are responsible for monitoring a limited section of the network. The fast increasing size and aggregate throughput of modern data centre networks have come to challenge the current approach to anomaly detection to satisfy the fast growing compute demand. In this paper, we propose a novel approach to distributed intrusion detection systems based on the architecture of recently proposed event processing frameworks. We have designed and implemented a prototype system using Apache Storm to show the benefits of the proposed approach as well as the architectural differences with traditional systems. Our system distributes modules across the available devices within the network fabric and uses a centralised controller for orchestration, management and correlation. Following the Software Defined Networking (SDN) paradigm, the controller maintains a complete view of the network but distributes the processing logic for quick event processing while performing complex event correlation centrally. We have evaluated the proposed system using publicly available data centre traces and demonstrated that the system can scale with the network topology while providing high performance and minimal impact on packet latency

Network anomaly detection: a survey and comparative analysis of stochastic and deterministic methods

7 pages. 1 more figure than final CDC 2013 versionWe present five methods to the problem of network anomaly detection. These methods cover most of the common techniques in the anomaly detection field, including Statistical Hypothesis Tests (SHT), Support Vector Machines (SVM) and clustering analysis. We evaluate all methods in a simulated network that consists of nominal data, three flow-level anomalies and one packet-level attack. Through analyzing the results, we point out the advantages and disadvantages of each method and conclude that combining the results of the individual methods can yield improved anomaly detection results

SENATUS: An Approach to Joint Traffic Anomaly Detection and Root Cause Analysis

In this paper, we propose a novel approach, called SENATUS, for joint traffic anomaly detection and root-cause analysis. Inspired from the concept of a senate, the key idea of the proposed approach is divided into three stages: election, voting and decision. At the election stage, a small number of \nop{traffic flow sets (termed as senator flows)}senator flows are chosen\nop{, which are used} to represent approximately the total (usually huge) set of traffic flows. In the voting stage, anomaly detection is applied on the senator flows and the detected anomalies are correlated to identify the most possible anomalous time bins. Finally in the decision stage, a machine learning technique is applied to the senator flows of each anomalous time bin to find the root cause of the anomalies. We evaluate SENATUS using traffic traces collected from the Pan European network, GEANT, and compare against another approach which detects anomalies using lossless compression of traffic histograms. We show the effectiveness of SENATUS in diagnosing anomaly types: network scans and DoS/DDoS attacks

Increasing resilience of ATM networks using traffic monitoring and automated anomaly analysis

Systematic network monitoring can be the cornerstone for the dependable operation of safety-critical distributed systems. In this paper, we present our vision for informed anomaly detection through network monitoring and resilience measurements to increase the operators' visibility of ATM communication networks. We raise the question of how to determine the optimal level of automation in this safety-critical context, and we present a novel passive network monitoring system that can reveal network utilisation trends and traffic patterns in diverse timescales. Using network measurements, we derive resilience metrics and visualisations to enhance the operators' knowledge of the network and traffic behaviour, and allow for network planning and provisioning based on informed what-if analysis