A Signal-Processing View on Packet Sampling and Anomaly Detection

International audienceAnomaly detection methods typically operate on preprocessed traffic traces. Firstly, most traffic capturing devices today employ random packet sampling, where each packet is selected with a certain probability, to cope with increasing link speeds. Secondly, temporal aggregation, where all packets in a measurement interval are represented by their temporal mean, is applied to transform the traffic trace to the observation timescale of interest for anomaly detection. These preprocessing steps affect the temporal correlation structure of traffic that is used by anomaly detection methods such as Kalman filtering or PCA, and have thus an impact on anomaly detection performance. Prior work has analyzed how packet sampling degrades the accuracy of anomaly detection methods; however, neither theoretical explanations nor solutions to the sampling problem have been provided. This paper makes the following key contributions: (i) It provides a thorough analysis and quantification of how random packet sampling and temporal aggregation modify the signal properties by introducing noise, distortion and aliasing. (ii) We show that aliasing introduced by the aggregation step has the largest impact on the correlation structure. (iii) We further propose to replace the aggregation step with a specifically designed low-pass filter that reduces the aliasing effect. (iv) Finally, we show that with our solution applied, the performance of anomaly detection systems can be considerably improved in the presence of packet sampling

Applying PCA for Traffic Anomaly Detection: Problems and Solutions

International audienceSpatial Principal Component Analysis (PCA) has been proposed for network-wide anomaly detection. A recent work has shown that PCA is very sensitive to calibration settings. Unfortunately, the authors did not provide further explanations for this observation. In this paper, we fill this gap and provide the reasoning behind the found discrepancies. We revisit PCA for anomaly detection and evaluate its performance on our data. We develop a slightly modified version of PCA that uses only data from a single router. Instead of correlating data across different spatial measurement points, we correlate the data across different metrics. With the help of the analyzed data, we explain the pitfalls of PCA and underline our argumentation with measurement results. We show that the main problem is that PCA fails to capture temporal correlation. We propose a solution to deal with this problem by replacing PCA with the Karhunen-Loeve transform. We find that when we consider temporal correlation, anomaly detection results are significantly improved

Automated Pattern-Based Service Deployment in Programmable Networks

This paper presents a flexible service deployment architecture for the automated, on-demand deployment of distributed services in programmable networks. The novelty of our approach is (a) the customization of the deployment protocol by utilizing modular building blocks, namely navigation patterns, aggregation patterns, and capability functions, and (b) the definition of a corresponding service descriptor. A customizable deployment protocol has several important advantages: It supports a multitude of services, and it allows for an ad hoc optimization of the protocol according to the specific needs of a service and the current network conditions. Moreover, our architecture provides an environment for studying new patterns which aim at reducing deployment latency and bandwidth for certain services. We demonstrate how the developed architecture can be used to setup a virtual private network, and we present measurements conducted with our prototype in the PlanetLab test network. Furthermore, a comparison of a distributed pattern with a centralized pattern illustrates the performance trade-off for different deployment strategie

A Signal Processing View on Packet Sampling and Anomaly Detection

This find is registered at Portable Antiquities of the Netherlands with number PAN-0002837

Automating root-cause analysis of network anomalies using frequent itemset mining

Finding the root-cause of a network security anomaly is essential for network operators. In our recent work [1, 5], we introduced a generic technique that uses frequent itemset mining to automatically extract and summarize the traffic flows causing an anomaly. Our evaluation using two different anomaly detectors (including a commercial one) showed that our approach works surprisingly well extracting the anomalous flows in most studied cases using sampled and unsampled NetFlow traces from two networks. In this demonstration, we will showcase an open-source anomaly-extraction system based on our technique, which we integrated with a commercial anomaly detector and use in the NOC of the GÉANT network since late 2009. We will report a number of detected security anomalies and will illustrate how an operator can use our system to automatically extract and summarize anomalous flows.Peer ReviewedPostprint (published version