Traffic Verification for Network Anomaly Detection in Sensor Networks

AbstractThe traffic that is being injected to the network is increasing every day. It can be either normal or anomalous. Anomalous traffic is variation in the communication pattern from the normal one and hence anomaly detection is an important procedure in ensuring network resiliency. Probabilistic models can be used to model traffic for anomaly detection. In this paper, we use Gaussian Mixture Model for traffic verification. The traffic is captured and is given to the model to verification. Traffic which obeys the model is normal and those which disobey are anomalies. Analysis shows that the proposed system has better performance in terms of delay, throughput and packet delivery rati

Recognition of traffic generated by WebRTC communication

Network traffic recognition serves as a basic condition for network operators to differentiate and prioritize traffic for a number of purposes, from guaranteeing the Quality of Service (QoS), to monitoring safety, as well as monitoring and detecting anomalies. Web Real-Time Communication (WebRTC) is an open-source project that enables real-time audio, video, and text communication among browsers. Since WebRTC does not include any characteristic pattern for semantically based traffic recognition, this paper proposes models for recognizing traffic generated during WebRTC audio and video communication based on statistical characteristics and usage of machine learning in Weka tool. Five classification algorithms have been used for model development, such as Naive Bayes, J48, Random Forest, REP tree, and Bayes Net. The results show that J48 and BayesNet have the best performances in this experimental case of WebRTC traffic recognition. Future work will be focused on comparison of a wide range of machine learning algorithms using a large enough dataset to improve the significance of the results

Selección de discriminadores de tráfico de red para clasificación en tiempo real

There are several techniques to select a set of traffic features for traffic classification. However, most studies ignore the domain knowledge where traffic analysis or classification is performed and do not consider the always moving information carried in the networks. This paper describes a selection process of online network-traffic discriminators. We obtained 24 traffic features that can be processed on the fly and propose them as a base attribute set for future domain-aware online analysis, processing, or classification. For the selection of a set of traffic discriminators, and to avoid the inconveniences mentioned, we carried out three steps. The first step is a context knowledge-based manual selection of traffic features that meet the condition of being obtained on the fly from the flow. The second step is focused on the quality analysis of previously selected attributes to ensure the relevance of each one when performing a traffic classification. In the third step, the implementation of several incremental learning algorithms verified the usefulness of such attributes in online traffic classification processes. Existen varias técnicas para seleccionar un conjunto de variables para clasificación del tráfico de red. Sin embargo, muchos estudios ignoran el ámbito del conocimiento en donde el análisis y clasificación del tráfico tiene lugar y no consideran la información, siempre en movimiento, que se transporta en dichas redes. Este artículo describe el proceso de selección de discriminadores tráfico de redes en línea. Se obtuvieron 24 características que pueden procesarse en tiempo real y se proponen como los conjuntos de atributos base para futuros análisis, procesamiento y calificación conscientes del dominio (domain-aware). Para la selección de un conjunto de discriminadores de tráfico y con el fin de evitar los inconvenientes mencionados anteriormente, se llevaron a cabo tres etapas. La primera consiste en la selección manual basada en el conocimiento contextual de las características de tráfico de red que tengan las condiciones de obtener en tiempo real a partir del flujo. La segunda etapa se enfoca en la calidad del análisis de los atributos previamente seleccionados para asegurar la relevancia de cada uno a la hora de efectuar la clasificación del tráfico. En la tercera etapa, la implementación de varios algoritmos de aprendizaje incremental verifican la idoneidad de tales atributos en procesos de clasificación de tráfico en línea

Design and Implementation of the Network Traffic Management System Based on Fine-grained Features

随着网络应用的蓬勃发展，特别是广泛应用的P2P技术，给网络的有效管理带来了很大的困难。为更好的实现网络的有效控制和管理，必须对对网络流量中各种应用进行准确的识别与分类。近几年来国内外学者围绕流量识别与分类进行了众多研究，并取得了相当可观的研究成果，但这些研究多是从宏观角度对网络流量进行建模分析，没有过多关注网络流内部随时间变化的动态特性和随用户行为变化的交互特性，且目前的网络流识别分类研究多为离线的方法，对网络流量的控制和管理作用有限。 针对这些不足，本论文采用了双向的动态网络流模型，充分考虑网络流内部的动态交互特性，细粒度的网络流量特征刻画弥补了目前网络流量微观特性研究方面的不足，为网络流...Rapid development of network applications, particularly P2P technology brings difficulties to network management. Network flow identification and classification is very important for network security and management, which attracts many researchers all over the world in recent years, the researchers in this field have yielded fruitful achievements, but most of them focused on describing the trend o...学位：工程硕士院系专业：软件学院_工程硕士(软件工程)学号：X201023065

An Intelligent System to Detect the Type of Devices Sending and Receiving Data in the Network

Nowadays mobile and fixed devices are used interchangeably for surfing the web due to the huge improvements performed in mobile devices in the recent years. Both mobile and fixed devices with Internet connectivity are supplied with different types of connection, thus users can select the best one at any time depending on their environment. In general, the mobile devices allow users access to Internet using the 3G network or a common WiFi connection, and the fixed ones generally use a wireless or wired connection. Selecting one or another type of connection implies different features of the network environment, so Internet Service Providers need to adapt their infrastructure to guarantee acceptable levels of Quality of Service in every type of connection. In this paper we study the behavior of the devices according to their nature, that is, if it is a mobile or fixed device. First, we have classified the most significant network parameters and software application values in order to know the nature of the device. Our proposal uses an intelligent system based on neural networks and finite state machines that lets the Internet Service Provider know the type of device belongs to the traffic going to its network. The system analyzes the transport and application layers from TCP packets to discriminate the percentage of Internet traffic generated by mobile and fixed devices. Test results show the success of the developed system.Bri Molinero, D.; Canovas Solbes, A.; Tomás Gironés, J.; Lloret, J. (2013). An Intelligent System to Detect the Type of Devices Sending and Receiving Data in the Network. Network Protocols and Algorithms. 5(2):72-91. doi:10.5296/npa.v5i2.3833S72915

Can Passive Mobile Application Traffic be Identified using Machine Learning Techniques

Mobile phone applications (apps) can generate background traffic when the end-user is not actively using the app. If this background traffic could be accurately identified, network operators could de-prioritise this traffic and free up network bandwidth for priority network traffic. The background app traffic should have IP packet features that could be utilised by a machine learning algorithm to identify app-generated (passive) traffic as opposed to user-generated (active) traffic. Previous research in the area of IP traffic classification focused on classifying high level network traffic types originating on a PC device. This research was concerned with classifying low level app traffic originating on mobile phone device. An innovative experiment setup was designed in order to answer the research question. A mobile phone running Android OS was configured to capture app network data. Three specific data trace procedures where then designed to comprehensively capture sample active and passive app traffic data. Feature generation in previous research recommend computing new features based on IP packet data. This research proposes a different approach. Feature generation was enabled by exposing inherent IP packet attributes as opposed to computing new features. Specific evaluation metrics were also designed in order to quantify the accuracy of the machine learning models at classifying active and passive app traffic. Three decision tree models were implemented; C5.0, C&R tree and CHAID tree. Each model was built using a standard implementation and with boosting. The findings indicate that passive app network traffic can be classified with an accuracy up to 84.8% using a CHAID decision tree algorithm with model boosting enabled. The finding also suggested that features derived from the inherent IP packet attributes, such as time frame delta and bytes in flight, had significant predictive value

Extending NetFlow Records for Increasing Encrypted Traffic Classification Capabilities

Diplomová práca sa zaoberá výberom atribútov sieťových tokov vhodných pre klasifikáciu šifrovanej prevádzky, rozšírením záznamov NetFlow týmito atribútmi a vytvorením nástroja pre klasifikáciu šifrovaných tokov, ktoré používajú protokol TLS. Ako atribúty pre klasifikáciu boli vybrané: veľkosti paketov, medzipaketové medzery, počet paketov v toku a veľkosť toku. Po zvolení atribútov nasleduje návrh rozšírenia záznamov NetFlow o tieto atribúty a návrh algoritmu pre klasifikáciu šifrovanej sieťovej prevádzky. Rozšírenie záznamov bolo implementované v jazyku C v rámci exportéru od Flowmon Networks a.s.. Klasifikátor pre kolektor bol implementovaný v jazyku Python. Klasifikačný algoritmus používa model, k čomu bolo potrebné získať trénovacie dáta. Algoritmus bol pridaný aj na exportér, miesto klasifikácie je možné zvoliť. Po implementácií nasledovalo vytvorenie testovacích dát a vyhodnotenie úspešnosti algoritmu a taktiež testovanie rýchlosti klasifikácie. V najlepšom prípade bola dosiahnutá úspešnosť 47%.Master's thesis deals with selection of attributes proper for classification of encrypted traffic, with the extension of NetFlow entries with these attributes and with creating a tool for classify encrypted TLS traffic. The following attributes were selected: size of packets, inter-packet arrival times, number of packets in flow and size of the flow. Selection of attributes was followed by design of extending NetFlow records with these attributes for classifying encrypted traffic. Extension of records was implemented in language C for exporter of the company Flowmon Networks a.s.. Classifier for collector was implemented in language Python. Classifier is based on a model, for which training data were needed. The exporter contains the classifying algorithm too, the place of the classification can be set. The implementation was followed by creation of testing data and evaluation of the accuracy. The speed of the classifier was tested too. In the best case scenario 47% accuracy was achieved.

Development of a module for identification of Internet applications and of an interface for the DTMS-P2P platform

Mestrado em Engenharia Electrónica e TelecomunicaçõesNos últimos anos tem-se registrado um enorme crescimento no número e variedade de aplicações IP. De entre estes numerosos protocolos, há alguns cujas características é importante estudar para conhecer o seu comportamento na rede. Por isso, conseguir efectuar uma exacta correspondência entre tráfego e aplicações reveste-se de grande importância num enorme número de tarefas relacionadas com a gestão de redes e de medições. Estas podem incluir engenharia de tráfego, diferenciação de serviços, monitorização de desempenho e segurança. Várias metodologias têm sido usadas e testadas. A metodologia baseada na análise dos portos utilizados tem-se tornado progressivamente ineficaz pois muitas destas novas aplicações usam portos que não são standard ou são utilizados por outros protocolos. Consequentemente, têm sido utilizados novos métodos para identificar estas aplicações, consistindo nomeadamente na análise das características estatísticas ou na análise do campo de dados dos pacotes. A primeira aproximação apresenta, no entanto, algumas limitações em fornecer a exacta identificação dos diferentes tipos de tráfego IP. Portanto, uma análise mais precisa exige a inspecção do payload dos pacotes. Esta dissertação propõe um módulo de software baseado nesta técnica. Este módulo pode funcionar de forma autónoma ou ser inserido numa plataforma de monitorização de tráfego com uma arquitectura peer-to-peer. Tirando partido da arquitectura distribuída da plataforma de monitorização, o módulo de identificação de tráfego poderá ainda melhorar o seu desempenho. A segunda parte desta dissertação propõe a implementação de uma Interface de Programação de Aplicações (API) para estabelecer a comunicação com a plataforma de monitorização de tráfego. Pretende-se que diferentes módulos consigam, deste modo, executar os diversos comandos na plataforma recorrendo à API para estabelecer a comunicação. Esta dissertação termina com a proposta de um interface gráfico para a mencionada plataforma como um meio de teste da API implementada. Deste modo, criou-se por um interface intuitivo que permite a execução das várias medições possíveis recorrendo à API para comunicar com a plataforma de medição. Também se pretende substituir o uso da linha de comandos, permitindo um uso mais simplificado dos vários comandos que o sistema de monitorização permite. O interface também fornece mensagens de erro para indicar ao utilizador como executar os comandos correctamente. O interface e a API foram desenvolvido na linguagem Java de modo a permitir uma maior portabilidade para outras plataformas computacionais. ABSTRACT: In the last years we have witnessed a major increase in the number and variety of IP applications. There are some applications whose characteristics are important to study in order to gain a complete knowledge about their behavior in the network. Therefore, an accurate mapping of traffic to applications is of a noticeable importance in a wide range of network management and measurement tasks. These can include traffic engineering, service differentiation, performance/failure monitoring and security. Several approaches have been used. Port-based identification approaches have become inaccurate as many of these emerging applications use non-standard or ephemeral ports or use ports associated to other applications. Thus, new methodologies have been used to identify these applications: analysis based on the traffic statistical properties and analysis based on packet payload inspection. The first approach also presents several severe limitations in providing an exact identification of the different types of traffic. Therefore a more exact identification demands the examination of the user’s payload. This thesis proposes an identification software module based on the payload analysis approach to complete traffic classification. This module will be inserted in a monitoring network system with a peer-to-peer architecture (although it can also be used autonomously) and will take advantage of this distributed architecture. The second part of this thesis provides the implementation of an Application Programming Interface (API) to establish the communication with the traffic monitoring platform. It is intended to allow different modules to execute the various commands in the platform through the use of the API for the establishment of the communication. This dissertation concludes with the proposal of a graphical interface to the peer-to-peer monitoring system as a means for testing the implemented API. Therefore, an intuitive interface was created which allows the execution of the various commands based on the API for the establishment of the communication with the platform. This interface is also intended to replace command line interfaces, allowing for a more intuitive, simpler, faster and more straightforward deployment of all facilities provided by the monitoring system. It also provides feedback messages that will show how to execute these commands in a correct way. The interface and the API are developed in the Java language to provide more portability to other computational platforms

Network traffic classification : from theory to practice

Since its inception until today, the Internet has been in constant transformation. The analysis and monitoring of data networks try to shed some light on this huge black box of interconnected computers. In particular, the classification of the network traffic has become crucial for understanding the Internet. During the last years, the research community has proposed many solutions to accurately identify and classify the network traffic. However, the continuous evolution of Internet applications and their techniques to avoid detection make their identification a very challenging task, which is far from being completely solved. This thesis addresses the network traffic classification problem from a more practical point of view, filling the gap between the real-world requirements from the network industry, and the research carried out. The first block of this thesis aims to facilitate the deployment of existing techniques in production networks. To achieve this goal, we study the viability of using NetFlow as input in our classification technique, a monitoring protocol already implemented in most routers. Since the application of packet sampling has become almost mandatory in large networks, we also study its impact on the classification and propose a method to improve the accuracy in this scenario. Our results show that it is possible to achieve high accuracy with both sampled and unsampled NetFlow data, despite the limited information provided by NetFlow. Once the classification solution is deployed it is important to maintain its accuracy over time. Current network traffic classification techniques have to be regularly updated to adapt them to traffic changes. The second block of this thesis focuses on this issue with the goal of automatically maintaining the classification solution without human intervention. Using the knowledge of the first block, we propose a classification solution that combines several techniques only using Sampled NetFlow as input for the classification. Then, we show that classification models suffer from temporal and spatial obsolescence and, therefore, we design an autonomic retraining system that is able to automatically update the models and keep the classifier accurate along time. Going one step further, we introduce next the use of stream-based Machine Learning techniques for network traffic classification. In particular, we propose a classification solution based on Hoeffding Adaptive Trees. Apart from the features of stream-based techniques (i.e., process an instance at a time and inspect it only once, with a predefined amount of memory and a bounded amount of time), our technique is able to automatically adapt to the changes in the traffic by using only NetFlow data as input for the classification. The third block of this thesis aims to be a first step towards the impartial validation of state-of-the-art classification techniques. The wide range of techniques, datasets, and ground-truth generators make the comparison of different traffic classifiers a very difficult task. To achieve this goal we evaluate the reliability of different Deep Packet Inspection-based techniques (DPI) commonly used in the literature for ground-truth generation. The results we obtain show that some well-known DPI techniques present several limitations that make them not recommendable as a ground-truth generator in their current state. In addition, we publish some of the datasets used in our evaluations to address the lack of publicly available datasets and make the comparison and validation of existing techniques easier