The Future of Network Flow Monitoring

Flow monitoring has been used for accounting and security for more than two decades. This paper describes how it was developed, what is its current status, and what challenges can be expected in this field in the following years

Pre-Congestion Notification Encoding Comparison

DiffServ mechanisms have been developed to support Quality of Service (QoS). However, the level of assurance that can be provided with DiffServ without substantial over-provisioning is limited. Pre-Congestion Notification (PCN) investigates the use of per-flow admission control to provide the required service guarantees for the admitted traffic. While admission control will protect the QoS under\ud normal operating conditions, an additional flow termination mechanism is necessary in the times of heavy congestion (e.g. caused by route changes due to link or node failure).\ud Encoding and their transport are required to carry the congestion and pre-congestion information from the congestion and pre-congestion points to the decision points. This document provides a survey of\ud several encoding methods, using comparisons amongst them as a way to explain their strengths and weaknesses.\u

Detection of Dictionary Attacks on Network Services Using IP Flow Analysis

Stávající výzkumy naznačují, že je možné detekovat slovníkové útoky pomocí toků dat. Tento typ detekce byl úspěšně implementován například pro protokoly SSH, LDAP a RDP. Pro zjištění, zda je možné stejné způsoby detekce použít i pro poštovní protokoly, bylo vytvořeno virtuální testovací prostředí.   Z dat, které jsem v tomto prostředí získal, se mi podařilo odvodit charakteristiky útoků v tocích a zvolit statistickou hodnotu, která útoky odliší od legitimního provozu. Za hlavní charakteristiku útoků jsem zvolil rozptyl určitých parametrů toků. IP adresy, jejichž toky mají malý rozptyl vybraných parametrů a vysokou frekvenci příchodu paketů jsou považovány za nedůvěryhodné. Aby jsme vyloučili falešné detekce, rozptyl je počítán z historie IP adresy, která v případě legitimního uživatele obsahuje různé toky a zabrání označení této IP adresy za nebezpečnou. Tento princip byl použit k vytvoření skriptu, který detekuje útoky z výstupů kolektoru nfdump. Úspěšnost detekce útoků byla testována na klasifikovaných datech z reálného prostředí. Výsledky testů ukázali, že při dobrém nastavení hraničních hodnot je procento zachycených útoků velmi vysoké a výsledky jsou bez falešných pozitivních detekcí. Detekce útoků není omezena jen na poštové protokoly. Vzhledem k tomu, že návrh je univerzální, skript dokáže detekovat slovníkové útoky na SSH, LDAP, SIP, RDP, SQL, telnet i některé další útoky.Existing research suggests that it is possible to detect dictionary attacks using IP flows. This type of detection was successfully implemented for SSH, LDAP and RDP protocols. To determine whether it is possible to use the same methods of detection for e-mail protocols virtual test environment was created. I deduced the characteristics of attacks in flows from the data, which I gained from this virtual environment. Than I chose the statistical value that separates the attacks from legitimate traffic. Variance of specific flow parameters was chosen as main characteristic of attacks. IP addresses with flows that have small variance of chosen parameters and high frequency of packet arrival are considered untrustworthy. Variance is calculated from IP history to rule out false positives. The IP history of legitimate user contains variation of flows which prevents marking this IP address as dangerous. On the basis of this principal the script, which detects the attacks from the nfdump output, was created. The success of detection of the attacks was tested on classificated data from the real environment. The results of tests showed, that with good configuration of marginal values the percentage of detected attacks is high and there are no false positives. Detection is not limited only on mail protocols. With regard to universal design, the script is able to detect dictionary attacks on SSH, LDAP, SIP, RDP, SQL, telnet and some other attacks.

Plataforma de medida e caracterização de redes sem fios

Mestrado em Engenharia Electrónica e TelecomunicaçõesTem-se assistido nos últimos anos a um crescente interesse nas tecnologias de redes sem fios que, por consequência, têm sido alvo de um estudo e desenvolvimento cada vez maiores. Surge assim a necessidade de testar as várias tecnologias e configurações desenvolvidas com uma cada vez maior transparência, rigor e replicabilidade. Neste contexto existem três soluções para o estudo de tecnologias de rede, baseadas em simulações, emulações e redes de teste. É nesta última possibilidade que irá incidir esta dissertação, mais nomeadamente na perspectiva de recolha de métricas de rede e caracterização da rede sem fios. Neste trabalho é analisado o estado da arte actual, abordando os processos utilizados e as propostas relevantes para esta área, com particular destaque para a biblioteca de medidas OML. É proposta a implementação de uma extensão IPFIX à biblioteca OML, permitindo a recolha de métricas através deste protocolo de transporte, sendo posteriormente armazenadas numa base de dados. A solução apresentada é exposta em detalhe e analisada ao longo de testes, com foco nas métricas taxa de transferência, perda de pacotes e jitter. Por fim, a implementação desenvolvida é comparada com a biblioteca OML, a nível de resultados obtidos e de consumo de recursos do sistema.In past recent years there has been a rising interest in wireless networks technology, consequently becoming the target of an increasing development and study. Therefore, there is a greater demand for the ability to test the various developed technologies and configurations, with transparency, accuracy and replicability. In this context there are three solutions to study networks technology, based on simulations, emulations and testbeds. This dissertation is about the latter, specifically on the gathering of network metrics and characterization of the wireless network. In this work it is done an analysis of the current state of the art, approaching the used methods and relevant proposals in this area, with particular emphasis on the measurement library OML. It is presented the implementation of an IPFIX extension to the OML library, enabling the collection of network metrics through this transport protocol, for later storage in a database. The presented solution is exposed in detail and analysed through tests, focusing on the metrics throughput, packet loss and jitter. Finally, the developed implementation is compared to the OML library, regarding the obtained results and the system’s resources consumption

Análisis longitudinal de medidas de red

Este proyecto está dedicado al estudio, automatización y presentación longitudinal de medidas de red. El estudio longitudinal de medidas de red pretende explicar y caracterizar el comportamiento de una red en el medio y largo plazo, esto es meses o años, dando respuestas y ayudando a los gestores de red de forma complementaria al estudio de medidas de red en el rango de segundos, minutos u horas. Por ejemplo este tipo de medidas resulta más útil para dimensionar una red que pretenda mantener la calidad de servicio con el paso del tiempo, identificar patrones de compartimientos anómalos, identificar comportamientos irregulares e identificar individualmente a los potenciales causantes. De forma más concreta, este trabajo en primer lugar ha consistido en la comprensión del sistema de monitorización de la red académica española RedIRIS, que nos ha facilitado medidas de red reales durante un periodo de tres años basadas en flujos de red. Luego se ha trabajado en la automatización del pretratado de estas medidas y, a continuación, se han calculado estadísticas significativas del comportamiento de una red de comunicaciones. Estas estadísticas incluyen la evolución del ancho de banda consumido, el número de direcciones IP, el número de heavy-hitters o las horas más/menos cargada. En cuarto lugar, se han implementado scripts para facilitar la visualización y estudio de las variaciones de estas estadísticas con el tiempo. Por último lugar, se ha aplicado todo este desarrollo en RedIRIS y los resultados son mostrados como un caso de estudio significativo.This project is dedicated to the study, automation and introduction of longitudinal-network measurements. This study seeks to explain and describe the behavior of a network in the medium and long term, i.e., months and years, providing answers and helping network managers in a complementary way to the analysis of measurements in the range of seconds, minutes and hours. For example, such long-term studies are more useful to dimension a network for maintaining its quality of service over time, identify patterns of both anomalous and irregular behaviors, and identify the potential causes. More specifically, this work has first consisted of the understanding of the Spanish academic network's (RedIRIS) monitoring system which has provided us with real network measurements, specifically network flows, during three years. After, we have worked in the automation of the pretreatment tasks of these measurements and then, we have calculated several significant statistics to describe some pieces of behavior of a communication network. These statistics include the evolution of bandwidth, the number of IP addresses, the number of heavy-hitters and more/less busy hours. Fourth, a number of scripts have been implemented for ease both the visualization and inspection of the increases/decreases of these statistics over time. Finally, all this development has been applied on RedIRIS' network and the results are shown in this work as a significant study case

A New Generation of an IPFIX Collector

Tato práce se zabývá zpracováním záznamů toků vzniklých monitorováním provozu počítačových sítí z pohledu IPFIX kolektoru. Analyzuje současné řešení modulárního kolektoru, který již prošel značným historickým vývojem, a zaměřuje se na odhalení jeho silných a slabých stránek. Na základě získaných znalostí navrhuje nový kolektor, který významně upravuje řešení jednotlivých komponent pro práci se záznamy toků, klade důraz na vysokou propustnost a přidává dosud chybějící funkcionality. V rámci práce došlo i ke srovnání výkonnosti obou generací, které vyznívá pozitivně pro nový kolektor.This master's thesis addresses processing of flow monitoring records from a point of view of an IPFIX collector. It analysis the current solution of the modular collector, which went through considerable historical development, and focuses on revealing its strengths and weaknesses. Based on acquired knowledge, a new collector is designed. The new solution, which significantly modifies individual components for processing of flow records, focuses on high throughput and adds missing functionalities. The document also compares performance of both generations and the new collector clearly dominates.