Flow Monitoring Explained: From Packet Capture to Data Analysis With NetFlow and IPFIX

Flow monitoring has become a prevalent method for monitoring traffic in high-speed networks. By focusing on the analysis of flows, rather than individual packets, it is often said to be more scalable than traditional packet-based traffic analysis. Flow monitoring embraces the complete chain of packet observation, flow export using protocols such as NetFlow and IPFIX, data collection, and data analysis. In contrast to what is often assumed, all stages of flow monitoring are closely intertwined. Each of these stages therefore has to be thoroughly understood, before being able to perform sound flow measurements. Otherwise, flow data artifacts and data loss can be the consequence, potentially without being observed. This paper is the first of its kind to provide an integrated tutorial on all stages of a flow monitoring setup. As shown throughout this paper, flow monitoring has evolved from the early 1990s into a powerful tool, and additional functionality will certainly be added in the future. We show, for example, how the previously opposing approaches of deep packet inspection and flow monitoring have been united into novel monitoring approaches

Adaptive Aggregation of Flow Records

This paper explores the problem of processing the immense volume of measurement data arising during network traffic monitoring. Due to the ever-increasing demands of current networks, observing accurate information about every single flow is virtually infeasible. In many cases the existing methods for the reduction of flow records are still not sufficient enough. Since the accurate knowledge of flows termed as "heavy-hitters" suffices to fulfill most of the monitoring purposes, we decided to aggregate the flow records pertaining to non-heavy-hitters. However, due to the ever-changing nature of traffic, their identification is a challenge. To overcome this challenge, our proposed approach - the adaptive aggregation of flow records - automatically adjusts its operation to the actual traffic load and to the monitoring requirements. Preliminary experiments in existing network topologies showed that adaptive aggregation efficiently reduces the number of flow records, while a significant proportion of traffic details is preserved

Detection of HTTPS brute-force attacks in high-speed computer networks

Tato práce představuje přehled metod pro detekci síťových hrozeb se zaměřením na útoky hrubou silou proti webovým aplikacím, jako jsou WordPress a Joomla. Byl vytvořen nový dataset, který se skládá z provozu zachyceného na páteřní síti a útoků generovaných pomocí open-source nástrojů. Práce přináší novou metodu pro detekci útoku hrubou silou, která je založena na charakteristikách jednotlivých paketů a používá moderní metody strojového učení. Metoda funguje s šifrovanou HTTPS komunikací, a to bez nutnosti dešifrování jednotlivých paketů. Stále více webových aplikací používá HTTPS pro zabezpečení komunikace, a proto je nezbytné aktualizovat detekční metody, aby byla zachována základní viditelnost do síťového provozu.This thesis presents a review of flow-based network threat detection, with the focus on brute-force attacks against popular web applications, such as WordPress and Joomla. A new dataset was created that consists of benign backbone network traffic and brute-force attacks generated with open-source attack tools. The thesis proposes a method for brute-force attack detection that is based on packet-level characteristics and uses modern machine-learning models. Also, it works with encrypted HTTPS traffic, even without decrypting the payload. More and more network traffic is being encrypted, and it is crucial to update our intrusion detection methods to maintain at least some level of network visibility

Flexible Network Flow Measurement

Táto diplomová práca sa zaoberá návrhom a implementáciou sondy pre meranie tokov na sieti. Obsahuje teoretický rozbor problematiky merania, popis algoritmov a techník používaných pri meraní na báze tokov. Pri návrhu architektúry sondy je kladený dôraz na efektívnu indexáciu záznamov tokov a flexibilitu záznamu tak, aby bola užívateľovi umožnená parametrizácia merania.This thesis deals with designing the probe used for measuring network flows. It contains theoretical analysis of network measurment topic, description of algorithms and principles used for network flow measurement. Emphasis on the probe architecture lies on efficient indexing algorithm and flow record flexibility, such that user is able to define format of flow record.

OpenFlowMon: a fully distributed monitoring framework for virtualized environments

Proceedings of: 2021 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), 9 November 2021, Heraklion, Greece.Network monitoring allows a continuous assessment on the health and performance of the network infrastructure. With the significant change on how networks are deployed and operated, mainly due to the advent of virtualization technologies, alternative monitoring approaches are emerging to provide a finer-grained flow monitoring to complement already existing mechanisms and capabilities. In this paper, we proposed and developed an Open-Source Flow Monitoring Framework (OpenFlowMon), a fully distributed monitoring framework implemented solely with open-source solutions. This framework is used to assess the performance and the overhead introduced by two different flow monitoring approaches: (i) switch level and (ii) compute node level monitoring. Results show that monitoring at compute node level not only reduces the overhead but also mitigates a potential complex post-processing in east-to-west traffic.This work has been (partially) funded by H2020 EU/TW 5G-DIVE (Grant 859881) and H2020 5Growth (Grant 856709)

Enhanced IPFIX flow monitoring for VXLAN based cloud overlay networks

The demands for cloud computing services is rapidly growing due to its fast adoption and the migration of workloads from private data centers to cloud data centers. Many companies, small and large, prefer switching their data to the enterprise cloud environment rather than expanding their own data centers. As a result, the network traffic in cloud data centers is increasing rapidly. However, due to the dynamic resource provisioning and high-speed virtualized cloud networks, the traditional flow-monitoring systems is unable to provide detail visibility and information of traffic traversing the cloud overlay network environment. Hence, it does not fulfill the monitoring requirement of cloud overlay traffic. As the growth of cloud network traffic causes difficulties for the service providers and end-users to manage the traffic efficiently, an enhanced IPFIX flow monitoring mechanism for cloud overlay networks was proposed to address this problem. The monitoring mechanism provided detail visibility and information of overlay network traffic that traversed the cloud environment, which is not available in the current network monitoring systems. The experimental results showed that the proposed monitoring system able to capture overlay network traffic and segregated the tenant traffic based on virtual machines as compare to the standard monitoring system

On-line monitoring of VoIP quality using IPFIX

The main goal of VoIP services is to provide a reliable and high-quality voice transmission over packet networks. In order to prove the quality of VoIP transmission, several approaches were designed. In our approach, we are concerned about on-line monitoring of RTP and RTCP traffic. Based on these data, we are able to compute main VoIP quality metrics including jitter, delay, packet loss, and finally R-factor and MOS values. This technique of VoIP quality measuring can be directly incorporated into IPFIX monitoring framework where an IPFIX probe analyses RTP/RTCP packets, computes VoIP quality metrics, and adds these metrics into extended IPFIX flow records. Then, these extended data are stored in a central IPFIX monitoring system called collector where can be used for monitoring purposes. This paper presents a functional implementation of IPFIX plugin for VoIP quality measurement and compares the results with results obtained by other tools

Monitoring multicast traffic in heterogeneous networks

Estágio realizado no INESC - Porto e orientado pelo Prof. Doutor Ricardo MorlaTese de mestrado integrado. Engenharia Electrotécnica e de Computadores - Major Telecomunicações. Faculdade de Engenharia. Universidade do Porto. 200