Adaptive Aggregation of Flow Records

This paper explores the problem of processing the immense volume of measurement data arising during network traffic monitoring. Due to the ever-increasing demands of current networks, observing accurate information about every single flow is virtually infeasible. In many cases the existing methods for the reduction of flow records are still not sufficient enough. Since the accurate knowledge of flows termed as "heavy-hitters" suffices to fulfill most of the monitoring purposes, we decided to aggregate the flow records pertaining to non-heavy-hitters. However, due to the ever-changing nature of traffic, their identification is a challenge. To overcome this challenge, our proposed approach - the adaptive aggregation of flow records - automatically adjusts its operation to the actual traffic load and to the monitoring requirements. Preliminary experiments in existing network topologies showed that adaptive aggregation efficiently reduces the number of flow records, while a significant proportion of traffic details is preserved

Flow Monitoring Explained: From Packet Capture to Data Analysis With NetFlow and IPFIX

Flow monitoring has become a prevalent method for monitoring traffic in high-speed networks. By focusing on the analysis of flows, rather than individual packets, it is often said to be more scalable than traditional packet-based traffic analysis. Flow monitoring embraces the complete chain of packet observation, flow export using protocols such as NetFlow and IPFIX, data collection, and data analysis. In contrast to what is often assumed, all stages of flow monitoring are closely intertwined. Each of these stages therefore has to be thoroughly understood, before being able to perform sound flow measurements. Otherwise, flow data artifacts and data loss can be the consequence, potentially without being observed. This paper is the first of its kind to provide an integrated tutorial on all stages of a flow monitoring setup. As shown throughout this paper, flow monitoring has evolved from the early 1990s into a powerful tool, and additional functionality will certainly be added in the future. We show, for example, how the previously opposing approaches of deep packet inspection and flow monitoring have been united into novel monitoring approaches

Detection of HTTPS brute-force attacks in high-speed computer networks

Tato práce představuje přehled metod pro detekci síťových hrozeb se zaměřením na útoky hrubou silou proti webovým aplikacím, jako jsou WordPress a Joomla. Byl vytvořen nový dataset, který se skládá z provozu zachyceného na páteřní síti a útoků generovaných pomocí open-source nástrojů. Práce přináší novou metodu pro detekci útoku hrubou silou, která je založena na charakteristikách jednotlivých paketů a používá moderní metody strojového učení. Metoda funguje s šifrovanou HTTPS komunikací, a to bez nutnosti dešifrování jednotlivých paketů. Stále více webových aplikací používá HTTPS pro zabezpečení komunikace, a proto je nezbytné aktualizovat detekční metody, aby byla zachována základní viditelnost do síťového provozu.This thesis presents a review of flow-based network threat detection, with the focus on brute-force attacks against popular web applications, such as WordPress and Joomla. A new dataset was created that consists of benign backbone network traffic and brute-force attacks generated with open-source attack tools. The thesis proposes a method for brute-force attack detection that is based on packet-level characteristics and uses modern machine-learning models. Also, it works with encrypted HTTPS traffic, even without decrypting the payload. More and more network traffic is being encrypted, and it is crucial to update our intrusion detection methods to maintain at least some level of network visibility

Flow Data Collection in Large Scale Networks

In this chapter, we present flow-based network traffic monitoring of large scale networks. Continuous Internet traffic increase requires a deployment of advanced monitoring techniques to provide near real-time and long-term network visibility. Collected flow data can be further used for network behavioral analysis to indicate legitimate and malicious traffic, proving cyber threats, etc. An early warning system should integrate flow-based monitoring to ensure network situational awareness.Kapitola představuje monitorování síťového provozu v rozsáhlých počítačových sítích založené na IP tocích. Nepřetržitý růst internetového provozu vyžaduje nasazení pokročilých monitorovacích technik, které poskytují v reálném čase a dlouhodobě pohled na dění v síti. Nasbíraná data mohou dále sloužit pro analýzu chování sítě k rozlišení legitimního a škodlivého provozu, dokazování kybernetických hrozeb atd. Systém včasného varování by měl integrovat monitorování síťových toků, aby mohl poskytovat přehled o situaci na síti

OpenFlowMon: a fully distributed monitoring framework for virtualized environments

Proceedings of: 2021 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), 9 November 2021, Heraklion, Greece.Network monitoring allows a continuous assessment on the health and performance of the network infrastructure. With the significant change on how networks are deployed and operated, mainly due to the advent of virtualization technologies, alternative monitoring approaches are emerging to provide a finer-grained flow monitoring to complement already existing mechanisms and capabilities. In this paper, we proposed and developed an Open-Source Flow Monitoring Framework (OpenFlowMon), a fully distributed monitoring framework implemented solely with open-source solutions. This framework is used to assess the performance and the overhead introduced by two different flow monitoring approaches: (i) switch level and (ii) compute node level monitoring. Results show that monitoring at compute node level not only reduces the overhead but also mitigates a potential complex post-processing in east-to-west traffic.This work has been (partially) funded by H2020 EU/TW 5G-DIVE (Grant 859881) and H2020 5Growth (Grant 856709)

The Future of Network Flow Monitoring

Flow monitoring has been used for accounting and security for more than two decades. This paper describes how it was developed, what is its current status, and what challenges can be expected in this field in the following years

Enhanced IPFIX flow monitoring for VXLAN based cloud overlay networks

The demands for cloud computing services is rapidly growing due to its fast adoption and the migration of workloads from private data centers to cloud data centers. Many companies, small and large, prefer switching their data to the enterprise cloud environment rather than expanding their own data centers. As a result, the network traffic in cloud data centers is increasing rapidly. However, due to the dynamic resource provisioning and high-speed virtualized cloud networks, the traditional flow-monitoring systems is unable to provide detail visibility and information of traffic traversing the cloud overlay network environment. Hence, it does not fulfill the monitoring requirement of cloud overlay traffic. As the growth of cloud network traffic causes difficulties for the service providers and end-users to manage the traffic efficiently, an enhanced IPFIX flow monitoring mechanism for cloud overlay networks was proposed to address this problem. The monitoring mechanism provided detail visibility and information of overlay network traffic that traversed the cloud environment, which is not available in the current network monitoring systems. The experimental results showed that the proposed monitoring system able to capture overlay network traffic and segregated the tenant traffic based on virtual machines as compare to the standard monitoring system

Threats and Surprises behind IPv6 Extension Headers

The concept of Extension Headers, newly introduced with IPv6, is elusive and enables new types of threats in the Internet. Simply dropping all traffic containing any Extension Header — a current practice by operators-seemingly is an effective solution, but at the cost of possibly dropping legitimate traffic as well. To determine whether threats indeed occur, and evaluate the actual nature of the traffic, measurement solutions need to be adapted. By implementing these specific parsing capabilities in flow exporters and performing measurements on two different production networks, we show it is feasible to quantify the metrics directly related to these threats, and thus allow for monitoring and detection. Analysing the traffic that is hidden behind Extension Headers, we find mostly benign traffic that directly affects end-user QoE: simply dropping all traffic containing Extension Headers is thus a bad practice with more consequences than operators might be aware of