Automatización y detección de anomalías en tráfico de Internet

En las redes multigigabit de hoy en día, TCP/IPv4 es el protocolo de comunicación estándar. Para controlar la salud de la red, es habitual utilizar mecanismos de monitorización pasiva para controlar distintos tipos de indicadores de problemas que pueden sufrir las conexiones TCP. Algunos de estos son el desorden de los paquetes, la existencia de tráfico duplicado, clientes que anuncian ventanas de tamaño 0, exceso de conexiones que terminan con RST, o un elevado número de retransmisiones. Precisamente, detectar retransmisiones en flujos TCP supone todo un reto, debido a que es difícil determinar si un paquete es una retransmisión o no midiendo en un punto intermedio de la red sin utilizar muchos recursos computacionales y de memoria. Se ha explorado el estado del arte y no se han encontrado soluciones capaces de funcionar en redes con tráfico a 10Gbps que cumplan estos requisitos. El objetivo final de estos algoritmos es determinar si un flujo tiene muchas retransmisiones con respecto a la proporción de paquetes que pertenecen al mismo. Para lograrlo, en este trabajo se estudian dos algoritmos que determinar de forma heurística si un paquete es o no una retransmisión. Los algoritmos estudiados se centran en el seguimiento del número de secuencia más alto que se ha visto en un flujo, y en función de la posición relativa del siguiente segmento decidir si es una retransmisión. Se ha desarrollado un programa que implemente ambos algoritmos para probarlos. Se han realizado pruebas con tráfico HTTP obtenido de entornos de producción. Las pruebas han sido tanto de validación contra un programa ya existente de análisis forense, como de rendimiento para comprobar que el algoritmo que daba mejores resultados se podría integrar en DetectPro, una aplicación comercial de análisis de redes a 10Gbps. El resultado de las pruebas ha resultado ser exitoso para uno de los dos algoritmos, y ha resultado viable para ser probado en entornos de producción.The TCP/IP protocol is the standard communication mechanism in nowadays multigigabit networks. Network administrators use passive monitoring solutions to check the network’s health status. They usually look for an assortment of possible problems which may arise, such as duplicate packets, packet reordering, TCP hosts sending zero window announcements, an increase of RST packets, or retransmissions. Incidentally, the focus of this work is to study why TCP retransmissions occur and how to detect them. Detecting TCP retransmissions is a challenging problem which has no simple accurate approach. Looking at the current state of the art, it seems there aren’t any solution readily available which can accurately rule if a packet is a retransmission without performing complex trace analysis with high computational and memory requirements. This work’s ultimate goal is to check whether it is possible to easily test if a TCP flow had too many retransmissions. For this purpose, this work presents two heuristics to evaluate if a packet is or isn’t a retransmission. Both algorithms keep track of the highest seen sequence number in a flow, and they decide if the following packet is a retransmission by looking to the partial ordering of the packet’s sequence number and the aforementioned flow’s current highest seen sequence number. Both algorithms should strive to give an informed hint of the approximate number of retransmissions a TCP flow had. In order to test both algorithms, a small test program has been implemented and it has been fed real world HTTP traffic. The algorithm’s output has been validated against an existing forensic analysis software. Unfortunately, only one of the algorithms gives correct results with a reasonable margin of error. That algorithm’s performance has been tested, and has achieved 10Gbps when integrated in a commercial application. I conclude that this work’s objectives have been successfully achieved, and it’s worth trying to deploy it in real word environments

Análisis de métricas ligeras indicadoras de la calidad en Internet

The number of devices connected to the Internet is growing exponentially with the now ubiquitous Internet of Things. Each new connected device demands network resources to carry out its tasks, and further stresses the infrastructures. In order to be able to cater for this devices, we need to be able to measure how they perceive the network, and if the Quality of Service Degrades past a breaking point, some applications might stop working. Thus, network monitoring is an important task for network operators. However, the usual model of a single high performance probe in an aggregation node does not seem to scale as the traffic rates are growing more quickly than the technology. Hence, it is interesting to deploy metrics for the quality of service in small embedded devices or even in the routers themselves, which would monitor smaller networks with lower traffic rates. These metrics could be collected periodically in an aggregation node, to make big picture analysis and take proactive actions to prevent quality of service degradations. However, these embedded devices usually have very small memory and computational resources. Thus, it is important to develop lightweight metrics that require to store little state in memory and only perform some simple computations per packet. In this thesis we explore the capabilities of a router and an ODROID C2 have as active measurement probes. Afterwards, some experiments are performed to determine the suitable amount of traffic an ODROID could capture in a local network. Then, we explore the resources required to detect TCP retransmissions in Internet flows, providing models for both the accuracy and the memory requirements of the proposed algorithms. Finally, these algorithms are tested versus Tshark with real enterprise traces and the predictions derived from the presented models are also validated.Con la llegada del Internet of Things (IoT) ha crecido de manera exponencial el número de dispositivos conectados a Internet, cada uno de ellos requiriendo distintas calidades de servicio mínimas para poder funcionar de manera satisfactoria para los usuarios. Esto pone de manifiesto la importancia de la monitorización de las redes, y de la aplicación proactiva de políticas que eviten que la calidad de servicio se degrade demasiado. Los procesos de monitorización y captura de alto rendimiento tienden a centralizarse en un solo nodo en el que se analice todo el tráfico, sin embargo, éstos sistemas no pueden escalar hasta tasas arbitrarias. Por tanto, es interesante analizar la posibilidad de trasladar estos procesos de monitorización a los dispositivos IoT de una red o incluso en los propios routers, llevando este proceso a los extremos de la red, donde la tasa y el nivel de agregación son menores. Sin embargo, estos dispositivos tienden a estar muy limitados en recursos computacionales y de memoria disponible, por lo que sería beneficioso estudiar si las métricas utilizadas pueden simplificarse para consumir menos recursos, aunque sea a costa de perder un poco de precisión. En este trabajo se estudian las capacidades como dispositivos de medida activa de un router y una ODROID C2, y se realizan experimentos para ver el impacto de utilizar canales inalámbricos. También se estudian las capacidades de la ODROID como motor de captura en una red local. A continuación se estudian mecanismos simplificados de detección de retransmisiones en flujos TCP adecuados para ser desplegados en dispositivos que no puedan mantener mucho estado. Se presentan modelos analíticos con los que predecir la tasa de error y de consumo de memoria de los algoritmos, y se validan con trazas con tráfico real tomado en entornos empresariales frente a Tshark

On the design and performance evaluation of automatic traffic report generation systems with huge data volumes

In this paper, we analyze the performance issues involved in the generation of automated traffic reports for large IT infrastructures. Such reports allow the IT manager to proactively detect possible abnormal situations and roll out the corresponding corrective actions. With the ever‐increasing bandwidth of current networks, the design of automated traffic report generation systems is very challenging. In a first step, the huge volumes of collected traffic are transformed into enriched flow records obtained from diverse collectors and dissectors. Then, such flow records, along with time series obtained from the raw traffic, are further processed to produce a usable report. As will be shown, the data volume in flow records turns out to be very large as well and requires careful selection of the key performance indicators (KPIs) to be included in the report. In this regard, we discuss the use of high‐level languages versus low‐level approaches, in terms of speed and versatility. Furthermore, our design approach is targeted for rapid development in commodity hardware, which is essential to cost‐effectively tackle demanding traffic analysis scenarios. Actually, the paper shows feasibility of delivering a large number of KPIs, as will be detailed later, for several TBytes of traffic per day using a commodity hardware architecture and high‐level languagesThis work has been partially supported by the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund under the projects TRÁFICA (MINECO/FEDER TEC2015‐69417‐C2‐1‐R) and Procesado Inteligente de Tráfico (MINECO/FEDER TEC2015‐69417‐C2‐2‐

Online detection of pathological TCP flows with retransmissions in high-speed networks

Online Quality of Service (QoS) assessment in high speed networks is one of the key concerns for service providers, namely to detect QoS degradation on-the-fly as soon as possible and avoid customers’ complaints. In this regard, a Key Performance Indicator (KPI) is the number of TCP retransmissions per flow, which is related to packet losses or increased network and/or client/server latency. However, to accurately detect TCP retransmissions the whole sequence number list should be tracked which is a challenging task in multi-Gb/s networks. In this paper we show that the simplest approach of counting as a retransmission a packet whose sequence number is smaller than the previous one is enough to detect pathological flows with severe retransmissions. Such a lightweight approach eliminates the need of tracking the whole TCP flow history, which severely restricts traffic analysis throughput. Our findings show that low False Positive Rates (FPR) and False Negative Rates (FNR) can be achieved in the detection of such pathological flows with severe retransmissions, which are of paramount importance for QoS monitoring. Most importantly, we show that live detection of such pathological flows at 10 Gb/s rate per processing core is feasibleThis work has been partially funded by the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund under the projects TRÁFICA (MINECO/ FEDER TEC2015-69417-C2-1-R), Preproceso Inteligente de Tráfico (MINECO / FEDER TEC2015-69417-C2-2-R) and RACING DRONES (MINECO / FEDER RTC-2016-4744-7

Evaluación de equipamiento de bajo coste para realizar medidas de red en entornos domésticos

[ES] En la actualidad, la proliferación de dispositivos móviles y accesos a Internet utilizando tecnologías inalámbricas en los entornos domésticos obliga a cambiar las metodologías para la realización de medidas de red.Para que éstas representen fidedignamente las condiciones ofrecidas a los usuarios, las prestaciones del equipamiento de medida y el número de dispositivos empleados deben adaptarse a las condiciones reales de un despliegue.Para facilitar y abaratar el desarrollo de medidas en estas condiciones, este trabajo presenta una evaluación de las capacidades de varias plataformas de propósito general y bajo coste.Nuestros resultados muestran que, aunque aparecen limitaciones relacionadas con cómo son conectadas a la red y los protocolos empleados, son aptas para medir una gran variedad de situaciones.Este trabajo ha sido parcialmente financiado por el Ministerio de Economía y Competitividad y del Fondo Europeo de Desarrollo Regional a traves de los proyectos ´ TRAFICA (MINECO / FEDER TEC2015-69417-C2-1-R) ´ y RACING DRONES (MINECO / FEDER RTC-2016- 4744-7). Los autores tambien agradecen al Ministerio de Educacion Cultura y Deporte por la beca de colaboración del primer autor.Miravalls Sierra, E.; Muelas, D.; López De Vergara, J.; Ramos, J.; Aracil, J. (2018). Evaluación de equipamiento de bajo coste para realizar medidas de red en entornos domésticos. En XIII Jornadas de Ingeniería telemática (JITEL 2017). Libro de actas. Editorial Universitat Politècnica de València. 118-123. https://doi.org/10.4995/JITEL2017.2017.6501OCS11812

On the Use of Affordable COTS Hardware for Network Measurements: Limits and Good Practices

Wireless access technologies are widespread in domestic scenarios, and end users extensively use mobile phones or tablets to browse the Web. Therefore, methods and platforms for the measurement of network key performance indicators must be adapted to the peculiarities of this environment. In this light, the experiments should capture the true conditions of such connections, particularly in terms of the hardware and multi-device interactions that are present in real networks. On the basis of this, this paper presents an evaluation of the capabilities of several affordable commercial off-the-shelf (COTS) devices as network measuring probes, for example, computers-on-module or domestic routers with software measurement tools. Our main goal is to detect the limits of such devices and define a guide of good practices to optimize them. Hence, our work paves the way for the development of fair measurement systems in domestic networks with low expenditures. The obtained experimental results show that these types of devices are suitable as network measuring probes, if they are adequately configured and minimal accuracy losses are assumable