High-speed, in-band performance measurement instrumentation for next generation IP networks

Facilitating always-on instrumentation of Internet traffic for the purposes of performance measurement is crucial in order to enable accountability of resource usage and automated network control, management and optimisation. This has proven infeasible to date due to the lack of native measurement mechanisms that can form an integral part of the network‟s main forwarding operation. However, Internet Protocol version 6 (IPv6) specification enables the efficient encoding and processing of optional per-packet information as a native part of the network layer, and this constitutes a strong reason for IPv6 to be adopted as the ubiquitous next generation Internet transport. In this paper we present a very high-speed hardware implementation of in-line measurement, a truly native traffic instrumentation mechanism for the next generation Internet, which facilitates performance measurement of the actual data-carrying traffic at small timescales between two points in the network. This system is designed to operate as part of the routers' fast path and to incur an absolutely minimal impact on the network operation even while instrumenting traffic between the edges of very high capacity links. Our results show that the implementation can be easily accommodated by current FPGA technology, and real Internet traffic traces verify that the overhead incurred by instrumenting every packet over a 10 Gb/s operational backbone link carrying a typical workload is indeed negligible

The Dark Side(-Channel) of Mobile Devices: A Survey on Network Traffic Analysis

In recent years, mobile devices (e.g., smartphones and tablets) have met an increasing commercial success and have become a fundamental element of the everyday life for billions of people all around the world. Mobile devices are used not only for traditional communication activities (e.g., voice calls and messages) but also for more advanced tasks made possible by an enormous amount of multi-purpose applications (e.g., finance, gaming, and shopping). As a result, those devices generate a significant network traffic (a consistent part of the overall Internet traffic). For this reason, the research community has been investigating security and privacy issues that are related to the network traffic generated by mobile devices, which could be analyzed to obtain information useful for a variety of goals (ranging from device security and network optimization, to fine-grained user profiling). In this paper, we review the works that contributed to the state of the art of network traffic analysis targeting mobile devices. In particular, we present a systematic classification of the works in the literature according to three criteria: (i) the goal of the analysis; (ii) the point where the network traffic is captured; and (iii) the targeted mobile platforms. In this survey, we consider points of capturing such as Wi-Fi Access Points, software simulation, and inside real mobile devices or emulators. For the surveyed works, we review and compare analysis techniques, validation methods, and achieved results. We also discuss possible countermeasures, challenges and possible directions for future research on mobile traffic analysis and other emerging domains (e.g., Internet of Things). We believe our survey will be a reference work for researchers and practitioners in this research field.Comment: 55 page

Optical performance monitoring in optical packet-switched networks

Para poder satisfacer la demanda de mayores anchos de banda y los requisitos de los nuevos servicios, se espera que se produzca una evolución de las redes ópticas hacia arquitecturas reconfigurables dinámicamente. Esta evolución subraya la importancia de ofrecer soluciones en la que la escalabilidad y la flexibilidad sean las principales directrices. De acuerdo a estas características, las redes ópticas de conmutación de paquetes (OPS) proporcionan altas capacidades de transmisión, eficiencia en ancho de banda y excelente flexibilidad, además de permitir el procesado de los paquetes directamente en la capa óptica. En este escenario, la solución all-optical label switching (AOLS) resuelve el cuello de botella impuesto por los nodos que realizan el procesado en el dominio eléctrico. A pesar de los progresos en el campo del networking óptico, las redes totalmente ópticas todavía se consideran una solución lejana . Por tanto, es importante desarrollar un escenario de migración factible y gradual desde las actuales redes ópticas basadas en la conmutación de circuitos (OCS). Uno de los objetivos de esta tesis se centra en la propuesta de escenarios de migración basados en redes híbridas que combinan diferentes tecnologías de conmutación. Además, se analiza la arquitectura de una red OPS compuesta de nodos que incorporan nuevas funcionalidades relacionadas con labores de monitorización y esquemas de recuperación. Las redes ópticas permiten mejorar la transparencia de la red, pero a costa de aumentar la complejidad de las tareas de gesión. En este escenario, la monitorización óptica de prestaciones (OPM) surge como una tecnología capaz de facilitar la administración de las redes OPS, en las que cada paquete sigue su propia ruta en la red y sufre un diferente nivel de degradación al llegar a su destino. Aquí reside la importancia de OPM para garantizar los requisitos de calidad de cada paquete.Vilar Mateo, R. (2010). Optical performance monitoring in optical packet-switched networks [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8926Palanci

Load shedding in network monitoring applications

Monitoring and mining real-time network data streams are crucial operations for managing and operating data networks. The information that network operators desire to extract from the network traffic is of different size, granularity and accuracy depending on the measurement task (e.g., relevant data for capacity planning and intrusion detection are very different). To satisfy these different demands, a new class of monitoring systems is emerging to handle multiple and arbitrary monitoring applications. Such systems must inevitably cope with the effects of continuous overload situations due to the large volumes, high data rates and bursty nature of the network traffic. These overload situations can severely compromise the accuracy and effectiveness of monitoring systems, when their results are most valuable to network operators. In this thesis, we propose a technique called load shedding as an effective and low-cost alternative to over-provisioning in network monitoring systems. It allows these systems to handle efficiently overload situations in the presence of multiple, arbitrary and competing monitoring applications. We present the design and evaluation of a predictive load shedding scheme that can shed excess load in front of extreme traffic conditions and maintain the accuracy of the monitoring applications within bounds defined by end users, while assuring a fair allocation of computing resources to non-cooperative applications. The main novelty of our scheme is that it considers monitoring applications as black boxes, with arbitrary (and highly variable) input traffic and processing cost. Without any explicit knowledge of the application internals, the proposed scheme extracts a set of features from the traffic streams to build an on-line prediction model of the resource requirements of each monitoring application, which is used to anticipate overload situations and control the overall resource usage by sampling the input packet streams. This way, the monitoring system preserves a high degree of flexibility, increasing the range of applications and network scenarios where it can be used. Since not all monitoring applications are robust against sampling, we then extend our load shedding scheme to support custom load shedding methods defined by end users, in order to provide a generic solution for arbitrary monitoring applications. Our scheme allows the monitoring system to safely delegate the task of shedding excess load to the applications and still guarantee fairness of service with non-cooperative users. We implemented our load shedding scheme in an existing network monitoring system and deployed it in a research ISP network. We present experimental evidence of the performance and robustness of our system with several concurrent monitoring applications during long-lived executions and using real-world traffic traces.Postprint (published version

Deteção de propagação de ameaças e exfiltração de dados em redes empresariais

Modern corporations face nowadays multiple threats within their networks. In an era where companies are tightly dependent on information, these threats can seriously compromise the safety and integrity of sensitive data. Unauthorized access and illicit programs comprise a way of penetrating the corporate networks, able to traversing and propagating to other terminals across the private network, in search of confidential data and business secrets. The efficiency of traditional security defenses are being questioned with the number of data breaches occurred nowadays, being essential the development of new active monitoring systems with artificial intelligence capable to achieve almost perfect detection in very short time frames. However, network monitoring and storage of network activity records are restricted and limited by legal laws and privacy strategies, like encryption, aiming to protect the confidentiality of private parties. This dissertation proposes methodologies to infer behavior patterns and disclose anomalies from network traffic analysis, detecting slight variations compared with the normal profile. Bounded by network OSI layers 1 to 4, raw data are modeled in features, representing network observations, and posteriorly, processed by machine learning algorithms to classify network activity. Assuming the inevitability of a network terminal to be compromised, this work comprises two scenarios: a self-spreading force that propagates over internal network and a data exfiltration charge which dispatch confidential info to the public network. Although features and modeling processes have been tested for these two cases, it is a generic operation that can be used in more complex scenarios as well as in different domains. The last chapter describes the proof of concept scenario and how data was generated, along with some evaluation metrics to perceive the model’s performance. The tests manifested promising results, ranging from 96% to 99% for the propagation case and 86% to 97% regarding data exfiltration.Nos dias de hoje, várias organizações enfrentam múltiplas ameaças no interior da sua rede. Numa época onde as empresas dependem cada vez mais da informação, estas ameaças podem compremeter seriamente a segurança e a integridade de dados confidenciais. O acesso não autorizado e o uso de programas ilícitos constituem uma forma de penetrar e ultrapassar as barreiras organizacionais, sendo capazes de propagarem-se para outros terminais presentes no interior da rede privada com o intuito de atingir dados confidenciais e segredos comerciais. A eficiência da segurança oferecida pelos sistemas de defesa tradicionais está a ser posta em causa devido ao elevado número de ataques de divulgação de dados sofridos pelas empresas. Desta forma, o desenvolvimento de novos sistemas de monitorização ativos usando inteligência artificial é crucial na medida de atingir uma deteção mais precisa em curtos períodos de tempo. No entanto, a monitorização e o armazenamento dos registos da atividade da rede são restritos e limitados por questões legais e estratégias de privacidade, como a cifra dos dados, visando proteger a confidencialidade das entidades. Esta dissertação propõe metodologias para inferir padrões de comportamento e revelar anomalias através da análise de tráfego que passa na rede, detetando pequenas variações em comparação com o perfil normal de atividade. Delimitado pelas camadas de rede OSI 1 a 4, os dados em bruto são modelados em features, representando observações de rede e, posteriormente, processados por algoritmos de machine learning para classificar a atividade de rede. Assumindo a inevitabilidade de um terminal ser comprometido, este trabalho compreende dois cenários: um ataque que se auto-propaga sobre a rede interna e uma tentativa de exfiltração de dados que envia informações para a rede pública. Embora os processos de criação de features e de modelação tenham sido testados para estes dois casos, é uma operação genérica que pode ser utilizada em cenários mais complexos, bem como em domínios diferentes. O último capítulo inclui uma prova de conceito e descreve o método de criação dos dados, com a utilização de algumas métricas de avaliação de forma a espelhar a performance do modelo. Os testes mostraram resultados promissores, variando entre 96% e 99% para o caso da propagação e entre 86% e 97% relativamente ao roubo de dados.Mestrado em Engenharia de Computadores e Telemátic

Towards all-optical label switching nodes with multicast

Fiber optics has developed so rapidly during the last decades that it has be- come the backbone of our communication systems. Evolved from initially static single-channel point-to-point links, the current advanced optical backbone net- work consists mostly of wavelength-division multiplexed (WDM) networks with optical add/drop multiplexing nodes and optical cross-connects that can switch data in the optical domain. However, the commercially implemented optical net- work nodes are still performing optical circuit switching using wavelength routing. The dedicated use of wavelength and infrequent recon¯guration result in relatively poor bandwidth utilization. The success of electronic packet switching has inspired researchers to improve the °exibility, e±ciency, granularity and network utiliza- tion of optical networks by introducing optical packet switching using short, local optical labels for forwarding decision making at intermediate optical core network nodes, a technique that is referred to as optical label switching (OLS). Various research demonstrations on OLS systems have been reported with transparent optical packet payload forwarding based on electronic packet label processing, taking advantage of the mature technologies of electronic logical cir- cuitry. This approach requires optic-electronic-optic (OEO) conversion of the op- tical labels, a costly and power consuming procedure particularly for high-speed labels. As optical packet payload bit rate increases from gigabit per second (Gb/s) to terabit per second (Tb/s) or higher, the increased speed of the optical labels will eventually face the electronic bottleneck, so that the OEO conversion and the electronic label processing will be no longer e±cient. OLS with label processing in the optical domain, namely, all-optical label switching (AOLS), will become necessary. Di®erent AOLS techniques have been proposed in the last ¯ve years. In this thesis, AOLS node architectures based on optical time-serial label processing are presented for WDM optical packets. The unicast node architecture, where each optical packet is to be sent to only one output port of the node, has been in- vestigated and partially demonstrated in the EU IST-LASAGNE project. This thesis contributes to the multicast aspects of the AOLS nodes, where the optical packets can be forwarded to multiple or all output ports of a node. Multicast capable AOLS nodes are becoming increasingly interesting due to the exponen- tial growth of the emerging multicast Internet and modern data services such as video streaming, high de¯nition TV, multi-party online games, and enterprise ap- plications such as video conferencing and optical storage area networks. Current electronic routers implement multicast in the Internet protocol (IP) layer, which requires not only the OEO conversion of the optical packets, but also exhaus- tive routing table lookup of the globally unique IP addresses. Despite that, there has been no extensive studies on AOLS multicast nodes, technologies and tra±c performance, apart from a few proof-of-principle experimental demonstrations. In this thesis, three aspects of the multicast capable AOLS nodes are addressed: 1. Logical design of the AOLS multicast node architectures, as well as func- tional subsystems and interconnections, based on state-of-the-art literature research of the ¯eld and the subject. 2. Computer simulations of the tra±c performance of di®erent AOLS unicast and multicast node architectures, using a custom-developed AOLS simulator AOLSim. 3. Experimental demonstrations in laboratory and computer simulations using the commercially available simulator VPItransmissionMakerTM, to evaluate the physical layer performance of the required all-optical multicast technolo- gies. A few selected multi-wavelength conversion (MWC) techniques are particularly looked into. MWC is an essential subsystem of the AOLS node for realizing optical packet multicast by making multiple copies of the optical packet all-optically onto di®er- ent wavelengths channels. In this thesis, theMWC techniques based on cross-phase modulation and four-wave mixing are extensively investigated. The former tech- nique o®ers more wavelength °exibility and good conversion e±ciency, but it is only applicable to intensity modulated signals. The latter technique, on the other hand, o®ers strict transparency in data rate and modulation format, but its work- ing wavelengths are limited by the device or component used, and the conversion e±ciency is considerably lower. The proposals and results presented in this thesis show feasibility of all-optical packet switching and multicasting at line speed without any OEO conversion and electronic processing. The scalability and the costly optical components of the AOLS nodes have been so far two of the major obstacles for commercialization of the AOLS concept. This thesis also introduced a novel, scalable optical labeling concept and a label processing scheme for the AOLS multicast nodes. The pro- posed scheme makes use of the spatial positions of each label bit instead of the total absolute value of all the label bits. Thus for an n-bit label, the complexity of the label processor is determined by n instead of 2n