Latency estimation of IP flows using NetFlow

A partir de la informació exportada per NetFlow, un protocol que representa el tràfic que va a traves de routers en forma de "flows" (packets agrupats si comparteixen certs camps, com origen destí, ports, protocol...), s'intenta esbrinar l'estat de la xarxa entre dos routers

System designs for bulk and user-generated content delivery in the internet

This thesis proposes and evaluates new system designs to support two emerging Internet workloads: (a) bulk content, such as downloads of large media and scientific libraries, and (b) user-generated content (UGC), such as photos and videos that users share online, typically on online social networks (OSNs). Bulk content accounts for a large and growing fraction of today\u27s Internet traffic. Due to the high cost of bandwidth, delivering bulk content in the Internet is expensive. To reduce the cost of bulk transfers, I proposed traffic shaping and scheduling designs that exploit the delay-tolerant nature of bulk transfers to allow ISPs to deliver bulk content opportunistically. I evaluated my proposals through software prototypes and simulations driven by real-world traces from commercial and academic ISPs and found that they result in considerable reductions in transit costs or increased link utilization. The amount of user-generated content (UGC) that people share online has been rapidly growing in the past few years. Most users share UGC using online social networking websites (OSNs), which can impose arbitrary terms of use, privacy policies, and limitations on the content shared on their websites. To solve this problem, I evaluated the feasibility of a system that allows users to share UGC directly from the home, thus enabling them to regain control of the content that they share online. Using data from popular OSN websites and a testbed deployed in 10 households, I showed that current trends bode well for the delivery of personal UGC from users\u27 homes. I also designed and deployed Stratus, a prototype system that uses home gateways to share UGC directly from the home.Schwerpunkt dieser Doktorarbeit ist der Entwurf und die Auswertung neuer Systeme zur Unterstützung von zwei entstehenden Internet-Workloads: (a) Bulk-Content, wie zum Beispiel die Übertragung von großen Mediendateien und wissenschaftlichen Datenbanken, und (b) nutzergenerierten Inhalten, wie zum Beispiel Fotos und Videos, die Benutzer üblicherweise in sozialen Netzwerken veröffentlichen. Bulk-Content macht einen großen und weiter zunehmenden Anteil im heutigen Internetverkehr aus. Wegen der hohen Bandbreitenkosten ist die Übertragung von Bulk-Content im Internet jedoch teuer. Um diese Kosten zu senken habe ich neue Scheduling- und Traffic-Shaping-Lösungen entwickelt, die die Verzögerungsresistenz des Bulk-Verkehrs ausnutzen und es ISPs ermöglichen, Bulk-Content opportunistisch zu übermitteln. Durch Software-Prototypen und Simulationen mit Daten aus dem gewerblichen und akademischen Internet habe ich meine Lösungen ausgewertet und herausgefunden, dass sich die Übertragungskosten dadurch erheblich senken lassen und die Ausnutzung der Netze verbessern lässt. Der Anteil an nutzergenerierten Inhalten (user-generated content, UGC), die im Internet veröffentlicht wird, hat in den letzen Jahren ebenfalls schnell zugenommen. Meistens wird UGC in sozialen Netzwerken (online social networks, OSN) veröffentlicht. Dadurch sind Benutzer den willkürlichen Nutzungsbedingungen, Datenschutzrichtlinien, und Einschränkungen des OSN-Providers unterworfen. Um dieses Problem zu lösen, habe ich die Machbarkeit eines Systems ausgewertet, anhand dessen Benutzer UGC direkt von zu Hause veröffentlichen und die Kontrolle über ihren UGC zurückgewinnen können. Meine Auswertung durch Daten aus zwei populären OSN-Websites und einem Feldversuch in 10 Haushalten deutet darauf hin, dass angesichts der Fortschritte in der Bandbreite der Zugangsnetze die Veröffentlichung von persönlichem UGC von zu Hause in der nahen Zukunft möglich sein könnte.Schließlich habe ich Stratus entworfen und entwickelt, ein System, das auf Home-Gateways basiert und mit dem Benutzer UGC direkt von zu Hause veröffentlichen können

Network monitoring and performance assessment: from statistical models to neural networks

Máster en Investigación e Innovación en Tecnologías de la Información y las ComunicacionesIn the last few years, computer networks have been playing a key role in many different fields. Companies have also evolved around the internet, getting advantage of the huge capacity of diffusion. Nevertheless, this also means that computer networks and IT systems have become a critical element for the business. In case of interruption or malfunction of the systems, this could result in devastating economic impact. In this light, it is necessary to provide models to properly evaluate and characterize the computer networks. Focusing on modeling, one has many different alternatives: from classical options based on statistic to recent alternatives based on machine learning and deep learning. In this work, we want to study the different models available for each context, paying attention to the advantage and disadvantages to provide the best solution for each case. To cover the majority of the spectrum, three cases have been studied: time-unaware phenomena, where we look at the bias-variance trade-off, time-dependent phenomena, where we pay attention the trends of the time series, and text processing to process attributes obtained by DPI. For each case, several alternatives have been studied and solutions have been tested both with synthetic data and real-world data, showing the successfulness of the proposa

Proactive measurement techniques for network monitoring in heterogeneous environments

Tesis doctoral inédita. Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Tecnología Electrónica y de las Comunicaciones, 201

A modular traffic sampling architecture for flexible network measurements

Dissertação de Mestrado (Programa Doutoral em Informática)The massive traffic volumes and the heterogeneity of services in today’s networks urge for flexible, yet simple measurement solutions to assist network management tasks, without impairing network performance. To turn treatable tasks requiring traffic analysis, sampling the traffic has become mandatory, triggering substantial research in the area. In fact, multiple sampling techniques have been proposed to assist network engineering tasks, each one targeting specific measurement goals and traffic scenarios. Despite that, there is still a lack of an encompassing solution able to support the flexible deployment of these techniques in production networks. In this context, this research work proposes a modular traffic sampling architecture able to foster the flexible design and deployment of efficient measurement strategies. The architecture is composed of three layers i.e., management plane, control plane and data plane covering key components to achieve versatile and lightweight measurements in diverse traffic scenarios and measurement activities. The flexibility and modularity in deploying different sampling strategies relies upon a novel taxonomy of sampling techniques, in which, current and emerging techniques are identified regarding their inner characteristics - granularity, selection trigger and selection scheme. Following the proposed taxonomy, a sampling framework prototype has been developed and used as an experimental implementation of the proposed architecture, providing a fair environment to assess and compare sampling techniques under distinct measurement scenarios. Supported by the sampling framework, distinct techniques have been evaluated regarding their performance in balancing the computational burden and the accuracy in supporting traffic workload estimation and flow analysis. The results have demonstrated the relevance and applicability of the proposed architecture, revealing that a modular and configurable approach to sampling is a step forward for improving sampling scope and efficiency.Os grandes volumes de tráfego e a heterogeneidade de serviços nas redes atuais requerem soluções de medição que sejam flexíveis e simples de modo a sustentar as tarefas de gestão de redes sem afetar o desempenho das mesmas. Para tornar tratável as tarefas que exigem análise de tráfego, tornou-se obrigatório recorrer a amostragem do tráfego, motivando uma investigação substancial na área. Como consequência, várias técnicas de amostragem foram propostas para auxiliar as tarefas de engenharia de redes, cada uma orientada a satisfazer objetivos de medição e cenários de tráfego específicos. Apesar disso, ainda não existe uma solução abrangente capaz de suportar a implantação flexível destas técnicas em redes de produção. Neste contexto, este trabalho propõe uma arquitetura modular de amostragem de tráfego capaz de fomentar a concepção flexível e a implementação de estratégias efi- cientes de medição de tráfego. A arquitetura é composta por três camadas, nomeadamente, camada de gestão, camada de controle e camada de dados, cobrindo os principais componentes para alcançar versatilidade e baixo custo computacional em variados cenários de tráfego e atividades de medição. A flexibilidade e modularidade na implementação de diferentes técnicas de amostragem baseia-se numa nova taxonomia, na qual técnicas atuais e emergentes são identificadas de acordo com suas características internas - granularidade, trigger de seleção e esquema de seleção. Seguindo a taxonomia proposta, um protótipo estruturando e agregando as diferentes técnicas de amostragem foi desenvolvido e utilizado na implementação experimental da arquitetura, permitindo avaliar e comparar as técnicas de amostragem em diversos cenários de medição. Suportado pelo protótipo desenvolvido, distintas técnicas foram avaliadas quanto ao seu desempenho em equilibrar a carga computacional e a acurácia na estimação do volume de tráfego e na análise de fluxos. Os resultados demonstraram a relevância e aplicabilidade da arquitetura de amostragem proposta, revelando que uma abordagem modular e configurável constitui um avanço no sentido de melhorar a eficiência na amostragem de tráfego

Retroactive Packet Sampling for Traffic Receipts

Is it possible to design a packet-sampling algorithm that prevents the network node that performs the sampling from treating the sampled packets preferentially? We study this problem in the context of designing a "network transparency" system. In this system, networks emit receipts for a small sample of the packets they observe, and a monitor collects these receipts to estimate each network's loss and delay performance. Sampling is a good building block for this system, because it enables a solution that is flexible and combines low resource cost with quantifiable accuracy. The challenge is cheating resistance: when a network's performance is assessed based on the conditions experienced by a small traffic sample, the network has a strong incentive to treat the sampled packets better than the rest. We contribute a sampling algorithm that is provably robust to such prioritization attacks, enables network performance estimation with quantifiable accuracy, and requires minimal resources. We confirm our analysis using real traffic traces

Domestic and mobile networks Measurements,analyses, and patterns

Cette thèse est structurée autour de contributions dans les domaines des réseaux domestiques et mobiles. Dans le contexte des réseaux domestiques, nous nous occupons à la fois de la caractérisation du trafic et de la dégradation des performances des applications. Dans le cas des réseaux mobiles, nous sommes intéressés par comprendre la relation entre la technologie sans fil et les opportunités de contact entre les nœuds mobiles. Nous résumons les principales contributions de cette thèse dans ce qui suit. Partie I (Optimisation des performances des applications dans les réseaux domestiques). L?augmentation du taux d'accès à Internet à la maison conduit à plus de populations avec des réseaux domestiques. Un réseau domestique connecte plusieurs appareils à l'internet permettant aux différents membres d'un ménage de partager l'accès à Internet et aux ressources du réseau local. Par conséquent, les applications fonctionnant en parallèle peuvent interférer les unes avec les autres. Par exemple, les enfants peuvent jouer à des jeux en ligne ralentissant la navigation sur le web de leurs parents. Le premier objectif de cette thèse est de contrôler l'utilisation des ressources du réseau domestique afin d'optimiser la performance des applications concurrentes. La passerelle domestique est responsable de la connexion du réseau domestique au reste de l'Internet. Parce que la passerelle a une vue d'ensemble de tout le trafic en provenance et vers le réseau domestique, elle est le point de départ idéal pour l'optimisation des applications. Dans cette thèse, nous proposons un système qui fonctionne sur la passerelle domestique pour détecter des dégradations de performances et optimiser l'allocation des ressources pour obtenir les meilleures performances des applications. En même temps, les passerelles résidentielles classiques ne comportent aucun mécanisme pour garantir une performance optimale aux applications. Une autre contribution de cette thèse est donc de proposer une approche d'optimisation des performances des applications pour les réseaux domestiques. En particulier, nous étudions la faisabilité du suivi des performances des applications sur les passerelles résidentielles. Nous montrons que, bien que la passerelle domestique a des ressources limitées, elle a encore la capacité de faire plus que simplement la transmission des paquets. Elle peut recueillir et exporter toutes les informations nécessaires pour effectuer notre méthode d'optimisation des performances. Partie II (Reproduction de traces de mobilité). La meilleure façon d'analyser ou de valider un protocole ou même le choix de conception dans les réseaux tolérants aux perturbations est à travers un déploiement réel. Néanmoins, en raison des difficultés de mise en œuvre et même de coûts financiers, I seulement quelques expérimentations ont été rapportées dans la littérature. En conséquence, plusieurs travaux s'appuient toujours sur des modèles de mobilité synthétiques. Alors que les modèles de mobilité synthétiques sont utiles pour isoler les paramètres spécifiques d'une solution ou aider à enquêter sur l'évolutivité d'un système, ils ne peuvent pas toujours refléter les conditions réelles. D'autre part, les traces de contact sont connues pour mieux représenter la mobilité de la vie réelle, mais aussi d'être difficile à obtenir. Et si une trace réelle était suffisante pour obtenir plusieurs autres, comme si nous avions effectué plusieurs expérimentations ? à cette fin, nous nous appuyons sur la mobilité plausible, un algorithme capable d'inférer un mouvement spatial à partir de traces de contact et nous proposons un système de reproduction de traces de mobilité qui, à partir d'une unique trace de contact réelle, offre de multiples traces de contact inspirées de la trace originale.This thesis is structured around contributions in the areas of domestic and mobile networks. In the context of home networks, we deal with both home traffic characterization and application performance degradation. In the case of mobile networks, we are interested in understanding the relationship between wireless technology and contact opportunities among nodes on the move.PARIS-JUSSIEU-Bib.électronique (751059901) / SudocSudocFranceF

Data transfer scheduling with advance reservation and provisioning

Over the years, scientific applications have become more complex and more data intensive. Although through the use of distributed resources the institutions and organizations gain access to the resources needed for their large-scale applications, complex middleware is required to orchestrate the use of these storage and network resources between collaborating parties, and to manage the end-to-end processing of data. We present a new data scheduling paradigm with advance reservation and provisioning. Our methodology provides a basis for provisioning end-to-end high performance data transfers which require integration between system, storage and network resources, and coordination between reservation managers and data transfer nodes. This allows researchers/users and higher level meta-schedulers to use data placement as a service where they can plan ahead and reserve time and resources for their data movement operations. We present a novel approach for evaluating time-dependent structures with bandwidth guaranteed paths. We present a practical online scheduling model using advance reservation in dynamic network with time constraints. In addition, we report a new polynomial algorithm presenting possible reservation options and alternatives for earliest completion and shortest transfer duration. We enhance the advance network reservation system by extending the underlying mechanism to provide a new service in which users submit their constraints and the system suggests possible reservation requests satisfying users\u27 requirements. We have studied scheduling data transfer operation with resource and time conflicts. We have developed a new scheduling methodology considering resource allocation in client sites and bandwidth allocation on network link connecting resources. Some other major contributions of our study include enhanced reliability, adaptability, and performance optimization of distributed data placement tasks. While designing this new data scheduling architecture, we also developed other important methodologies such as early error detection, failure awareness, job aggregation, and dynamic adaptation of distributed data placement tasks. The adaptive tuning includes dynamically setting data transfer parameters and controlling utilization of available network capacity. Our research aims to provide a middleware to improve the data bottleneck in high performance computing systems

Traffic Receipts for Network Transparency

Today's Internet is not transparent: when packets get lost or delayed, there is typically no information about where the problem occurred, hence no information about who is responsible. This results in Internet service providers (ISPs) offering service level agreements (SLAs) that cannot be verified, and governments enacting neutrality regulations that cannot be enforced. To remedy this, we propose a "transparency system," where each participating network emits receipts for traffic it receives and delivers; an independent monitor collects these receipts and makes decisions regarding the network's performance and neutrality (or lack thereof). The main challenge we face is misbehavior: On the one hand, a network that participates in such a system has a clear incentive to game the system and influence the monitor's decisions to its advantage, by manipulating either the receipts it emits or the corresponding traffic. On the other hand, the monitor (or, more precisely, an adversary who has access to the same information as the monitor, e.g., a government that has subpoenaed the monitor's records) may have an incentive to use the receipts emitted by a network in order to infer information that is otherwise private to the network, in particular, its internal topology. We make three contributions, each one to prevent a different type of misbehavior: (1) Incentive-compatible reporting, which ensures that networks have no incentive to manipulate the receipts they emit in order to claim better performance or fake neutrality. The key to our solution is a trade-off that we discover between network performance and neutrality: we design our system such that the more a network tries to exaggerate its estimated performance the more likely it is to be perceived to violate neutrality (and vice versa). (2) Unbiased reporting, which ensures that networks cannot manipulate the traffic for which they emit receipts in order to claim better performance. The key to our solution is delayed disclosure: we design receipt generation such that, by the time a network has all the information it needs to emit a correct receipt, the network has already forwarded the traffic that this receipt concerns, hence cannot manipulate it. (3) Topology-obfuscation reporting, which enables networks to emit the information that is necessary for the monitor to make correct decisions without leaking any information about internal network topology. The key to our solution is the observation that topology inference exploits the diversity of pairwise similarities between the delay vectors of different network paths; hence, we design receipt generation such that any delay vectors that the monitor might compute have almost 0 pairwise similarities. We conclude that it is possible to design a transparency system that enables networks to report on their own performance such that networks have no incentive to game the system and no fear of leaking information about their private topology