Static Web content distribution and request routing in a P2P overlay

The significance of collaboration over the Internet has become a corner-stone of modern computing, as the essence of information processing and content management has shifted to networked and Webbased systems. As a result, the effective and reliable access to networked resources has become a critical commodity in any modern infrastructure. In order to cope with the limitations introduced by the traditional client-server networking model, most of the popular Web-based services have employed separate Content Delivery Networks (CDN) to distribute the server-side resource consumption. Since the Web applications are often latency-critical, the CDNs are additionally being adopted for optimizing the content delivery latencies perceived by the Web clients. Because of the prevalent connection model, the Web content delivery has grown to a notable industry. The rapid growth in the amount of mobile devices further contributes to the amount of resources required from the originating server, as the content is also accessible on the go. While the Web has become one of the most utilized sources of information and digital content, the openness of the Internet is simultaneously being reduced by organizations and governments preventing access to any undesired resources. The access to information may be regulated or altered to suit any political interests or organizational benefits, thus conflicting with the initial design principle of an unrestricted and independent information network. This thesis contributes to the development of more efficient and open Internet by combining a feasibility study and a preliminary design of a peer-to-peer based Web content distribution and request routing mechanism. The suggested design addresses both the challenges related to effectiveness of current client-server networking model and the openness of information distributed over the Internet. Based on the properties of existing peer-to-peer implementations, the suggested overlay design is intended to provide low-latency access to any Web content without sacrificing the end-user privacy. The overlay is additionally designed to increase the cost of censorship by forcing a successful blockade to isolate the censored network from the rest of the Internet

Making broadband access networks transparent to researchers, developers, and users

Broadband networks are used by hundreds of millions of users to connect to the Internet today. However, most ISPs are hesitant to reveal details about their network deployments,and as a result the characteristics of broadband networks are often not known to users,developers, and researchers. In this thesis, we make progress towards mitigating this lack of transparency in broadband access networks in two ways. First, using novel measurement tools we performed the first large-scale study of thecharacteristics of broadband networks. We found that broadband networks have very different characteristics than academic networks. We also developed Glasnost, a system that enables users to test their Internet access links for traffic differentiation. Glasnost has been used by more than 350,000 users worldwide and allowed us to study ISPs' traffic management practices. We found that ISPs increasingly throttle or even block traffic from popular applications such as BitTorrent. Second, we developed two new approaches to enable realistic evaluation of networked systems in broadband networks. We developed Monarch, a tool that enables researchers to study and compare the performance of new and existing transport protocols at large scale in broadband environments. Furthermore, we designed SatelliteLab, a novel testbed that can easily add arbitrary end nodes, including broadband nodes and even smartphones, to existing testbeds like PlanetLab.Breitbandanschlüsse werden heute von hunderten Millionen Nutzern als Internetzugang verwendet. Jedoch geben die meisten ISPs nur ungern über Details ihrer Netze Auskunft und infolgedessen sind Nutzern, Anwendungsentwicklern und Forschern oft deren Eigenheiten nicht bekannt. Ziel dieser Dissertation ist es daher Breitbandnetze transparenter zu machen. Mit Hilfe neuartiger Messwerkzeuge konnte ich die erste groß angelegte Studie über die Besonderheiten von Breitbandnetzen durchführen. Dabei stellte sich heraus, dass Breitbandnetze und Forschungsnetze sehr unterschiedlich sind. Mit Glasnost habe ich ein System entwickelt, das mehr als 350.000 Nutzern weltweit ermöglichte ihren Internetanschluss auf den Einsatz von Verkehrsmanagement zu testen. Ich konnte dabei zeigen, dass ISPs zunehmend BitTorrent Verkehr drosseln oder gar blockieren. Meine Studien zeigten dar überhinaus, dass existierende Verfahren zum Testen von Internetsystemen nicht die typischen Eigenschaften von Breitbandnetzen berücksichtigen. Ich ging dieses Problem auf zwei Arten an: Zum einen entwickelte ich Monarch, ein Werkzeug mit dem das Verhalten von Transport-Protokollen über eine große Anzahl von Breitbandanschlüssen untersucht und verglichen werden kann. Zum anderen habe ich SatelliteLab entworfen, eine neuartige Testumgebung, die, anders als zuvor, beliebige Internetknoten, einschließlich Breitbandknoten und sogar Handys, in bestehende Testumgebungen wie PlanetLab einbinden kann

Architectures for the Future Networks and the Next Generation Internet: A Survey

Networking research funding agencies in the USA, Europe, Japan, and other countries are encouraging research on revolutionary networking architectures that may or may not be bound by the restrictions of the current TCP/IP based Internet. We present a comprehensive survey of such research projects and activities. The topics covered include various testbeds for experimentations for new architectures, new security mechanisms, content delivery mechanisms, management and control frameworks, service architectures, and routing mechanisms. Delay/Disruption tolerant networks, which allow communications even when complete end-to-end path is not available, are also discussed

Walkabout : an asynchronous messaging architecture for mobile devices

Modern mobile devices are prolific producers and consumers of digital data, and wireless networking capabilities enable them to transfer their data over the Internet while moving. Applications running on these devices may perform transfers to upload data for backup or distribution, or to download new content on demand. Unfortunately, the limited connectivity that mobile devices experience can make these transfers slow and impractical as the amount of data increases. This thesis argues that asynchronous messaging supported by local proxies can improve the transfer capabilities of mobile devices, making it practical for them to participate in large Internet transfers. The design of the Walkabout architecture follows this approach: proxies form store-and-forward overlay networks to deliver messages asynchronously across the Internet on behalf of devices. A mobile device uploads a message to a local proxy at rapid speed, and the overlay delivers it to one or more destination devices, caching the message until each one is able to retrieve it from a local proxy. A device is able to partially upload or download a message whenever it has network connectivity, and can resume this transfer at any proxy if interrupted through disconnection. Simulation results show that Walkabout provides better throughput for mobile devices than is possible under existing methods, for a range of movement patterns. Upload and end-to-end to transfer speeds are always high when the device sending the message is mobile. In the basic Walkabout model, a message sent to a mobile device that is repeatedly moving between a small selection of connection points experiences high download and end-to-end transfer speeds, but these speeds fall as the number of connection points grows. Pre-emptive message delivery extensions improve this situation, making fast end-to-end transfers and device downloads possible under any pattern of movement. This thesis describes the design and evaluation of Walkabout, with both practical implementation and extensive simulation under real-world scenarios

System designs for bulk and user-generated content delivery in the internet

This thesis proposes and evaluates new system designs to support two emerging Internet workloads: (a) bulk content, such as downloads of large media and scientific libraries, and (b) user-generated content (UGC), such as photos and videos that users share online, typically on online social networks (OSNs). Bulk content accounts for a large and growing fraction of today\u27s Internet traffic. Due to the high cost of bandwidth, delivering bulk content in the Internet is expensive. To reduce the cost of bulk transfers, I proposed traffic shaping and scheduling designs that exploit the delay-tolerant nature of bulk transfers to allow ISPs to deliver bulk content opportunistically. I evaluated my proposals through software prototypes and simulations driven by real-world traces from commercial and academic ISPs and found that they result in considerable reductions in transit costs or increased link utilization. The amount of user-generated content (UGC) that people share online has been rapidly growing in the past few years. Most users share UGC using online social networking websites (OSNs), which can impose arbitrary terms of use, privacy policies, and limitations on the content shared on their websites. To solve this problem, I evaluated the feasibility of a system that allows users to share UGC directly from the home, thus enabling them to regain control of the content that they share online. Using data from popular OSN websites and a testbed deployed in 10 households, I showed that current trends bode well for the delivery of personal UGC from users\u27 homes. I also designed and deployed Stratus, a prototype system that uses home gateways to share UGC directly from the home.Schwerpunkt dieser Doktorarbeit ist der Entwurf und die Auswertung neuer Systeme zur Unterstützung von zwei entstehenden Internet-Workloads: (a) Bulk-Content, wie zum Beispiel die Übertragung von großen Mediendateien und wissenschaftlichen Datenbanken, und (b) nutzergenerierten Inhalten, wie zum Beispiel Fotos und Videos, die Benutzer üblicherweise in sozialen Netzwerken veröffentlichen. Bulk-Content macht einen großen und weiter zunehmenden Anteil im heutigen Internetverkehr aus. Wegen der hohen Bandbreitenkosten ist die Übertragung von Bulk-Content im Internet jedoch teuer. Um diese Kosten zu senken habe ich neue Scheduling- und Traffic-Shaping-Lösungen entwickelt, die die Verzögerungsresistenz des Bulk-Verkehrs ausnutzen und es ISPs ermöglichen, Bulk-Content opportunistisch zu übermitteln. Durch Software-Prototypen und Simulationen mit Daten aus dem gewerblichen und akademischen Internet habe ich meine Lösungen ausgewertet und herausgefunden, dass sich die Übertragungskosten dadurch erheblich senken lassen und die Ausnutzung der Netze verbessern lässt. Der Anteil an nutzergenerierten Inhalten (user-generated content, UGC), die im Internet veröffentlicht wird, hat in den letzen Jahren ebenfalls schnell zugenommen. Meistens wird UGC in sozialen Netzwerken (online social networks, OSN) veröffentlicht. Dadurch sind Benutzer den willkürlichen Nutzungsbedingungen, Datenschutzrichtlinien, und Einschränkungen des OSN-Providers unterworfen. Um dieses Problem zu lösen, habe ich die Machbarkeit eines Systems ausgewertet, anhand dessen Benutzer UGC direkt von zu Hause veröffentlichen und die Kontrolle über ihren UGC zurückgewinnen können. Meine Auswertung durch Daten aus zwei populären OSN-Websites und einem Feldversuch in 10 Haushalten deutet darauf hin, dass angesichts der Fortschritte in der Bandbreite der Zugangsnetze die Veröffentlichung von persönlichem UGC von zu Hause in der nahen Zukunft möglich sein könnte.Schließlich habe ich Stratus entworfen und entwickelt, ein System, das auf Home-Gateways basiert und mit dem Benutzer UGC direkt von zu Hause veröffentlichen können

Effective techniques for detecting and locating traffic differentiation in the internet

Orientador: Elias P. Duarte Jr.Coorientador: Luis C. E. BonaTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 24/09/2019Inclui referências: p. 115-126Área de concentração: Ciência da ComputaçãoResumo: A Neutralidade da Rede torna-se cada vez mais relevante conforme se intensifica o debate global e diversos governos implementam regulações. Este princípio diz que todo tráfego deve ser processado sem diferenciação, independentemente da origem, destino e/ou conteúdo. Práticas de diferenciação de tráfego (DT) devem ser transparentes, independentemente de regulações, pois afetam significativamente usuários finais. Assim, é essencial monitorar DT na Internet. Várias soluções já foram propostas para detectar DT. Essas soluções baseiam-se em medições de rede e inferência estatística. Porém, existem desafios em aberto. Esta tese tem três objetivos principais: (i) consolidar o estado da arte referente ao problema de detectar DT; (ii) investigar a DT em contextos ainda não explorados, especificamente a Internet das Coisas (IoT); e (iii) propor novas soluções para detecção de DT que solucionem alguns dos desafios em aberto, em particular localizar a fonte de DT. Primeiramente descrevemos o atual estado da arte, incluindo várias soluções de detecção de DT. Também propomos uma taxonomia para os diferentes tipos de DT e de detecção, e identificamos desafios em aberto. Em seguida, avaliamos o impacto da DT na IoT, simulando DT de diferentes padrões de tráfego IoT. Resultados mostram que mesmo uma priorização pequena pode ter um impacto significativo no desempenho de dispositivos de IoT. Propomos então uma solução para detectar DT na Internet, que baseia-se em uma nova estratégia que combina diversas métricas para detectar tipos diferente de DT. Resultados de simulação mostram que esta estratégia é capaz de detectar DT em diversas situações. Em seguida, propomos um modelo geral para monitoramento contínuo de DT na Internet, que se propõe a unificar as soluções atuais e futuras de detecção de DT, ao mesmo tempo que tira proveito de tecnologias atuais e emergentes. Neste contexto, uma nova solução para identificar a fonte de DT na Internet é proposta. O objetivo desta proposta é tanto viabilizar a implementação do nosso modelo geral quanto solucionar o problema de localizar DT. A proposta tira proveito de propriedades de roteamento da Internet para identificar em qual Sistema Autônomo (AS) DT acontece. Medições de vários pontos de vista são combinadas, e a fonte de DT é inferida com base nos caminhos em nível de AS entre os pontos de medição. Para avaliar esta proposta, primeiramente executamos experimentos para confirmar que rotas na Internet realmente apresentam as propriedades requeridas. Diversas simulações foram então executadas para avaliar a eficiência da proposta de localização de DT. Resultados mostram que em diversas situações, efetuar medições a partir de poucos nodos no núcleo da Internet obtém resultados similares a efetuar medições a partir de muitos nodos na borda. Palavras-chave: Neutralidade da Rede, Diferenciação de Tráfego, Medição de Rede.Abstract: Network Neutrality is becoming increasingly important as the global debate intensifies and governments worldwide implement and withdraw regulations. According to this principle, all traffic must be processed without differentiation, regardless of origin, destination and/or content. Traffic Differentiation (TD) practices should be transparent, regardless of regulations, since they can significantly affect end-users. It is thus essential to monitor TD in the Internet. Several solutions have been proposed to detect TD. These solutions are based on network measurements and statistical inference. However, there are still open challenges. This thesis has three main objectives: (i) to consolidate the state of the art regarding the problem of detecting TD; (ii) to investigate TD on contexts not yet explored, in particular the Internet of Things (IoT); and (iii) to propose new solutions regarding TD detection that address open challenges, in particular locating the source of TD. We first describe the current state of the art, including a description of multiple solutions for detecting TD. We also propose a taxonomy for the different types of TD and the different types of detection, and identify open challenges. Then, we evaluate the impact of TD on IoT, by simulating TD on different IoT traffic patterns. Results show that even a small prioritization may have a significant impact on the performance of IoT devices. Next, we propose a solution for detecting TD in the Internet. This solution relies on a new strategy of combining several metrics to detect different types of TD. Simulation results show that this strategy is capable of detecting TD under several conditions. We then propose a general model for continuously monitoring TD on the Internet, which aims at unifying current and future TD detection solutions, while taking advantage of current and emerging technologies. In this context, a new solution for locating the source of TD in the Internet is proposed. The goal of this proposal is to both enable the implementation of our general model and address the problem of locating TD. The proposal takes advantage of properties of Internet peering to identify in which Autonomous System (AS) TD occurs. Probes from multiple vantage points are combined, and the source of TD is inferred based on the AS-level routes between the measurement points. To evaluate this proposal, we first ran several experiments to confirm that indeed Internet routes do present the required properties. Then, several simulations were performed to assess the efficiency of the proposal for locating TD. The results show that for several different scenarios issuing probes from a few end-hosts in core Internet ASes achieves similar results than from numerous end-hosts on the edge. Keywords: Network Neutrality, Traffic Differentiation, Network Measurement

Energy Efficiency of P2P and Distributed Clouds Networks

Since its inception, the Internet witnessed two major approaches to communicate digital content to end users: peer to peer (P2P) and client/server (C/S) networks. Both approaches require high bandwidth and low latency physical underlying networks to meet the users’ escalating demands. Network operators typically have to overprovision their systems to guarantee acceptable quality of service (QoS) and availability while delivering content. However, more physical devices led to more ICT power consumption over the years. An effective approach to confront these challenges is to jointly optimise the energy consumption of content providers and transportation networks. This thesis proposes a number of energy efficient mechanisms to optimise BitTorrent based P2P networks and clouds based C/S content distribution over IP/WDM based core optical networks. For P2P systems, a mixed integer linear programming (MILP) optimisation, two heuristics and an experimental testbed are developed to minimise the power consumption of IP/WDM networks that deliver traffic generated by an overlay layer of homogeneous BitTorrent users. The approach optimises peers’ selection where the goal is to minimise IP/WDM network power consumption while maximising peers download rate. The results are compared to typical C/S systems. We also considered Heterogeneous BitTorrent peers and developed models that optimise P2P systems to compensate for different peers behaviour after finishing downloading. We investigated the impact of core network physical topology on the energy efficiency of BitTorrent systems. We also investigated the power consumption of Video on Demand (VoD) services using CDN, P2P and hybrid CDN-P2P architectures over IP/WDM networks and addressed content providers efforts to balance the load among their data centres. For cloud systems, a MILP and a heuristic were developed to minimise content delivery induced power consumption of both clouds and IP/WDM networks. This was done by optimally determining the number, location and internal capability in terms of servers, LAN and storage of each cloud, subject to daily traffic variation. Different replication schemes were studied revealing that replicating content into multiple clouds based on content popularity is the optimum approach with respect to energy. The model was extended to study Storage as a Service (StaaS). We also studied the problem of virtual machine placement in IP/WDM networks and showed that VM Slicing is the best approach compared to migration and replication schemes to minimise energy. Finally, we have investigated the utilisation of renewable energy sources represented by solar cells and wind farms in BitTorrent networks and content delivery clouds, respectively. Comprehensive modelling and simulation as well as experimental demonstration were developed, leading to key contributions in the field of energy efficient telecommunications

Actas da 10ª Conferência sobre Redes de Computadores

Universidade do MinhoCCTCCentro AlgoritmiCisco SystemsIEEE Portugal Sectio

Contributions to the solution of the energy efficient file distribution problem

It has been realized that energy; one of the key requirements for modern human civilization, must be used efficiently for the civilization to be sustainable. The Information and Communications Technology (ICI) sector is no exception. It has been shown through research that ICT is consuming energy comparable to the aviation sector and is still increasing rapidly. In order to address this issue, many energy efficient approaches applicable to ICT sector have been proposed in the literature. In this Thesis, we pick one of the most ubiquitous task in ICT, file distribution and concentrate on finding ways of transferring a file from one server to many hosts in the most energy efficient manner. We study the problem for one server and many host problem but our algorithms can be applied to many general scenarios inducing P2P file distri­bution, replication of content in a doud, synchronization of caches in content distribution networks, downloading software updates to millions of PCs worldwide, and many more applications w here the data disseminated does not have to be consumed instantaneous y; for example, in video streaming. We study the problem for one server and many hosts but our algorithms can be applied to more general scenarios inducing P2P file distribution, replication of content in a doud, synchronization of caches in content distribution networks, downloading soft­ware updates to millions of PCs worldwide, We assume that the time is slotted and that the file is transferred in units of data called blocks. Each host can have arbitrary power consumption, upload and download capacities. To begin with, we prove that the prob­lem of energy efficient file distribution is NP-complete. In order to solve the problem optimally; we assume additional constraints and impose that all the hosts involved in the file transfer should have same upload and download capacity. Moreover; we also assume that the upload and download capacities are such that they are integral multi­ples of each other, which is typically the case. Under these conditions, we prove lower bounds on energy and design algorithms for file distribution that achieve the calculated lower bounds. Our algorithms minimize the amount of time a host has to be on to down­load and/ or upload in the distribution process. Apart from being theoretically sound, we also evaluate our model by extending our analysis through extensive numerical evaluation to compare the proposed algorithms with the already existing schemes of transfers. Our algorithms show promising improvement over not just the traditional energy agnostic approaches but also over the schemes designed for energy efficient file distribution. It has been shown that our algorithms are at least 50% more energy efficient than any of the proposals compared with. We advance our numerical analysis to relax the constraints in the theoretical analysis and conclude that our algorithms are also applicable in scenarios in which the comput­ing and networking hardware is energy efficient. Our algorithms can exploit the power proportionality of the devices. No efficiency comes without a cost. In this case, we pay the cost in terms of the tight synchronization that our algorithms require. However, we argue that such a tight syn­chronization at each slot level is possible in today's Internet particularly if the algorithms are applied to the hosts inside a corporation in which all the hosts and network are controlled by a central entity. For example, servers of a cloud, content distribution network, software updates inside a corporation, etc.Programa Oficial de Doctorado en Ingeniería TelemáticaPresidente: Fidel Cacheda Seijo.- Secretario: María Carmen Guerrero López.- Vocal: Guillermo Agustín Ibáñez Fernánde