Incentive-driven QoS in peer-to-peer overlays

A well known problem in peer-to-peer overlays is that no single entity has control over the software, hardware and configuration of peers. Thus, each peer can selfishly adapt its behaviour to maximise its benefit from the overlay. This thesis is concerned with the modelling and design of incentive mechanisms for QoS-overlays: resource allocation protocols that provide strategic peers with participation incentives, while at the same time optimising the performance of the peer-to-peer distribution overlay. The contributions of this thesis are as follows. First, we present PledgeRoute, a novel contribution accounting system that can be used, along with a set of reciprocity policies, as an incentive mechanism to encourage peers to contribute resources even when users are not actively consuming overlay services. This mechanism uses a decentralised credit network, is resilient to sybil attacks, and allows peers to achieve time and space deferred contribution reciprocity. Then, we present a novel, QoS-aware resource allocation model based on Vickrey auctions that uses PledgeRoute as a substrate. It acts as an incentive mechanism by providing efficient overlay construction, while at the same time allocating increasing service quality to those peers that contribute more to the network. The model is then applied to lagsensitive chunk swarming, and some of its properties are explored for different peer delay distributions. When considering QoS overlays deployed over the best-effort Internet, the quality received by a client cannot be adjudicated completely to either its serving peer or the intervening network between them. By drawing parallels between this situation and well-known hidden action situations in microeconomics, we propose a novel scheme to ensure adherence to advertised QoS levels. We then apply it to delay-sensitive chunk distribution overlays and present the optimal contract payments required, along with a method for QoS contract enforcement through reciprocative strategies. We also present a probabilistic model for application-layer delay as a function of the prevailing network conditions. Finally, we address the incentives of managed overlays, and the prediction of their behaviour. We propose two novel models of multihoming managed overlay incentives in which overlays can freely allocate their traffic flows between different ISPs. One is obtained by optimising an overlay utility function with desired properties, while the other is designed for data-driven least-squares fitting of the cross elasticity of demand. This last model is then used to solve for ISP profit maximisation

Dynamic data placement and discovery in wide-area networks

The workloads of online services and applications such as social networks, sensor data platforms and web search engines have become increasingly global and dynamic, setting new challenges to providing users with low latency access to data. To achieve this, these services typically leverage a multi-site wide-area networked infrastructure. Data access latency in such an infrastructure depends on the network paths between users and data, which is determined by the data placement and discovery strategies. Current strategies are static, which offer low latencies upon deployment but worse performance under a dynamic workload. We propose dynamic data placement and discovery strategies for wide-area networked infrastructures, which adapt to the data access workload. We achieve this with data activity correlation (DAC), an application-agnostic approach for determining the correlations between data items based on access pattern similarities. By dynamically clustering data according to DAC, network traffic in clusters is kept local. We utilise DAC as a key component in reducing access latencies for two application scenarios, emphasising different aspects of the problem: The first scenario assumes the fixed placement of data at sites, and thus focusses on data discovery. This is the case for a global sensor discovery platform, which aims to provide low latency discovery of sensor metadata. We present a self-organising hierarchical infrastructure consisting of multiple DAC clusters, maintained with an online and distributed split-and-merge algorithm. This reduces the number of sites visited, and thus latency, during discovery for a variety of workloads. The second scenario focusses on data placement. This is the case for global online services that leverage a multi-data centre deployment to provide users with low latency access to data. We present a geo-dynamic partitioning middleware, which maintains DAC clusters with an online elastic partition algorithm. It supports the geo-aware placement of partitions across data centres according to the workload. This provides globally distributed users with low latency access to data for static and dynamic workloads.Open Acces

Development of a system compliant with the Application-Layer Traffic Optimization Protocol

Dissertação de mestrado integrado em Engenharia InformáticaWith the ever-increasing Internet usage that is following the start of the new decade, the need to optimize this world-scale network of computers becomes a big priority in the technological sphere that has the number of users rising, as are the Quality of Service (QoS) demands by applications in domains such as media streaming or virtual reality. In the face of rising traffic and stricter application demands, a better understand ing of how Internet Service Providers (ISPs) should manage their assets is needed. An important concern regards to how applications utilize the underlying network infras tructure over which they reside. Most of these applications act with little regard for ISP preferences, as exemplified by their lack of care in achieving traffic locality during their operation, which would be a preferable feature for network administrators, and that could also improve application performance. However, even a best-effort attempt by applications to cooperate will hardly succeed if ISP policies aren’t clearly commu nicated to them. Therefore, a system to bridge layer interests has much potential in helping achieve a mutually beneficial scenario. The main focus of this thesis is the Application-Layer Traffic Optimization (ALTO) work ing group, which was formed by the Internet Engineering Task Force (IETF) to explore standardizations for network information retrieval. This group specified a request response protocol where authoritative entities provide resources containing network status information and administrative preferences. Sharing of infrastructural insight is done with the intent of enabling a cooperative environment, between the network overlay and underlay, during application operations, to obtain better infrastructural re sourcefulness and the consequential minimization of the associated operational costs. This work gives an overview of the historical network tussle between applications and service providers, presents the ALTO working group’s project as a solution, im plements an extended system built upon their ideas, and finally verifies the developed system’s efficiency, in a simulation, when compared to classical alternatives.Com o acrescido uso da Internet que acompanha o início da nova década, a necessidade de otimizar esta rede global de computadores passa a ser uma grande prioridade na esfera tecnológica que vê o seu número de utilizadores a aumentar, assim como a exigência, por parte das aplicações, de novos padrões de Qualidade de Serviço (QoS), como visto em domínios de transmissão de conteúdo multimédia em tempo real e em experiências de realidade virtual. Face ao aumento de tráfego e aos padrões de exigência aplicacional mais restritos, é necessário melhor compreender como os fornecedores de serviços Internet (ISPs) devem gerir os seus recursos. Um ponto fulcral é como aplicações utilizam os seus recursos da rede, onde muitas destas não têm consideração pelas preferências dos ISPs, como exemplificado pela sua falta de esforço em localizar tráfego, onde o contrário seria preferível por administradores de rede e teria potencial para melhorar o desempenho aplicacional. Uma tentativa de melhor esforço, por parte das aplicações, em resolver este problema, não será bem-sucedida se as preferências administrativas não forem claramente comunicadas. Portanto, um sistema que sirva de ponte de comunicação entre camadas pode potenciar um cenário mutuamente benéfico. O foco principal desta tese é o grupo de trabalho Application-Layer Traffic Optimization (ALTO), que foi formado pelo Internet Engineering Task Force (IETF) para explorar estandardizações para recolha de informação da rede. Este grupo especificou um protocolo onde entidades autoritárias disponibilizam recursos com informação de estado de rede, e preferências administrativas. A partilha de conhecimento infraestrutural é feita para possibilitar um ambiente cooperativo entre redes overlay e underlay, para uma mais eficiente utilização de recursos e a consequente minimização de custos operacionais. É pretendido dar uma visão da histórica disputa entre aplicações e ISPs, assim como apresentar o projeto do grupo de trabalho ALTO como solução, implementar e melhorar sobre as suas ideias, e finalmente verificar a eficiência do sistema numa simulação, quando comparado com alternativas clássicas

On social and technical aspects of managing mobile Ad-hoc communities

Soziale Software beschreibt eine Klasse von Anwendungen, die es Benutzern erlaubt ueber das Internet mit Freunden zu kommunizieren und Informationen auszutauschen. Mit zunehmender Leistungsfaehigkeit mobiler Prozessoren verwandeln sich Mobiltelefone in vollwertige Computer und eroeffnen neue Moeglichkeiten fuer die mobile Nutzung sozialer Software. Da Menschen Mobiltelefone haeufig bei sich fuehren, koennen vergleichbare mobile Anwendungen staerker auf ihre unmittelbare Umgebungssituation zugeschnitten werden. Moegliche Szenarien sind die Unterstuetzung realer Treffen und damit verbundenen Mitgliederinteraktionen. Client-Server-Plattformen, die dabei haeufig zum Einsatz kommen wurden allerdings nie fuer solche hochflexiblen Gruppensituationen konstruiert. Mobile Encounter Netzwerke (MENe) verprechen hier mehr Flexibilitaet. Ein MEN stellt eine mobiler Peer-to-Peer-Plattformen dar, das ueber ein kurzreichweitiges Funknetz betrieben wird. Mit diesem Netzwerk werden Beitraege ueber einen raeumlichen Diffusionsprozess von einem mobilen Endgeraet zum naechsten verbreitet. Das hat zwei entscheidende Vorteile: Zunaechst ist der direkte Nachrichtenaustausch besser geeignet zur Verbreitung von situationsspezifischer Information, da die Informationsrelevanz mit ihrer Entfehrnung abnimmt. Gleichzeitig koennen aber auch Inhalte, die fuer einen breiten Interessenkreis bestimmt sind ueber Mitglieder mit herausragenden Mobilitaetscharakteristik in weit entfernte Gebiete transportiert werden. Ein Nachteil ist jedoch der hohe Ressourcenverbrauch. Zur Loesung dieses Problems entwickeln wir ein Rahmenwerk zur Unterstuetzung mobiler ad-hoc Gruppen, das es uns erlaubt, Gruppensynergien gezielt auszunutzen. Dieses Rahmenwerk bietet Dienstleistungen zur Verwaltung der Gruppendynamik und zur Verbreitung von Inhalten an. Mittels soziale Netzwerkanalyse wird die technische Infrastruktur ohne notwendige Benutzereingriffe kontinuierlich an die reale Umgebungssituation angepasst. Dabei werden moegliche Beziehungen zwischen benachbarten Personen anhand frueher Begegnungen analysiert, spontane Gruppenbildungen mit Clusterverfahren identifiziert und jedem Gruppenmitglied eine geeignete Rolle durch eine Positionsanalyse zugewiesen. Eine Grundvorraussetzung fuer eine erfolgreiche Kooperation ist ein effizienter Wissensaustausch innerhalb einer Gemeinschaft. Wie die Small World-Theorie zeigt, koennen Menschen Wissen auch dann effizient verbreiten, wenn ihre Entscheidung nur auf lokaler Umgebungsinformation basiert. Verschiedene Forscher machten sich das zu nutze, indem sie kurze Verbreitungspfade durch eine Verkettung hochvernetzter Mitglieder innerhalb einer Gemeinschaft konstruierten. Allerdings laesst sich dieses Verfahren nicht einfach auf MENe uebertragen, da die Transferzeit im Gegensatz zu dem drahtgebundenen Internet beschraenkt ist. Unser Ansatz beruht daher, auf der von Reagan et al. vorgestellten Least Effort Transfer-Hypothese. Diese Hypothese besagt, dass Menschen Wissen nur dann weitergeben, wenn sich der Aufwand zur Informationsuebertragung innerhalb bestimmter Grenzen bewegt. Eine erfolgreiche Wissensuebertragung haengt in diesem Fall vom Hintergrundwissen aller Beteiligter ab, was wiederum von unterschiedlichen kognitiven und sozialen Faktoren abhaengt. Entsprechend leiten wir ein Diffusionsverfahren ab, dass in der Lage ist, Inhalte in verschiedene Kompexitaetstufen einzuteilen und Datenuebertragungen an die vorgefundene soziale Situation anzupassen. Mit einem Prototyp evaluieren wir die Machbarkeit der Gruppen- und Informationsmanagementkomponente unseres Rahmenwerkes. Da Laborexperimente keinen ausreichenden Aufschluss ueber Diffusionseigenschaften im groesseren Massstab geben koennen, simulieren wir die Beitragsdiffusion. Dazu dient uns eine Verkehrsimulation, bei der Agenten zusaetzlich mit aktivitaetsbezogenen, sozialen und territorialen Modellen erweitern werden. Um eine realitaetsnahe Simulation zu gewaehrleisten, werden diese Modelle in Uebereinstimmung mit verschiedenen Studien zum Stadtleben generiert. Der technische Uebertragungsprozess wird anhand der Ergebnisse einer vorangegangenen Prototypuntersuchung parametrisiert. Waehrend eines Simulationslaufes bewegen sich Agenten auf einem Stadtplan und sammeln Kontakt- und Beitragsdaten. Analysiert man anschliessend die Netzwerktopologie auf Small World-Eigenschaften, so findet man eine Netzstruktur mit einer ausgepraegten Neigung zum Clustering (Freundschaftsnetzwerke) und einer ueberdurschnittlichen kurzen Weglaenge. Offensichtlich reicht die Alltagsmobilitaet aus, um ausreichend viele Verknuepfungen zwischen Gemeinschaftmitgliedern zu bilden. Die nachfolgende Diffusionsanalyse zeigt, dass vergleichbare Reichweiten wie bei einem flutungsbasierten Ansatz erzielt werden, allerdings mit anfaenglichen Verzoegerungen. Da unser Verfahren bei einem Ortswechsel die Anzahl der Informationsuebermittler auf zentrale Gruppenmitglieder begrenzt, steht mehr Bandbreite fuer den Datenaustausch zur Verfuegung. Herkoemliche Mitglieder (ohne Leitungsaufgaben) tauschen Inhalte vornehmlich in zeitunkritschen Situationen aus. Das hat den positiven Nebeneffekt, dass im Cache erheblich weniger Kopien aussortiert werden muessen. Wechselt man waehrend der Simulation die Beitragskategorie so erkennt man, dass zeitabhaengige Inhalte besser ueber regelmaessige Kontakte und zeitunabhaengig Inhalte durch zufaellige Kontakte verbreitet werden. Eine abschliessende Precision-Recall Analyse zeigt, dass herkoemmliche Gruppenmitglieder eine bessere Genauigkeit (Precision), und zentrale Mitglieder eine bessere Trefferquote (Recall) im Vergleich zu traditionellen Ansaetzen besitzen. Eine Erklaerung dafuer ist, dass der von uns gewaehlte gruppenbasierte Cacheansatz zu weniger Saeuberungszyklen aller Gruppenmitglieder fuehrt und somit nachhaltiger ausgerichtet ist.Social software encompasses a range of software systems that allow users to interact and share data. This computer-mediated communication has become very popular with social networking sites like Facebook and Twitter. The evolvement of smart phones toward mobile computers opens new possibilities to use social software also in mobile usage scenarios. Since mobile phones are permanently carried by their owners, the support focus is, however, much stronger set on promoting and augmenting real group gatherings. Traditional client-server platforms are not flexible enough to support complex and dynamic human encounter behavior. Mobile encounter networks (MENs) which represent a mobile peer-to-peer platform on top of a short range wireless network promise better flexibility. MENs diffuse content from neighbor-to-neighbor in a spatial diffusion process. For physical group gatherings this is advantageous for two reasons. Direct device-to-device interactions encourage sharing of situation-dependent content. Moreover, content is not necessarily locked within friend groups and may trigger networking effects by reaching larger audiences through user mobility. One disadvantage is, however, the high resource usage. We develop a social software framework for mobile ad-hoc groups, which partly solves this problem. This framework supports services for the management of group dynamics and content diffusion within and between groups. Social network analysis as an inherent part of the framework is used to adapt internal community states continuously with real world encounter situations. We hereby qualify interpersonal relationships based on encounter and communication statistics, identify social groups through incremental clustering and assign diffusion roles through position analysis. To achieve efficient content dissemination we make use of social diffusion phenomena. Other researchers have experimented extensively with the small world model as it proofs that people transfer knowledge based on local knowledge but are still capable of diffusing it efficiently on a global scale. Their approach is often based on identifying short paths through member connectivity. However, this scenario is not applicable in MENs as transfer time is limited in contrast to the wired Internet. Our approach is therefore based on the least effort transfer theory. Following Reagan et al., who first postulated this hypothesis, people transfer knowledge only if the transfer effort is within specific limits, which depends on different social and cognitive factors. We derive routing mechanisms, which are capable of distinguishing between different content complexities and apply information about peer's expertise and social network to identify advantageous paths and content transfers options. We evaluate the feasibility of the group management and content transfer component with prototypes. Since labor settings do not allow to obtain information about large scale diffusion experiences, we also conduct a multi-agent simulation to evaluate the diffusion capabilities of the system. Experiences from an earlier prototype implementation have been used to quantify the technical routing process. To emulate realistic community life, we assigned to each agent an individual daily agenda, social contacts and territory preferences specified according to outcomes from different urban city life surveys. During the simulation agents move on a city map according to these models and collect contact and content specific data. Analyzing the network topology according to small world characteristics shows a structure with a high tendency for clustering (friend networks) and a short average path length. Daily urban mobility creates enough opportunities to form shortcuts through the community. Content diffusion analysis shows that our approach reaches a similar amount of peers as network flooding but with delays in the beginning. Since our approach artificially limits the number of intermediates to central community peers more bandwidth is available during traveling and more content can be transferred as in the case of the flooding approach. Ordinary peers seem to have significantly fewer content replications if an unlimited cache is assumed proofing that our mechanism is more efficient. By varying the content type used during the simulation we recognize that time dependent content is better disseminated through frequent contacts and time independent content through random contacts. Performing a precision-recall analysis on peers caches shows that ordinary peers gain an overall better context precision, and central peers a better community recall. One explanation is that the shared cache approach leads to fewer content replacements in the cache as for instance the least recently used cache strategy

Flexible Application-Layer Multicast in Heterogeneous Networks

This work develops a set of peer-to-peer-based protocols and extensions in order to provide Internet-wide group communication. The focus is put to the question how different access technologies can be integrated in order to face the growing traffic load problem. Thereby, protocols are developed that allow autonomous adaptation to the current network situation on the one hand and the integration of WiFi domains where applicable on the other hand

Routing, Localization And Positioning Protocols For Wireless Sensor And Actor Networks

Wireless sensor and actor networks (WSANs) are distributed systems of sensor nodes and actors that are interconnected over the wireless medium. Sensor nodes collect information about the physical world and transmit the data to actors by using one-hop or multi-hop communications. Actors collect information from the sensor nodes, process the information, take decisions and react to the events. This dissertation presents contributions to the methods of routing, localization and positioning in WSANs for practical applications. We first propose a routing protocol with service differentiation for WSANs with stationary nodes. In this setting, we also adapt a sports ranking algorithm to dynamically prioritize the events in the environment depending on the collected data. We extend this routing protocol for an application, in which sensor nodes float in a river to gather observations and actors are deployed at accessible points on the coastline. We develop a method with locally acting adaptive overlay network formation to organize the network with actor areas and to collect data by using locality-preserving communication. We also present a multi-hop localization approach for enriching the information collected from the river with the estimated locations of mobile sensor nodes without using positioning adapters. As an extension to this application, we model the movements of sensor nodes by a subsurface meandering current mobility model with random surface motion. Then we adapt the introduced routing and network organization methods to model a complete primate monitoring system. A novel spatial cut-off preferential attachment model and iii center of mass concept are developed according to the characteristics of the primate groups. We also present a role determination algorithm for primates, which uses the collection of spatial-temporal relationships. We apply a similar approach to human social networks to tackle the problem of automatic generation and organization of social networks by analyzing and assessing interaction data. The introduced routing and localization protocols in this dissertation are also extended with a novel three dimensional actor positioning strategy inspired by the molecular geometry. Extensive simulations are conducted in OPNET simulation tool for the performance evaluation of the proposed protocol

Building Blocks for Mapping Services

Mapping services are ubiquitous on the Internet. These services enjoy a considerable user base. But it is often overlooked that providing a service on a global scale with virtually millions of users has been the playground of an oligopoly of a select few service providers are able to do so. Unfortunately, the literature on these solutions is more than scarce. This thesis adds a number of building blocks to the literature that explain how to design and implement a number of features