Optimising Structured P2P Networks for Complex Queries

With network enabled consumer devices becoming increasingly popular, the number of connected devices and available services is growing considerably - with the number of connected devices es- timated to surpass 15 billion devices by 2015. In this increasingly large and dynamic environment it is important that users have a comprehensive, yet efficient, mechanism to discover services. Many existing wide-area service discovery mechanisms are centralised and do not scale to large numbers of users. Additionally, centralised services suffer from issues such as a single point of failure, high maintenance costs, and difficulty of management. As such, this Thesis seeks a Peer to Peer (P2P) approach. Distributed Hash Tables (DHTs) are well known for their high scalability, financially low barrier of entry, and ability to self manage. They can be used to provide not just a platform on which peers can offer and consume services, but also as a means for users to discover such services. Traditionally DHTs provide a distributed key-value store, with no search functionality. In recent years many P2P systems have been proposed providing support for a sub-set of complex query types, such as keyword search, range queries, and semantic search. This Thesis presents a novel algorithm for performing any type of complex query, from keyword search, to complex regular expressions, to full-text search, over any structured P2P overlay. This is achieved by efficiently broadcasting the search query, allowing each peer to process the query locally, and then efficiently routing responses back to the originating peer. Through experimentation, this technique is shown to be successful when the network is stable, however performance degrades under high levels of network churn. To address the issue of network churn, this Thesis proposes a number of enhancements which can be made to existing P2P overlays in order to improve the performance of both the existing DHT and the proposed algorithm. Through two case studies these enhancements are shown to improve not only the performance of the proposed algorithm under churn, but also the performance of traditional lookup operations in these networks

Flexible Application-Layer Multicast in Heterogeneous Networks

This work develops a set of peer-to-peer-based protocols and extensions in order to provide Internet-wide group communication. The focus is put to the question how different access technologies can be integrated in order to face the growing traffic load problem. Thereby, protocols are developed that allow autonomous adaptation to the current network situation on the one hand and the integration of WiFi domains where applicable on the other hand

Adaptation and Robustness in Peer-to-Peer Streaming

The rapid development of network communication infrastructure enables networked multimedia streaming applications ranging from on-demand video streaming to highly interactive video conferencing. Peer-to-Peer (P2P) technologies have emerged as a powerful and popular paradigm for bringing such emerging multimedia services to a large number of users. The essential advantage of P2P systems is that the system capacity scales up when more peers join, as peer upload capacity is utilized. However, providing satisfactory streaming services over P2P networks is challenging because of their inherent instability and unreliability and the limited adaptability of traditional video coding techniques. On one hand, different from dedicated servers, users may not have enough bandwidth to serve other users as most user connections are asymmetric in their upload and download capacity, and they are heterogeneous in terms of bandwidth and preferences. In addition, users can join and leave the system at any time as there are no guarantees on their contribution to the system. On the other hand, although traditional video coding techniques are efficient in terms of resource consumption, compression ratio, and coding and decoding speed, they do not support scalable modes efficiently as such modes come along with high computation cost. Consequently, in traditional P2P streaming systems, the bit rate (the video quality) of media streams is determined based on the capacities of the low-end users, i.e. the lowest common denominator, to make sure that most of their users can perceive acceptable quality. This causes two critical limitations of the current P2P streaming systems. First, users perceive the same quality regardless of their bandwidth capacity, i.e., no differentiated QoS. Second, with the current best-effort Internet and peer dynamics, the streaming quality at each peer is easily impaired, i.e., no continuous playback. Recently, multiple layer codec research has become more refined, as SVC (the scalable extension of the H.264/AVC standard) has been standardized with a bit rate overhead of around 10% and an indistinguishable visual quality compared to the state of the art single layer codec. The hypothesis of this research work is that the adaptable coding technique can bring significant benefits to P2P streaming as it enables adaptability in P2P streaming. In addition, to improve the robustness of the system to network fluctuations and peer dynamics, network coding and social networking are also applied. The overall goal of this research is to achieve adaptive and robust P2P streaming services, which are believed to be the next generation of P2P streaming on the Internet. Several major contributions are presented in this dissertation. First, to use SVC in P2P streaming, a segmentation method to segment SVC streams into scalable units is proposed such that they can be delivered adaptively by the P2P paradigm. The method is demonstrated to be able to preserve the scalability features of a stream, i.e., adaptation can be applied on segments and the re-generated stream at each peer is a valid stream. Second, a novel and complete adaptive P2P streaming protocol, named Chameleon, is presented. Chameleon uses the segmentation method to use SVC and combine it with network coding in P2P streaming to achieve high performance streaming. The core of Chameleon is studied, including neighbor selection, quality adaptation, receiver-driven peer coordination, and sender selection, with different design options. Experiments on Chameleon reveal that overlay construction is important to system performance, and traditional gossip-based protocols are not good enough for layered P2P streaming. Therefore, third, a SCAMP-based neighbor selection protocol and a peer sampling-based membership management protocol for layered P2P streaming are proposed. These gossip-based protocols are quality- and context-aware as they form robust and adaptable overlays for layered P2P streaming so that high capacity peers have a higher priority to be located at good positions in the overlay, e.g. closer to the server, and peers with similar capacity are connected to each other to better utilize resources. Fourth, to better deal with peer dynamics, Stir, a social-based P2P streaming system, is suggested. In Stir, the novel idea of spontaneous social networking is introduced. Stir users who join the same streaming session can make friends and communicate with each other by cheap yet efficient communication means, e.g., instant messaging and Twitter-like commenting. Such friendship networks are exploited directly by the underlying social-based P2P streaming protocol. The tight integration between the high level social networking of users and the low level overlay of peers is demonstrated to be beneficial in dealing with high churn rates and providing personalized streaming services. Finally, as the approaches are about different aspects of adaptive and robust P2P streaming, to complete the picture, Chameleon++, which combines Chameleon and Stir, is presented. The design and the evaluation of Chameleon++ demonstrate the feasibility and the benefits of the approaches, and the consistency of the study

Scalable discovery of networked data : Algorithms, Infrastructure, Applications

Harmelen, F.A.H. van [Promotor]Siebes, R.M. [Copromotor

Hybrid routing in delay tolerant networks

This work addresses the integration of today\\u27s infrastructure-based networks with infrastructure-less networks. The resulting Hybrid Routing System allows for communication over both network types and can help to overcome cost, communication, and overload problems. Mobility aspect resulting from infrastructure-less networks are analyzed and analytical models developed. For development and deployment of the Hybrid Routing System an overlay-based framework is presented

Hybrid Routing in Delay Tolerant Networks

This work addresses the integration of today\u27s infrastructure-based networks with infrastructure-less networks. The resulting Hybrid Routing System allows for communication over both network types and can help to overcome cost, communication, and overload problems. Mobility aspect resulting from infrastructure-less networks are analyzed and analytical models developed. For development and deployment of the Hybrid Routing System an overlay-based framework is presented

Distributed resource discovery: architectures and applications in mobile networks

As the amount of digital information and services increases, it becomes increasingly important to be able to locate the desired content. The purpose of a resource discovery system is to allow available resources (information or services) to be located using a user-defined search criterion. This work studies distributed resource discovery systems that guarantee all existing resources to be found and allow a wide range of complex queries. Our goal is to allocate the load uniformly between the participating nodes, or alternatively to concentrate the load in the nodes with the highest available capacity. The first part of the work examines the performance of various existing unstructured architectures and proposes new architectures that provide features especially valuable in mobile networks. To reduce the network traffic, we use indexing, which is particularly useful in scenarios, where searches are frequent compared to resource modifications. The ratio between the search and update frequencies determines the optimal level of indexing. Based on this observation, we develop an architecture that adjusts itself to changing network conditions and search behavior while maintaining optimal indexing. We also propose an architecture based on large-scale indexing that we later apply to resource sharing within a user group. Furthermore, we propose an architecture that relieves the topology constraints of the Parallel Index Clustering architecture. The performance of the architectures is evaluated using simulation. In the second part of the work we apply the architectures to two types of mobile networks: cellular networks and ad hoc networks. In the cellular network, we first consider scenarios where multiple commercial operators provide a resource sharing service, and then a scenario where the users share resources without operator support. We evaluate the feasibility of the mobile peer-to-peer concept using user opinion surveys and technical performance studies. Based on user input we develop access control and group management algorithms for peer-to-peer networks. The technical evaluation is performed using prototype implementations. In particular, we examine whether the Session Initiation Protocol can be used for signaling in peer-to-peer networks. Finally, we study resource discovery in an ad hoc network. We observe that in an ad hoc network consisting of consumer devices, the capacity and mobility among nodes vary widely. We utilize this property in order to allocate the load to the high-capacity nodes, which serve lower-capacity nodes. We propose two methods for constructing a virtual backbone connecting the nodes

Dynamic data placement and discovery in wide-area networks

The workloads of online services and applications such as social networks, sensor data platforms and web search engines have become increasingly global and dynamic, setting new challenges to providing users with low latency access to data. To achieve this, these services typically leverage a multi-site wide-area networked infrastructure. Data access latency in such an infrastructure depends on the network paths between users and data, which is determined by the data placement and discovery strategies. Current strategies are static, which offer low latencies upon deployment but worse performance under a dynamic workload. We propose dynamic data placement and discovery strategies for wide-area networked infrastructures, which adapt to the data access workload. We achieve this with data activity correlation (DAC), an application-agnostic approach for determining the correlations between data items based on access pattern similarities. By dynamically clustering data according to DAC, network traffic in clusters is kept local. We utilise DAC as a key component in reducing access latencies for two application scenarios, emphasising different aspects of the problem: The first scenario assumes the fixed placement of data at sites, and thus focusses on data discovery. This is the case for a global sensor discovery platform, which aims to provide low latency discovery of sensor metadata. We present a self-organising hierarchical infrastructure consisting of multiple DAC clusters, maintained with an online and distributed split-and-merge algorithm. This reduces the number of sites visited, and thus latency, during discovery for a variety of workloads. The second scenario focusses on data placement. This is the case for global online services that leverage a multi-data centre deployment to provide users with low latency access to data. We present a geo-dynamic partitioning middleware, which maintains DAC clusters with an online elastic partition algorithm. It supports the geo-aware placement of partitions across data centres according to the workload. This provides globally distributed users with low latency access to data for static and dynamic workloads.Open Acces

Approximate algorithms for efficient indexing, clustering, and classification in Peer-to-peer networks

[no abstract

An Overlay Architecture for Personalized Object Access and Sharing in a Peer-to-Peer Environment

Due to its exponential growth and decentralized nature, the Internet has evolved into a chaotic repository, making it difficult for users to discover and access resources of interest to them. As a result, users have to deal with the problem of information overload. The Semantic Web's emergence provides Internet users with the ability to associate explicit, self-described semantics with resources. This ability will facilitate in turn the development of ontology-based resource discovery tools to help users retrieve information in an efficient manner. However, it is widely believed that the Semantic Web of the future will be a complex web of smaller ontologies, mostly created by various groups of web users who share a similar interest, referred to as a Community of Interest. This thesis proposes a solution to the information overload problem using a user driven framework, referred to as a Personalized Web, that allows individual users to organize themselves into Communities of Interests based on ontologies agreed upon by all community members. Within this framework, users can define and augment their personalized views of the Internet by associating specific properties and attributes to resources and defining constraint-functions and rules that govern the interpretation of the semantics associated with the resources. Such views can then be used to capture the user's interests and integrate these views into a user-defined Personalized Web. As a proof of concept, a Personalized Web architecture that employs ontology-based semantics and a structured Peer-to-Peer overlay network to provide a foundation of semantically-based resource indexing and advertising is developed. In order to investigate mechanisms that support the resource advertising and retrieval of the Personalized Web architecture, three agent-driven advertising and retrieval schemes, the Aggressive scheme, the Crawler-based scheme, and the Minimum-Cover-Rule scheme, were implemented and evaluated in both stable and churn environments. In addition to the development of a Personalized Web architecture that deals with typical web resources, this thesis used a case study to explore the potential of the Personalized Web architecture to support future web service workflow applications. The results of this investigation demonstrated that the architecture can support the automation of service discovery, negotiation, and invocation, allowing service consumers to actualize a personalized web service workflow. Further investigation will be required to improve the performance of the automation and allow it to be performed in a secure and robust manner. In order to support the next generation Internet, further exploration will be needed for the development of a Personalized Web that includes ubiquitous and pervasive resources