Statistical structures for internet-scale data management

Efficient query processing in traditional database management systems relies on statistics on base data. For centralized systems, there is a rich body of research results on such statistics, from simple aggregates to more elaborate synopses such as sketches and histograms. For Internet-scale distributed systems, on the other hand, statistics management still poses major challenges. With the work in this paper we aim to endow peer-to-peer data management over structured overlays with the power associated with such statistical information, with emphasis on meeting the scalability challenge. To this end, we first contribute efficient, accurate, and decentralized algorithms that can compute key aggregates such as Count, CountDistinct, Sum, and Average. We show how to construct several types of histograms, such as simple Equi-Width, Average-Shifted Equi-Width, and Equi-Depth histograms. We present a full-fledged open-source implementation of these tools for distributed statistical synopses, and report on a comprehensive experimental performance evaluation, evaluating our contributions in terms of efficiency, accuracy, and scalability

Use of latent semantic indexing for content based searching and routing of mobile agents on P2P network

The peer-to-peer (P2P) system has a number of nodes that are connected to each other in an unstructured or a structured overlay network. One of the most important problems in a P2P system is locating of resources that are shared by various nodes. Techniques such as Flooding and Distributed Hash-Table (DHT) have been proposed to locate resources shared by various nodes. Flooding suffers from saturation as number of nodes increase, while DHT cannot handle multiple keys to define and search a resource. Various further research works including multi agent systems (MAS) have been pursued that take unstructured or structured networks as a backbone and hence inherently suffer from problems. We present the solution that is more efficient and effective for discovering shared resources on a network that is influenced by content shared by nodes. Our solution presents use of multiple agents that manage the shared information on a node and a mobile agent called Reconnaissance Agent (RA) that is responsible for querying various nodes. To reduce the search load on nodes that have unrelated content, an efficient migration route is proposed for RA that is based on cosine similarity of content shared by nodes and user query. Results show reduction in search load and traffic due to communication, and increase in recall value for locating of resources defined by multiple keys using RA that are logically similar to user query. Furthermore, the results indicate that by use of our technique the relevance of search results is higher; that is obtained by minimal traffic generation/communication and hops made by RA

Supporting Complex Queries in P2P Networks

Ph.DDOCTOR OF PHILOSOPH

A HOLISTIC REDUNDANCY- AND INCENTIVE-BASED FRAMEWORK TO IMPROVE CONTENT AVAILABILITY IN PEER-TO-PEER NETWORKS

Peer-to-Peer (P2P) technology has emerged as an important alternative to the traditional client-server communication paradigm to build large-scale distributed systems. P2P enables the creation, dissemination and access to information at low cost and without the need of dedicated coordinating entities. However, existing P2P systems fail to provide high-levels of content availability, which limit their applicability and adoption. This dissertation takes a holistic approach to device mechanisms to improve content availability in large-scale P2P systems. Content availability in P2P can be impacted by hardware failures and churn. Hardware failures, in the form of disk or node failures, render information inaccessible. Churn, an inherent property of P2P, is the collective effect of the users’ uncoordinated behavior, which occurs when a large percentage of nodes join and leave frequently. Such a behavior reduces content availability significantly. Mitigating the combined effect of hardware failures and churn on content availability in P2P requires new and innovative solutions that go beyond those applied in existing distributed systems. To addresses this challenge, the thesis proposes two complementary, low cost mechanisms, whereby nodes self-organize to overcome failures and improve content availability. The first mechanism is a low complexity and highly flexible hybrid redundancy scheme, referred to as Proactive Repair (PR). The second mechanism is an incentive-based scheme that promotes cooperation and enforces fair exchange of resources among peers. These mechanisms provide the basis for the development of distributed self-organizing algorithms to automate PR and, through incentives, maximize their effectiveness in realistic P2P environments. Our proposed solution is evaluated using a combination of analytical and experimental methods. The analytical models are developed to determine the availability and repair cost properties of PR. The results indicate that PR’s repair cost outperforms other redundancy schemes. The experimental analysis was carried out using simulation and the development of a testbed. The simulation results confirm that PR improves content availability in P2P. The proposed mechanisms are implemented and tested using a DHT-based P2P application environment. The experimental results indicate that the incentive-based mechanism can promote fair exchange of resources and limits the impact of uncooperative behaviors such as “free-riding”

Video-on-Demand over Internet: a survey of existing systems and solutions

Video-on-Demand is a service where movies are delivered to distributed users with low delay and free interactivity. The traditional client/server architecture experiences scalability issues to provide video streaming services, so there have been many proposals of systems, mostly based on a peer-to-peer or on a hybrid server/peer-to-peer solution, to solve this issue. This work presents a survey of the currently existing or proposed systems and solutions, based upon a subset of representative systems, and defines selection criteria allowing to classify these systems. These criteria are based on common questions such as, for example, is it video-on-demand or live streaming, is the architecture based on content delivery network, peer-to-peer or both, is the delivery overlay tree-based or mesh-based, is the system push-based or pull-based, single-stream or multi-streams, does it use data coding, and how do the clients choose their peers. Representative systems are briefly described to give a summarized overview of the proposed solutions, and four ones are analyzed in details. Finally, it is attempted to evaluate the most promising solutions for future experiments. Résumé La vidéo à la demande est un service où des films sont fournis à distance aux utilisateurs avec u

Prospects of peer-to-peer SIP for mobile operators

Tämän diplomityön tarkoituksena on esitellä kehitteillä oleva Peer-to-Peer Session Initiation Protocol (P2PSIP), jonka avulla käyttäjät voivat itsenäisesti ja helposti luoda keskenään puhe- ja muita multimediayhteyksiä vertaisverkko-tekniikan avulla. Lisäksi tarkoituksena on arvioida P2PSIP protokollan vaikutuksia ja mahdollisuuksia mobiilioperaattoreille, joille sitä voidaan pitää uhkana. Tästä huolimatta, P2PSIP:n ei ole kuitenkaan tarkoitus korvata nykyisiä puhelinverkkoja. Työn alussa esittelemme SIP:n ja vertaisverkkojen (Peer-to-Peer) periaatteet, joihin P2PSIP-protokollan on suunniteltu perustuvan. SIP mahdollistaa multimedia-istuntojen luomisen, sulkemisen ja muokkaamisen verkossa, mutta sen monipuolinen käyttö vaatii keskitettyjen palvelimien käyttöä. Vertaisverkon avulla käyttäjät voivat suorittaa keskitettyjen palvelimien tehtävät keskenään hajautetusti. Tällöin voidaan ylläpitää laajojakin verkkoja tehokkaasti ilman palvelimista aiheutuvia ylläpito-kustannuksia. Mobiilioperaattorit ovat haasteellisen tilanteen edessä, koska teleliikennemaailma on muuttumassa yhä avoimemmaksi. Tällöin operaattoreiden asiakkaille aukeaa mahdollisuuksia käyttää kilpailevia Internet-palveluja (kuten Skype) helpommin ja tulevaisuudessa myös itse muodostamaan kommunikointiverkkoja P2PSIP:n avulla. Tutkimukset osoittavat, että näistä uhista huolimatta myös operaattorit pystyvät näkemään P2PSIP:n mahdollisuutena mukautumisessa nopeasti muuttuvan teleliikennemaailman haasteisiin. Nämä mahdollisuudet sisältävät operaattorin oman verkon optimoinnin lisäksi vaihtoehtoisten ja monipuolisempien palveluiden tarjoamisen asiakkailleen edullisesti. Täytyy kuitenkin muistaa, että näiden mahdollisuuksien toteuttamisten vaikutusten ei tulisi olla ristiriidassa operaattorin muiden palveluiden kanssa. Lisäksi tulisi muistaa, että tällä hetkellä keskeneräisen P2PSIP-standardin lopullinen luonne ja ominaisuudet voivat muuttaa sen vaikutuksia.The purpose of this thesis is to present the Peer-to-Peer Session Initiation Protocol (P2PSIP) being developed. In addition, the purpose of this thesis is to evaluate the impacts and prospects of P2PSIP to mobile operators, to whom it can be regarded as a threat. In P2PSIP, users can independently and easily establish voice and other multimedia connections using peer-to-peer (P2P) networking. However, P2PSIP is not meant to replace the existing telephony networks of the operators. We start by introducing the principles of SIP and P2P networking that the P2PSIP is intended to use. SIP enables to establish, terminate and modify multimedia sessions, but its versatile exploitation requires using centralized servers. By using P2P networking, users can decentralize the functions of centralized servers by performing them among themselves. This enables to maintain large and robust networks without maintenance costs resulted of running such centralized servers. Telecommunications market is transforming to a more open environment, where mobile operators and other service providers are challenged to adapt to the upcoming changes. Subscribers have easier access to rivalling Internet-services (such as Skype) and in future they can form their own communication communities by using P2PSIP. The results show that despite of these threats, telecom operators can find potential from P2PSIP in concurrence in adaptation to the challenges of the rapidly changing telecom environment. These potential roles include optimization of the network of the operator, but as well roles to provide alternative and more versatile services to their subscribers at low cost. However, the usage of P2PSIP should not conflict with the other services of the operator. Also, as P2PSIP is still under development, its final nature and features may change its impacts and prospects

Naming and discovery in networks : architecture and economics

In less than three decades, the Internet was transformed from a research network available to the academic community into an international communication infrastructure. Despite its tremendous success, there is a growing consensus in the research community that the Internet has architectural limitations that need to be addressed in a effort to design a future Internet. Among the main technical limitations are the lack of mobility support, and the lack of security and trust. The Internet, and particularly TCP/IP, identifies endpoints using a location/routing identifier, the IP address. Coupling the endpoint identifier to the location identifier hinders mobility and poorly identifies the actual endpoint. On the other hand, the lack of security has been attributed to limitations in both the network and the endpoint. Authentication for example is one of the main concerns in the architecture and is hard to implement partly due to lack of identity support. The general problem that this dissertation is concerned with is that of designing a future Internet. Towards this end, we focus on two specific sub-problems. The first problem is the lack of a framework for thinking about architectures and their design implications. It was obvious after surveying the literature that the majority of the architectural work remains idiosyncratic and descriptions of network architectures are mostly idiomatic. This has led to the overloading of architectural terms, and to the emergence of a large body of network architecture proposals with no clear understanding of their cross similarities, compatibility points, their unique properties, and architectural performance and soundness. On the other hand, the second problem concerns the limitations of traditional naming and discovery schemes in terms of service differentiation and economic incentives. One of the recurring themes in the community is the need to separate an entity\u27s identifier from its locator to enhance mobility and security. Separation of identifier and locator is a widely accepted design principle for a future Internet. Separation however requires a process to translate from the identifier to the locator when discovering a network path to some identified entity. We refer to this process as identifier-based discovery, or simply discovery, and we recognize two limitations that are inherent in the design of traditional discovery schemes. The first limitation is the homogeneity of the service where all entities are assumed to have the same discovery performance requirements. The second limitation is the inherent incentive mismatch as it relates to sharing the cost of discovery. This dissertation addresses both subproblems, the architectural framework as well as the naming and discovery limitations

Hybrid Multicasting Using Automatic Multicast Tunnels (AMT)

Native Multicast plays an important role in distributing and managing delivery of some of the most popular Internet applications, such as IPTV and media delivery. However, due to patchy support and the existence of multiple approaches for Native Multicast, the support for Native Multicast is fragmented into isolated areas termed Multicast Islands. This renders Native Multicast unfit to be used as an Internet wide application. Instead, Application Layer Multicast, which does not have such network requirements but is more expensive in terms of bandwidth and overhead, can be used to connect the native multicast islands. This thesis proposes Opportunistic Native Multicast (ONM) which employs Application LayerMulticast (ALM), on top of a DHT-based P2P overlay network, and AutomaticMulticast Tunnelling (AMT) to connect these islands. ALM will be used for discovery and initiating the AMT tunnels. The tunnels will encapsulate the traffic going between islands' Primary Nodes (PNs). AMT was used for its added benefits such as security and being better at traffic shaping and Quality Of Service (QoS). While different approaches for connecting multicast islands exists, the system proposed in the thesis was designed with the following characteristics in mind: scalability, availability, interoperability, self-adaptation and efficiency. Importantly, by utilising AMT tunnels, this approach has unique properties that improve network security and management