Effective bootstrapping of Peer-to Peer networks over Mobile Ad-hoc networks

Mobile Ad-hoc Networks (MANETs) and Peer-to-Peer (P2P) networks are vigorous, revolutionary communication technologies in the 21st century. They lead the trend of decentralization. Decentralization will ultimately win clients over client/server model, because it gives ordinary network users more control, and stimulates their active participation. It is a determinant factor in shaping the future of networking. MANETs and P2P networks are very similar in nature. Both are dynamic, distributed. Both use multi-hop broadcast or multicast as major pattern of traffic. Both set up connection by self-organizing and maintain connection by self-healing. Embodying the slogan networking without networks, both abandoned traditional client/server model and disclaimed pre-existing infrastructure. However, their status quo levels of real world application are widely divergent. P2P networks are now accountable for about 50 ~ 70% internet traffic, while MANETs are still primarily in the laboratory. The interesting and confusing phenomenon has sparked considerable research effort to transplant successful approaches from P2P networks into MANETs. While most research in the synergy of P2P networks and MANETs focuses on routing, the network bootstrapping problem remains indispensable for any such transplantation to be realized. The most pivotal problems in bootstrapping are: (1) automatic configuration of nodes addresses and IDs, (2) topology discovery and transformation in different layers and name spaces. In this dissertation research, we have found novel solutions for these problems. The contributions of this dissertation are: (1) a non-IP, flat address automatic configuration scheme, which integrates lower layer addresses and P2P IDs in application layer and makes simple cryptographical assignment possible. A related paper entitled Pastry over Ad-Hoc Networks with Automatic Flat Address Configuration was submitted to Elsevier Journal of Ad Hoc Networks in May. (2) an effective ring topology construction algorithm which builds perfect ring in P2P ID space using only simplest multi-hop unicast or multicast. Upon this ring, popular structured P2P networks like Chord, Pastry could be built with great ease. A related paper entitled Chord Bootstrapping on MANETs - All Roads lead to Rome will be ready for submission after defense of the dissertation

Integrating Wireless Sensor Networks and Mobile Ad-hoc NETworks for enhanced value-added services

In some situations where the standard telecommunication infrastructure is not available, Mobile Ad hoc NETworks (MANETs) can be deployed to provide the required communication. These networks are established "on the fly" without a need for prior communication organization and are composed of autonomous mobile devices, such as cell phones, PDAs or laptops. In similar conditions, such as in emergency response operations, integrating MANETs and Wireless Sensor Networks (WSNs) can notably enhance the MANET participant's end-user experience. WSNs sense and aggregate ambient information, such as physiological, environmental or physical data related to a nearby phenomenon. The integration, which provides end-user availability to WSN required information, is feasible via gateways. However, when the ambient information collected by WSNs is intended for applications residing in MANETs, centralized and fixed gateways are not practicably feasible. This is mainly due to ad-hoc nature, lack of centralized control and constraints on the end-user devices that are used in MANETs. These devices are usually limited in power and capacity and cannot host centralized gateways. In this thesis we exploit the integration of WSN and MANET in order to provide novel value-added services which enhance the end-user experience of MANET participants. Motivating scenarios are introduced, background information is presented, requirements are derived and the state of the art regarding the integration of WSN with existing networks, including MANETs, is evaluated. Based on the evaluation, none of the existing solutions satisfies all of our derived requirements. Therefore, we propose an overall two-level overlay architecture to integrate WSNs (with mobile sinks) and MANETs. This architecture is based on the distributed gateway and applications which form the P2P overlays. Overlays are application-layer networks which are created on top of the exiting MANET. To interconnect gateway and application overlays we derive corresponding requirements and evaluate the existing approaches. Since none of these approaches fulfills all of our requirements, we propose protocols, mechanisms and design corresponding modules for the interconnection of overlays. Finally we refine our overall architecture based on the interconnection aspects. As a proof of concept, we implement a prototype for the inter-overlay information exchange. This implementation is based on SIP extensions and uses two existing P2P middlewares. We also simulate our prototype using Oversim simulation tool and collect experimental results. Based on these results, we can see that our architecture is a valid and promising approach for interconnecting different P2P overlays and can be deployed to provide the overall solution for WSN and MANET integrated system

Decentralized data fusion and data harvesting framework for heterogeneous dynamic network systems

Diese Dissertation behandelt das Thema der dezentralisieren Sammlung und Fusion von Daten in heterogenen, ressourcenbeschraekten und dynamischen Netzwerkszenarien. Dazu wird ein generisches Framework vorgestellt, dass es erlaubt die Datensammlung, den Datenaustausch und auch die Datenfusion dynamisch zu konfigurieren. Im Zuge dessen wird auch eine Methode zur gerichteten Fusion von Daten auf graphentheoretischer Basis eingefrt, die es erlaubt eine logische Struktur fuer die Fusion von Informationen zu modellieren. Eine Markup-Sprache, die sowohl menschen- als auch maschinenlesbar ist, erlaubt es diese Struktur leicht zu editieren. Im Bereich der Protokolle zum Datenaustausch liegt der Fokus dieser Arbeit auf Energieeffizienz, um auch ressourcenbeschraenkte Geraete einzubinden. Ein weiterer Schwerpunkt liegt auf Robustheit fuer die betrachteten dynamischen Szenarien. Diese Dissertation schlaet zudem Design-Richtlinien vor, um verschiedene Ziele fuer unterschiedliche Applikationen umzusetzen. Diese lassen sich leicht in das vorgestellte Framework integrieren und darueber konfigurieren. Dadurch ergibt sich im Ganzen eine flexible Architektur, die sich leicht an dynamische Umgebungen anpassen laesst.With the increasing number of available smart phones, sensor nodes, and novel mobile smart devices such as Google glass, a large volume of data reflecting the environment is generated in the form of sensing data sources (such as GPS, received signal strength identification, accelerometer, microphone, images, videos and gyroscope, etc.). Some context-aware and data centric applications require the online processing of the data collected. The thesis researches on the decentralized data fusion and data harvesting framework for heterogeneous dynamic network system consisting of various devices with resource constraints. In order to achieve the flexible design, a general architecture is provided while the detailed data fusion and data exchange functions can be dynamically configured. A novel method to use directed fusion graph to model the logical structure of the distributed information fusion architecture is introduced. This directed fusion graph can accurately portray the interconnection among different data fusion components and the data exchange protocols, as well as the detailed data streams. The directed fusion graph is then transformed into a format with marked language, so that both human and machine can easily understand and edit. In the field of data exchange protocols, this thesis targets energy-efficiency considering the resource constraints of the devices and robustness, as the dynamic environment might cause failures to the system. It proposes a refined gossip strategy to reduce retransmission of redundant data. The thesis also suggests a design guideline to achieve different design aims for different applications. These results in this field can be integrated into the framework effortlessly. The configuration mechanism is another feature of this framework. Different from other research work which consider configuration as a post-design work separated from the main design of any middle-ware. This thesis considers the configuration part as another dimension of the framework. The whole strategy in configuration sets up the foundation for the flexible architecture, and makes it easy to adapt to the dynamic environment. The contributions in the above fields lead to a light-weight data fusion and data harvesting framework which can be deployed easily above wireless based, heterogeneous, dynamic network systems, even in extreme conditions, to handle data-centric applications

Robust P2P Live Streaming

Projecte fet en col.laboració amb la Fundació i2CATThe provisioning of robust real-time communication services (voice, video, etc.) or media contents through the Internet in a distributed manner is an important challenge, which will strongly influence in current and future Internet evolution. Aware of this, we are developing a project named Trilogy leaded by the i2CAT Foundation, which has as main pillar the study, development and evaluation of Peer-to-Peer (P2P) Live streaming architectures for the distribution of high-quality media contents. In this context, this work concretely covers media coding aspects and proposes the use of Multiple Description Coding (MDC) as a flexible solution for providing robust and scalable live streaming over P2P networks. This work describes current state of the art in media coding techniques and P2P streaming architectures, presents the implemented prototype as well as its simulation and validation results

Conceptual Models for Assessment & Assurance of Dependability, Security and Privacy in the Eternal CONNECTed World

This is the first deliverable of WP5, which covers Conceptual Models for Assessment & Assurance of Dependability, Security and Privacy in the Eternal CONNECTed World. As described in the project DOW, in this document we cover the following topics: • Metrics definition • Identification of limitations of current V&V approaches and exploration of extensions/refinements/ new developments • Identification of security, privacy and trust models WP5 focus is on dependability concerning the peculiar aspects of the project, i.e., the threats deriving from on-the-fly synthesis of CONNECTors. We explore appropriate means for assessing/guaranteeing that the CONNECTed System yields acceptable levels for non-functional properties, such as reliability (e.g., the CONNECTor will ensure continued communication without interruption), security and privacy (e.g., the transactions do not disclose confidential data), trust (e.g., Networked Systems are put in communication only with parties they trust). After defining a conceptual framework for metrics definition, we present the approaches to dependability in CONNECT, which cover: i) Model-based V&V, ii) Security enforcement and iii) Trust management. The approaches are centered around monitoring, to allow for on-line analysis. Monitoring is performed alongside the functionalities of the CONNECTed System and is used to detect conditions that are deemed relevant by its clients (i.e., the other CONNECT Enablers). A unified lifecycle encompassing dependability analysis, security enforcement and trust management is outlined, spanning over discovery time, synthesis time and execution time

Flexible Application-Layer Multicast in Heterogeneous Networks

This work develops a set of peer-to-peer-based protocols and extensions in order to provide Internet-wide group communication. The focus is put to the question how different access technologies can be integrated in order to face the growing traffic load problem. Thereby, protocols are developed that allow autonomous adaptation to the current network situation on the one hand and the integration of WiFi domains where applicable on the other hand

SoS: self-organizing substrates

Large-scale networked systems often, both by design or chance exhibit self-organizing properties. Understanding self-organization using tools from cybernetics, particularly modeling them as Markov processes is a first step towards a formal framework which can be used in (decentralized) systems research and design.Interesting aspects to look for include the time evolution of a system and to investigate if and when a system converges to some absorbing states or stabilizes into a dynamic (and stable) equilibrium and how it performs under such an equilibrium state. Such a formal framework brings in objectivity in systems research, helping discern facts from artefacts as well as providing tools for quantitative evaluation of such systems. This thesis introduces such formalism in analyzing and evaluating peer-to-peer (P2P) systems in order to better understand the dynamics of such systems which in turn helps in better designs. In particular this thesis develops and studies the fundamental building blocks for a P2P storage system. In the process the design and evaluation methodology we pursue illustrate the typical methodological approaches in studying and designing self-organizing systems, and how the analysis methodology influences the design of the algorithms themselves to meet system design goals (preferably with quantifiable guarantees). These goals include efficiency, availability and durability, load-balance, high fault-tolerance and self-maintenance even in adversarial conditions like arbitrarily skewed and dynamic load and high membership dynamics (churn), apart of-course the specific functionalities that the system is supposed to provide. The functionalities we study here are some of the fundamental building blocks for various P2P applications and systems including P2P storage systems, and hence we call them substrates or base infrastructure. These elemental functionalities include: (i) Reliable and efficient discovery of resources distributed over the network in a decentralized manner; (ii) Communication among participants in an address independent manner, i.e., even when peers change their physical addresses; (iii) Availability and persistence of stored objects in the network, irrespective of availability or departure of individual participants from the system at any time; and (iv) Freshness of the objects/resources' (up-to-date replicas). Internet-scale distributed index structures (often termed as structured overlays) are used for discovery and access of resources in a decentralized setting. We propose a rapid construction from scratch and maintenance of the P-Grid overlay network in a self-organized manner so as to provide efficient search of both individual keys as well as a whole range of keys, doing so providing good load-balancing characteristics for diverse kind of arbitrarily skewed loads - storage and replication, query forwarding and query answering loads. For fast overlay construction we employ recursive partitioning of the key-space so that the resulting partitions are balanced with respect to storage load and replication. The proper algorithmic parameters for such partitioning is derived from a transient analysis of the partitioning process which has Markov property. Preservation of ordering information in P-Grid such that queries other than exact queries, like range queries can be efficiently and rather trivially handled makes P-Grid suitable for data-oriented applications. Fast overlay construction is analogous to building an index on a new set of keys making P-Grid suitable as the underlying indexing mechanism for peer-to-peer information retrieval applications among other potential applications which may require frequent indexing of new attributes apart regular updates to an existing index. In order to deal with membership dynamics, in particular changing physical address of peers across sessions, the overlay itself is used as a (self-referential) directory service for maintaining the participating peers' physical addresses across sessions. Exploiting this self-referential directory, a family of overlay maintenance scheme has been designed with lower communication overhead than other overlay maintenance strategies. The notion of dynamic equilibrium study for overlays under continuous churn and repairs, modeled as a Markov process, was introduced in order to evaluate and compare the overlay maintenance schemes. While the self-referential directory was originally invented to realize overlay maintenance schemes with lower overheads than existing overlay maintenance schemes, the self-referential directory is generic in nature and can be used for various other purposes, e.g., as a decentralized public key infrastructure. Persistence of peer identity across sessions, in spite of changes in physical address, provides a logical independence of the overlay network from the underlying physical network. This has many other potential usages, for example, efficient maintenance mechanisms for P2P storage systems and P2P trust and reputation management. We specifically look into the dynamics of maintaining redundancy for storage systems and design a novel lazy maintenance strategy. This strategy is algorithmically a simple variant of existing maintenance strategies which adapts to the system dynamics. This randomized lazy maintenance strategy thus explores the cost-performance trade-offs of the storage maintenance operations in a self-organizing manner. We model the storage system (redundancy), under churn and maintenance, as a Markov process. We perform an equilibrium study to show that the system operates in a more stable dynamic equilibrium with our strategy than for the existing maintenance scheme for comparable overheads. Particularly, we show that our maintenance scheme provides substantial performance gains in terms of maintenance overhead and system's resilience in presence of churn and correlated failures. Finally, we propose a gossip mechanism which works with lower communication overhead than existing approaches for communication among a relatively large set of unreliable peers without assuming any specific structure for their mutual connectivity. We use such a communication primitive for propagating replica updates in P2P systems, facilitating management of mutable content in P2P systems. The peer population affected by a gossip can be modeled as a Markov process. Studying the transient spread of gossips help in choosing proper algorithm parameters to reduce communication overhead while guaranteeing coverage of online peers. Each of these substrates in themselves were developed to find practical solutions for real problems. Put together, these can be used in other applications, including a P2P storage system with support for efficient lookup and inserts, membership dynamics, content mutation and updates, persistence and availability. Many of the ideas have already been implemented in real systems and several others are in the way to be integrated into the implementations. There are two principal contributions of this dissertation. It provides design of the P2P systems which are useful for end-users as well as other application developers who can build upon these existing systems. Secondly, it adapts and introduces the methodology of analysis of a system's time-evolution (tools typically used in diverse domains including physics and cybernetics) to study the long run behavior of P2P systems, and uses this methodology to (re-)design appropriate algorithms and evaluate them. We observed that studying P2P systems from the perspective of complex systems reveals their inner dynamics and hence ways to exploit such dynamics for suitable or better algorithms. In other words, the analysis methodology in itself strongly influences and inspires the way we design such systems. We believe that such an approach of orchestrating self-organization in internet-scale systems, where the algorithms and the analysis methodology have strong mutual influence will significantly change the way future such systems are developed and evaluated. We envision that such an approach will particularly serve as an important tool for the nascent but fast moving P2P systems research and development community

Global connectivity architecture of mobile personal devices

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2008.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 193-207).The Internet's architecture, designed in the days of large, stationary computers tended by technically savvy and accountable administrators, fails to meet the demands of the emerging ubiquitous computing era. Nontechnical users now routinely own multiple personal devices, many of them mobile, and need to share information securely among them using interactive, delay-sensitive applications.Unmanaged Internet Architecture (UIA) is a novel, incrementally deployable network architecture for modern personal devices, which reconsiders three architectural cornerstones: naming, routing, and transport. UIA augments the Internet's global name system with a personal name system, enabling users to build personal administrative groups easily and intuitively, to establish secure bindings between his devices and with other users' devices, and to name his devices and his friends much like using a cell phone's address book. To connect personal devices reliably, even while mobile, behind NATs or firewalls, or connected via isolated ad hoc networks, UIA gives each device a persistent, location-independent identity, and builds an overlay routing service atop IP to resolve and route among these identities. Finally, to support today's interactive applications built using concurrent transactions and delay-sensitive media streams, UIA introduces a new structured stream transport abstraction, which solves the efficiency and responsiveness problems of TCP streams and the functionality limitations of UDP datagrams. Preliminary protocol designs and implementations demonstrate UIA's features and benefits. A personal naming prototype supports easy and portable group management, allowing use of personal names alongside global names in unmodified Internet applications. A prototype overlay router leverages the naming layer's social network to provide efficient ad hoc connectivity in restricted but important common-case scenarios.(cont) Simulations of more general routing protocols--one inspired by distributed hash tables, one based on recent compact routing theory--explore promising generalizations to UIA's overlay routing. A library-based prototype of UIA's structured stream transport enables incremental deployment in either OS infrastructure or applications, and demonstrates the responsiveness benefits of the new transport abstraction via dynamic prioritization of interactive web downloads. Finally, an exposition and experimental evaluation of NAT traversal techniques provides insight into routing optimizations useful in UIA and elsewhere.by Bryan Alexander Ford.Ph.D

Change Detection in Streaming Data

Change detection is the process of identifying differences in the state of an object or phenomenon by observing it at different times or different locations in space. In the streaming context, it is the process of segmenting a data stream into different segments by identifying the points where the stream dynamics changes. Decentralized change detection can be used in many interesting, and important applications such environmental observing systems, medicare monitoring systems. Although there is great deal of work on distributed detection and data fusion, most of work focuses on the one-time change detection solutions. One-time change detection method requires to proceed data once in response to the change occurring. The trade-off of a continuous distributed detection of changes include detection accuracy, spaceefficiency, detection delay, and communication-efficiency. To achieve these goals, the wildfire warning system is used as a motivating scenario. From the challenges and requirements of the wildfire warning system, the change detection algorithms for streaming data are proposed a part of the solution to the wildfire warning system. By selecting various models of local change detection, different schemes for distributed change detections, and the data exchange protocols, different designs can be achieved. Based on this approach, the contributions of this dissertation are as follows. A general two-window framework for detecting changes in a single data stream is presented. A general synopsis-based change detection framework is proposed. Theoretical and empirical analysis shows that the detection performance of synopsisbased detector is similar to that of non-synopsis change detector if a distance function quantifying the changes is preserved under the process of constructing synopsis. A clustering-based change detection and clustering maintenance method over sliding window is presented. Clustering-based detector can automatically detect the changes in the multivariate streaming data. A framework for decentralized change detection in wireless sensor networks is proposed. A distributed framework for clustering streaming data is proposed by extending the two-phased stream clustering approach which is widely used to cluster a single data stream.Unter Änderungserkennung wird der Prozess der Erkennung von Unterschieden im Zustand eines Objekts oder Phänomens verstanden, wenn dieses zu verschiedenen Zeitpunkten oder an verschiedenen Orten beobachtet wird. Im Kontext der Datenstromverarbeitung stellt dieser Prozess die Segmentierung eines Datenstroms anhand der identifizierten Punkte, an denen sich die Stromdynamiken ändern, dar. Die Fähigkeit, Änderungen in den Stromdaten zu erkennen, darauf zu reagieren und sich daran anzupassen, spielt in vielen Anwendungsbereichen, wie z.B. dem Aktivitätsüberwachung, dem Datenstrom-Mining und Maschinenlernen sowie dem Datenmanagement hinsichtlich Datenmenge und Datenqualität, eine wichtige Rolle. Dezentralisierte Änderungserkennung kann in vielen interessanten und wichtigen Anwendungsbereichen, wie z.B. in Umgebungsüberwachungssystemen oder medizinischen Überwachungssystemen, eingesetzt werden. Obgleich es eine Vielzahl von Arbeiten im Bereich der verteilten Änderungserkennung und Datenfusion gibt, liegt der Fokus dieser Arbeiten meist lediglich auf der Erkennung von einmaligen Änderungen. Die einmalige Änderungserkennungsmethode erfordert die einmalige Verarbeitung der Daten als Antwort auf die auftretende Änderung. Der Kompromiss einer kontinuierlichen, verteilten Erkennung von Änderungen umfasst die Erkennungsgenauigkeit, die Speichereffizienz sowie die Berechnungseffizienz. Um dieses Ziel zu erreichen, wird das Flächenbrandwarnsystem als motivierendes Szenario genutzt. Basierend auf den Herausforderungen und Anforderungen dieses Warnsystems wird ein Algorithmus zur Erkennung von Änderungen in Stromdaten als Teil einer Gesamtlösung für das Flächenbrandwarnsystem vorgestellt. Durch die Auswahl verschiedener Modelle zur lokalen und verteilten Änderungserkennung sowie verschiedener Datenaustauschprotokolle können verschiedene Systemdesigns entwickelt werden. Basierend auf diesem Ansatz leistet diese Dissertation nachfolgend aufgeführte Beiträge. Es wird ein allgemeines 2-Fenster Framework zur Erkennung von Änderungen in einem einzelnen Datenstrom vorgestellt. Weiterhin wird ein allgemeines synopsenbasiertes Framework zur Änderungserkennung beschrieben. Mittels theoretischer und empirischer Analysen wird gezeigt, dass die Erkennungs-Performance des synopsenbasierten Änderungsdetektors ähnlich der eines nicht-synopsenbasierten ist, solange eine Distanzfunktion, welche die Änderungen quantifiziert, während der Erstellung der Synopse eingehalten wird. Es wird Cluster-basierte Änderungserkennung und Cluster-Pflege über gleitenden Fenstern vorgestellt.Weiterhin wird ein Framework zur verteilten Änderungserkennung in drahtlosen Sensornetzwerken beschrieben. Basierend auf dem 2-Phasen Stromdaten-Cluster-Ansatz, welcher weitestgehend zur Clusterung eines einzelnen Datenstroms eingesetzt wird, wird ein verteiltes Framework zur Clusterung von Stromdaten vorgestellt

Robust P2P Live Streaming

Projecte fet en col.laboració amb la Fundació i2CATThe provisioning of robust real-time communication services (voice, video, etc.) or media contents through the Internet in a distributed manner is an important challenge, which will strongly influence in current and future Internet evolution. Aware of this, we are developing a project named Trilogy leaded by the i2CAT Foundation, which has as main pillar the study, development and evaluation of Peer-to-Peer (P2P) Live streaming architectures for the distribution of high-quality media contents. In this context, this work concretely covers media coding aspects and proposes the use of Multiple Description Coding (MDC) as a flexible solution for providing robust and scalable live streaming over P2P networks. This work describes current state of the art in media coding techniques and P2P streaming architectures, presents the implemented prototype as well as its simulation and validation results