Fast Freenet: Improving Freenet Performance by Preferential Partition Routing and File Mesh Propagation

The Freenet Peer-to-Peer network is doing a good job in providing anonymity to the users. But the performance of the network in terms of download speed and request hit ratio is not that good. We propose two modifications to Freenet in order to improve the download speed and request hit ratio for all participants. To improve download speed we propose Preferential Partition Routing, where nodes are grouped according to bandwidth and slow nodes are discriminated when routing. For improvements in request hit ratio we propose File Mesh propagation where each node sends fuzzy information about what documents it posesses to its neigbors. To verify our proposals we simulate the Freenet network and the bandwidth restrictions present between nodes as well as using observed distributions for user actions to show how it affects the network. Our results show an improvement of the request hit ratio by over 30 times and an increase of the average download speed with six times, compared to regular Freenet routing

Large-Scale Distributed Coalition Formation

The CyberCraft project is an effort to construct a large scale Distributed Multi-Agent System (DMAS) to provide autonomous Cyberspace defense and mission assurance for the DoD. It employs a small but flexible agent structure that is dynamically reconfigurable to accommodate new tasks and policies. This document describes research into developing protocols and algorithms to ensure continued mission execution in a system of one million or more agents, focusing on protocols for coalition formation and Command and Control. It begins by building large-scale routing algorithms for a Hierarchical Peer to Peer structured overlay network, called Resource-Clustered Chord (RC-Chord). RC-Chord introduces the ability to efficiently locate agents by resources that agents possess. Combined with a task model defined for CyberCraft, this technology feeds into an algorithm that constructs task coalitions in a large-scale DMAS. Experiments reveal the flexibility and effectiveness of these concepts for achieving maximum work throughput in a simulated CyberCraft environment

Polar: proxies collaborating to achieve anonymous web browsing

User tracking and profiling is a growing threat to online privacy. Whilst Internet users can choose to withhold their personal information, their Internet usage can still be traced back to a unique IP address. This study considers anonymity as a strong and useful form of privacy protection. More specifically, we examine how current anonymity solutions suffer from a number of deficiencies: they are not commonly used, are vulnerable to a host of attacks or are impractical or too cumbersome for daily use. Most anonymity solutions are centralised or partially centralised and require trust in the operators. It is additionally noted how current solutions fail to promote anonymity for common Web activities such as performing online search queries and general day-to-day Web browsing. A primary objective of this research is to develop an anonymising Web browsing protocol which aims to be (1) fully distributed, (2) offer adequate levels of anonymity and (3) enable users to browse the Internet anonymously without overly complex mixing techniques. Our research has led to an anonymising protocol called Polar. Polar is a peer-to-peer network which relays Web requests amongst peers before forwarding it to a Web server, thus protecting the requester's identity. This dissertation presents the Polar model. Design choices and enhancements to the model are discussed. The author's implementation of Polar is also presented demonstrating that an implementation of Polar is feasible.Dissertation (MSc (Computer Science))--University of Pretoria, 2007.Computer Scienceunrestricte

Effects of Data Replication on Data Exfiltration in Mobile Ad hoc Networks Utilizing Reactive Protocols

A swarm of autonomous UAVs can provide a significant amount of ISR data where current UAV assets may not be feasible or practical. As such, the availability of the data the resides in the swarm is a topic that will benefit from further investigation. This thesis examines the impact of le replication and swarm characteristics such as node mobility, swarm size, and churn rate on data availability utilizing reactive protocols. This document examines the most prominent factors affecting the networking of nodes in a MANET. Factors include network routing protocols and peer-to-peer le protocols. It compares and contrasts several open source network simulator environments. Experiment implementation is documented, covering design considerations, assumptions, and software implementation, as well as detailing constant, response and variable factors. Collected data is presented and the results show that in swarms of sizes of 30, 45, and 60 nodes, le replication improves data availability until network saturation is reached, with the most significant benefit gained after only one copy is made. Mobility, churn rate, and swarm density all influence the replication impact

An ant-inspired, deniable routing approach in ad hoc question & answer networks

The ubiquity of the Internet facilitates electronic question and answering (Q&A) between real people with ease via community portals and social networking websites. It is a useful service which allows users to appeal to a broad range of answerers. In most cases however, Q&A services produce answers by presenting questions to the general public or associated digital community with little regard for the amount of time users spend examining and answering them. Ultimately, a question may receive large amounts of attention but still not be answered adequately. Several existing pieces of research investigate the reasons why questions do not receive answers on Q&A services and suggest that it may be associated with users being afraid of expressing themselves. Q&A works well for solving information needs, however, it rarely takes into account the privacy requirements of the users who form the service. This thesis was motivated by the need for a more targeted approach towards Q&A by distributing the service across ad hoc networks. The main contribution of this thesis is a novel routing technique and networking environment (distributed Q&A) which balances answer quality and user attention while protecting privacy through plausible deniability. Routing approaches are evaluated experimentally by statistics gained from peer-to-peer network simulations, composed of Q&A users modelled via features extracted from the analysis of a large Yahoo! Answers dataset. Suggestions for future directions to this work are presented from the knowledge gained from our results and conclusion

Distributed music-sharing system

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2003.Includes bibliographical references (p. 47-49).Text search has been an important feature of file sharing applications since Napster. This thesis explores a directory hierarchy for categorizing and retrieving content in peer-to-peer systems as an alternative to keyword search. This thesis discusses Melody, a music sharing system that implements a directory hierarchy. Melody is built on Chord, a distributed lookup algorithm, and DHash, a distributed hash table. We evaluate the performance consequences and usability of Melody. In addition, this thesis presents two support applications: Autocat, to support automatic categorization, and Paynet, to encourage users to pay for the songs they listen to.by James Robertson.M.Eng

SoS: self-organizing substrates

Large-scale networked systems often, both by design or chance exhibit self-organizing properties. Understanding self-organization using tools from cybernetics, particularly modeling them as Markov processes is a first step towards a formal framework which can be used in (decentralized) systems research and design.Interesting aspects to look for include the time evolution of a system and to investigate if and when a system converges to some absorbing states or stabilizes into a dynamic (and stable) equilibrium and how it performs under such an equilibrium state. Such a formal framework brings in objectivity in systems research, helping discern facts from artefacts as well as providing tools for quantitative evaluation of such systems. This thesis introduces such formalism in analyzing and evaluating peer-to-peer (P2P) systems in order to better understand the dynamics of such systems which in turn helps in better designs. In particular this thesis develops and studies the fundamental building blocks for a P2P storage system. In the process the design and evaluation methodology we pursue illustrate the typical methodological approaches in studying and designing self-organizing systems, and how the analysis methodology influences the design of the algorithms themselves to meet system design goals (preferably with quantifiable guarantees). These goals include efficiency, availability and durability, load-balance, high fault-tolerance and self-maintenance even in adversarial conditions like arbitrarily skewed and dynamic load and high membership dynamics (churn), apart of-course the specific functionalities that the system is supposed to provide. The functionalities we study here are some of the fundamental building blocks for various P2P applications and systems including P2P storage systems, and hence we call them substrates or base infrastructure. These elemental functionalities include: (i) Reliable and efficient discovery of resources distributed over the network in a decentralized manner; (ii) Communication among participants in an address independent manner, i.e., even when peers change their physical addresses; (iii) Availability and persistence of stored objects in the network, irrespective of availability or departure of individual participants from the system at any time; and (iv) Freshness of the objects/resources' (up-to-date replicas). Internet-scale distributed index structures (often termed as structured overlays) are used for discovery and access of resources in a decentralized setting. We propose a rapid construction from scratch and maintenance of the P-Grid overlay network in a self-organized manner so as to provide efficient search of both individual keys as well as a whole range of keys, doing so providing good load-balancing characteristics for diverse kind of arbitrarily skewed loads - storage and replication, query forwarding and query answering loads. For fast overlay construction we employ recursive partitioning of the key-space so that the resulting partitions are balanced with respect to storage load and replication. The proper algorithmic parameters for such partitioning is derived from a transient analysis of the partitioning process which has Markov property. Preservation of ordering information in P-Grid such that queries other than exact queries, like range queries can be efficiently and rather trivially handled makes P-Grid suitable for data-oriented applications. Fast overlay construction is analogous to building an index on a new set of keys making P-Grid suitable as the underlying indexing mechanism for peer-to-peer information retrieval applications among other potential applications which may require frequent indexing of new attributes apart regular updates to an existing index. In order to deal with membership dynamics, in particular changing physical address of peers across sessions, the overlay itself is used as a (self-referential) directory service for maintaining the participating peers' physical addresses across sessions. Exploiting this self-referential directory, a family of overlay maintenance scheme has been designed with lower communication overhead than other overlay maintenance strategies. The notion of dynamic equilibrium study for overlays under continuous churn and repairs, modeled as a Markov process, was introduced in order to evaluate and compare the overlay maintenance schemes. While the self-referential directory was originally invented to realize overlay maintenance schemes with lower overheads than existing overlay maintenance schemes, the self-referential directory is generic in nature and can be used for various other purposes, e.g., as a decentralized public key infrastructure. Persistence of peer identity across sessions, in spite of changes in physical address, provides a logical independence of the overlay network from the underlying physical network. This has many other potential usages, for example, efficient maintenance mechanisms for P2P storage systems and P2P trust and reputation management. We specifically look into the dynamics of maintaining redundancy for storage systems and design a novel lazy maintenance strategy. This strategy is algorithmically a simple variant of existing maintenance strategies which adapts to the system dynamics. This randomized lazy maintenance strategy thus explores the cost-performance trade-offs of the storage maintenance operations in a self-organizing manner. We model the storage system (redundancy), under churn and maintenance, as a Markov process. We perform an equilibrium study to show that the system operates in a more stable dynamic equilibrium with our strategy than for the existing maintenance scheme for comparable overheads. Particularly, we show that our maintenance scheme provides substantial performance gains in terms of maintenance overhead and system's resilience in presence of churn and correlated failures. Finally, we propose a gossip mechanism which works with lower communication overhead than existing approaches for communication among a relatively large set of unreliable peers without assuming any specific structure for their mutual connectivity. We use such a communication primitive for propagating replica updates in P2P systems, facilitating management of mutable content in P2P systems. The peer population affected by a gossip can be modeled as a Markov process. Studying the transient spread of gossips help in choosing proper algorithm parameters to reduce communication overhead while guaranteeing coverage of online peers. Each of these substrates in themselves were developed to find practical solutions for real problems. Put together, these can be used in other applications, including a P2P storage system with support for efficient lookup and inserts, membership dynamics, content mutation and updates, persistence and availability. Many of the ideas have already been implemented in real systems and several others are in the way to be integrated into the implementations. There are two principal contributions of this dissertation. It provides design of the P2P systems which are useful for end-users as well as other application developers who can build upon these existing systems. Secondly, it adapts and introduces the methodology of analysis of a system's time-evolution (tools typically used in diverse domains including physics and cybernetics) to study the long run behavior of P2P systems, and uses this methodology to (re-)design appropriate algorithms and evaluate them. We observed that studying P2P systems from the perspective of complex systems reveals their inner dynamics and hence ways to exploit such dynamics for suitable or better algorithms. In other words, the analysis methodology in itself strongly influences and inspires the way we design such systems. We believe that such an approach of orchestrating self-organization in internet-scale systems, where the algorithms and the analysis methodology have strong mutual influence will significantly change the way future such systems are developed and evaluated. We envision that such an approach will particularly serve as an important tool for the nascent but fast moving P2P systems research and development community

Building Robust Distributed Infrastructure Networks

Many competing designs for Distributed Hash Tables exist exploring multiple models of addressing, routing and network maintenance. Designing a general theoretical model and implementation of a Distributed Hash Table allows exploration of the possible properties of Distributed Hash Tables. We will propose a generalized model of DHT behavior, centered on utilizing Delaunay triangulation in a given metric space to maintain the networks topology. We will show that utilizing this model we can produce network topologies that approximate existing DHT methods and provide a starting point for further exploration. We will use our generalized model of DHT construction to design and implement more efficient Distributed Hash Table protocols, and discuss the qualities of potential successors to existing DHT technologies

A DHT-based Peer-to-peer Architecture for Distributed Internet Applications

La tecnologia peer-to-peer é divenuta popolare soprattutto per applicazioni di file-sharing come Napster, Gnutella, Kazaa ed eMule, che sono state la componente principale del traffico di Internet per diversi anni. La tecnologia peer-to-peer, tuttavia, non é solo relativa al file-sharing. Molte applicazioni, utilizzate da milioni di utenti ogni giorno, come Skype, sono applicazioni basate sul paradigma peer-to-peer. Il paradigma peer-to-peer (P2P) é un modello di comunicazione in cui una moltitudine di dispositivi indipendenti ed eterogenei interagiscono come pari (peer). In una rete P2P pura, ogni nodo implementa le funzionalità sia di client che di server, e ciascun peer può instaurare una sessione di comunicazione in qualsiasi momento. I nodi sono disposti in un'overlay network, costruita sopra ad una rete esistente, come Internet. Molte applicazioni peer-to-peer sono basate su una particolare classe di reti peer-to-peer: le Distributed Hash Tables (DHT). Le DHT sono reti peer-to-peer strutturate che forniscono un servizio di memorizzazione e recupero di informazioni simile ad una classica hash table, in cui le chiavi sono mappate a valori, in modo scalabile, flessibile ed auto-configurante. Questa tesi riporta i risultati della ricerca sull'applicazione delle tecnologie peer-to-peer al di là del file sharing. Il lavoro é stato concentrato in primo luogo sullo studio ed analisi delle implementazioni esistenti di reti peer-to-peer, specialmente le Distributed Hash Tables, e le proposte per protocolli peer-to-peer definite dall'IETF P2PSIP Working Group. La principale attività di ricerca é stata la definizione di un'architettura peer-to-peer, chiamata Distributed Location Service (DLS), che permette di instaurare connessioni dirette tra gli estremi di una comunicazione senza la necessità di dipendere da server centralizzati. Il Distributed Location Server é un servizio peer-to-peer basato su DHT che può essere utilizzato per memorizzare e recuperare informazioni relative a dove e come accedere alle risorse, eliminando il bisogno di dipendere (parzialmente) dal sistema DNS e da servizi di localizzazione centralizzati, come il SIP Location Service. Le informazioni di accesso sono memorizzate nel DLS come coppie chiave-valore, che sono mantenute da una moltitudine di nodi che partecipano alla DHT su cui si basa il DLS. Il DLS é stato implementato come un framework, definendo un set di interfacce standard per la comunicazione tra i componenti del DLS, al fine di consentire la massima flessibilità sui componenti, come l'algoritmo di DHT e il protocollo di comunicazione in uso, in quanto nessuna ipotesi é stata formulata al riguardo nella definizione dell'architettura del DLS. L'algoritmo di DHT Kademlia e il protocollo di comunicazione dSIP sono stati implementati ed integrati nel framework DLS per creare applicazioni basate su DLS al fine di dimostrare la praticabilità dell'approccio DLS. Queste applicazioni dimostrative sono state realizzate altresì con l'intento di mostrare che il peer-to-peer non può essere ridotto al solo file sharing, ma che applicazioni di comunicazione real-time, come il VoIP, file system distribuiti, e Social Netowrks possono essere realizzati utilizzando come base un'architettura peer-to-peer. Sebbene l'attività di ricerca sia stata condotta in maniera indipendente dall'IETF P2PSIP Working Group, il Distributed Location Service si é rivelato molto simile alla proposta ufficiale, chiamata RELOAD, con la quale condivide diversi concetti ed idee. Un altro aspetto studiato é stato il problema del bootstrapping nelle reti peer-to-peer. Quando un nodo intende unirsi ad una rete P2P esistente, esso deve contattare un nodo che appartiene già all'overlay P2P, il quale ammetterà il nuovo nodo. Tipicamente, la scoperta di un nodo che partecipa già all'overlay avviene attraverso meccanismi quali l'utilizzo di cache, liste di nodi pre-configurate e l'interrogazione di server centralizzati. Sebbene questi approcci abbiano funzionato finora, essi non appartengono alla filosofia peer-to-peer, in cui la decentralizzazione, la scalabilità e l'auto-configurazione sono aspetti cruciali. Si é quindi definito e validato un approccio basato su Multicast, il cui scopo é quello di ottenere un servizio caratterizzato da scalabilità ed auto-configurazione.Peer-to-peer technology has become popular primarily due to file sharing applications, such as Napster, Gnutella, Kazaa, and eMule, which have been the dominant component of usage of Internet bandwidth for several years. However, peer-to-peer technology is not all about file sharing. Many famous applications used by millions of users every day, such as Skype, are applications based on the peer-to-peer paradigm. The peer-to-peer (P2P) paradigm is a communication model in which multiple independent and heterogeneous devices interact as equals (peers). In a pure P2P network each node implements functions of both client and server, and either peer can initiate a communication session at any moment. Nodes are arranged on an overlay network, built on top of an existing network, such as the Internet. Many peer-to-peer applications are based on a particular class of peer-to-peer networks: Distributed Hash Tables (DHT). DHTs are structured peer-to-peer networks which provide a service of information storage and retrieval similar to a regular hash table where keys are mapped to values, in a scalable, flexible, and self-organizing fashion. This thesis reports the results of the research activity on applying peer-to-peer technology beyond file sharing. The work has been focused first on the study and analysis of existing peer-to-peer network implementations, especially on Distributed Hash Tables, and the proposals for peer-to-peer protocols presented by the IETF P2PSIP Working Group. The main research activity has been the definition of a peer-to-peer architecture, called Distributed Location Service (DLS), which allows the establishment of direct connections among the endpoints of a communication without the need of central servers. The Distributed Location Service is a DHT-based peer-to-peer service which can be used to store and retrieve information about where resources can be accessed, thus eliminating the need to rely (partially) on the DNS system and on central location servers, such as SIP Location Services. Access information is stored in the DLS as key-to-value mappings, which are maintained by a number of nodes that participate in the DHT overlay the DLS is built upon. The DLS has been implemented as a framework, by defining a standard set of interfaces between the components of the DLS, in order to allow maximum flexibility on components such as the DHT algorithm and communication protocol in use, as no assumption has been made in the definition of the DLS architecture. The Kademlia DHT algorithm and the dSIP communication protocol have been implemented and integrated in the DLS framework in order to create real-world DLS-based application to show the feasibility of the DLS approach. These demonstrative DLS-based applications have been realized with the intent to show that peer-to-peer is not just about file sharing, but real-time communication applications, such as VoIP, distributed file systems, and Online Social Networks, can also be built on top of a peer-to-peer architecture. Even though the research activity has been conducted independently from the IETF P2PSIP Working Group, the Distributed Location Service has been eventually found quite similar to the official proposal, named RELOAD, with whom it shares several concepts and ideas. Another aspect that was studied is the issue of bootstrapping in peer-to-peer networks. When a node wants to join an existing P2P network, it needs to gather information about one node that already belongs to the P2P overlay network which will then admit the new node. Typically, the discovery of a node that is already participating in the overlay is made through mechanisms such as caching, pre-configured list of nodes, or the use of central servers. Even though these approaches have worked so far, they are not in the true philosophy of peer-to-peer networks, where decentralization, scalability, and self-organization are critical features. A Multicast-based approach has therefore been defined and validated, with the goal of achieving true scalability and self-organization