Modeling, simulations, and experiments to balance performance and fairness in P2P file-sharing systems

Doctor of PhilosophyDepartment of Electrical and Computer EngineeringDon GruenbacherCaterina ScoglioIn this dissertation, we investigate research gaps still existing in P2P file-sharing systems: the necessity of fairness maintenance during the content information publishing/retrieving process, and the stranger policies on P2P fairness. First, through a wide range of measurements in the KAD network, we present the impact of a poorly designed incentive fairness policy on the performance of looking up content information. The KAD network, designed to help peers publish and retrieve sharing information, adopts a distributed hash table (DHT) technology and combines itself into the aMule/eMule P2P file-sharing network. We develop a distributed measurement framework that employs multiple test nodes running on the PlanetLab testbed. During the measurements, the routing tables of around 20,000 peers are crawled and analyzed. More than 3,000,000 pieces of source location information from the publishing tables of multiple peers are retrieved and contacted. Based on these measurements, we show that the routing table is well maintained, while the maintenance policy for the source-location-information publishing table is not well designed. Both the current maintenance schedule for the publishing table and the poor incentive policy on publishing peers eventually result in the low availability of the publishing table, which accordingly cause low lookup performance of the KAD network. Moreover, we propose three possible solutions to address these issues: the self-maintenance scheme with short period renewal interval, the chunk-based publishing/retrieving scheme, and the fairness scheme. Second, using both numerical analyses and agent-based simulations, we evaluate the impact of different stranger policies on system performance and fairness. We explore that the extremely restricting stranger policy brings the best fairness at a cost of performance degradation. The varying tendency of performance and fairness under different stranger policies are not consistent. A trade-off exists between controlling free-riding and maintaining system performance. Thus, P2P designers are required to tackle strangers carefully according to their individual design goals. We also show that BitTorrent prefers to maintain fairness with an extremely restricting stranger policy, while aMule/eMule’s fully rewarding stranger policy promotes free-riders’ benefit

Query processing in P2P systems

Peer-to-peer (P2P) computing offers new opportunities for building highly distributed data systems. Unlike client-server computing, P2P is a very dynamic environment where peers can join and leave the network at any time. This yields important advantages such as operation without central coordination, peers autonomy, and scale up to large number of peers. However, providing high-level data management services is difficult. Most techniques designed in distributed database systems which statically exploit schema and network information no longer apply. New techniques are needed which should be decentralized, dynamic and self-adaptive. In this paper, we survey the techniques which have been developed for query processing in P2P systems. We first give an overview of the existing P2P networks, and com-pare their properties from the perspective of data management. Then, we discuss the ap-proaches which are used for schema mapping. Then, we describe the algorithms which have been proposed for query routing. In particular, we focus on query routing in unstructured net-works and DHTs. Finally, we present the techniques which have been proposed for processing complex queries, e.g. top-k queries, in P2P systems, in particular in DHTs

A holistic architecture using peer to peer (P2P) protocols for the internet of things and wireless sensor networks

Wireless Sensor Networks (WSNs) interact with the physical world using sensing and/or actuation. The wireless capability of WSN nodes allows them to be deployed close to the sensed phenomenon. Cheaper processing power and the use of micro IP stacks allow nodes to form an “Internet of Things” (IoT) integrating the physical world with the Internet in a distributed system of devices and applications. Applications using the sensor data may be located across the Internet from the sensor network, allowing Cloud services and Big Data approaches to store and analyse this data in a scalable manner, supported by new approaches in the area of fog and edge computing. Furthermore, the use of protocols such as the Constrained Application Protocol (CoAP) and data models such as IPSO Smart Objects have supported the adoption of IoT in a range of scenarios. IoT has the potential to become a realisation of Mark Weiser’s vision of ubiquitous computing where tiny networked computers become woven into everyday life. This presents the challenge of being able to scale the technology down to resource-constrained devices and to scale it up to billions of devices. This will require seamless interoperability and abstractions that can support applications on Cloud services and also on node devices with constrained computing and memory capabilities, limited development environments and requirements on energy consumption. This thesis proposes a holistic architecture using concepts from tuple-spaces and overlay Peer-to-Peer (P2P) networks. This architecture is termed as holistic, because it considers the flow of the data from sensors through to services. The key contributions of this work are: development of a set of architectural abstractions to provide application layer interoperability, a novel cache algorithm supporting leases, a tuple-space based data store for local and remote data and a Peer to Peer (P2P) protocol with an innovative use of a DHT in building an overlay network. All these elements are designed for implementation on a resource constrained node and to be extensible to server environments, which is shown in a prototype implementation. This provides the basis for a new P2P holistic approach that will allow Wireless Sensor Networks and IoT to operate in a self-organising ad hoc manner in order to deliver the promise of IoT

Improving end-to-end availability using overlay networks

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2005.Includes bibliographical references (p. 139-150).The end-to-end availability of Internet services is between two and three orders of magnitude worse than other important engineered systems, including the US airline system, the 911 emergency response system, and the US public telephone system. This dissertation explores three systems designed to mask Internet failures, and, through a study of three years of data collected on a 31-site testbed, why these failures happen and how effectively they can be masked. A core aspect of many of the failures that interrupt end-to-end communication is that they fall outside the expected domain of well-behaved network failures. Many traditional techniques cope with link and router failures; as a result, the remaining failures are those caused by software and hardware bugs, misconfiguration, malice, or the inability of current routing systems to cope with persistent congestion.The effects of these failures are exacerbated because Internet services depend upon the proper functioning of many components-wide-area routing, access links, the domain name system, and the servers themselves-and a failure in any of them can prove disastrous to the proper functioning of the service. This dissertation describes three complementary systems to increase Internet availability in the face of such failures. Each system builds upon the idea of an overlay network, a network created dynamically between a group of cooperating Internet hosts. The first two systems, Resilient Overlay Networks (RON) and Multi-homed Overlay Networks (MONET) determine whether the Internet path between two hosts is working on an end-to-end basis. Both systems exploit the considerable redundancy available in the underlying Internet to find failure-disjoint paths between nodes, and forward traffic along a working path. RON is able to avoid 50% of the Internet outages that interrupt communication between a small group of communicating nodes.MONET is more aggressive, combining an overlay network of Web proxies with explicitly engineered redundant links to the Internet to also mask client access link failures. Eighteen months of measurements from a six-site deployment of MONET show that it increases a client's ability to access working Web sites by nearly an order of magnitude. Where RON and MONET combat accidental failures, the Mayday system guards against denial- of-service attacks by surrounding a vulnerable Internet server with a ring of filtering routers. Mayday then uses a set of overlay nodes to act as mediators between the service and its clients, permitting only properly authenticated traffic to reach the server.by David Godbe Andersen.Ph.D

Enabling peer-to-peer remote experimentation in distributed online remote laboratories

Remote Access Laboratories (RALs) are online platforms that allow human user interaction with physical instruments over the Internet. Usually RALs follow a client-server paradigm. Dedicated providers create and maintain experiments and corresponding educational content. In contrast, this dissertation focuses on a Peer-to-Peer (P2P) service model for RALs where users are encouraged to host experiments at their location. This approach can be seen as an example of an Internet of Things (IoT) system. A set of smart devices work together providing a cyber-physical interface for users to run experiments remotely via the Internet. The majority of traditional RAL learning activities focus on undergraduate education where hands-on experience such as building experiments, is not a major focus. In contrast this work is motivated by the need to improve Science, Technology, Engineering and Mathematics (STEM) education for school-aged children. Here physically constructing experiments forms a substantial part of the learning experience. In the proposed approach, experiments can be designed with relatively simple components such as LEGO Mindstorms or Arduinos. The user interface can be programed using SNAP!, a graphical programming tool. While the motivation for the work is educational in nature, this thesis focuses on the technical details of experiment control in an opportunistic distributed environment. P2P RAL aims to enable any two random participants in the system - one in the role of maker creating and hosting an experiment and one in the role of learner using the experiment - to establish a communication session during which the learner runs the remote experiment through the Internet without requiring a centralized experiment or service provider. The makers need to have support to create the experiment according to a common web based programing interface. Thus, the P2P approach of RALs requires an architecture that provides a set of heterogeneous tools which can be used by makers to create a wide variety of experiments. The core contribution of this dissertation is an automaton-based model (twin finite state automata) of the controller units and the controller interface of an experiment. This enables the creation of experiments based on a common platform, both in terms of software and hardware. This architecture enables further development of algorithms for evaluating and supporting the performance of users which is demonstrated through a number of algorithms. It can also ensure the safety of instruments with intelligent tools. The proposed network architecture for P2P RALs is designed to minimise latency to improve user satisfaction and learning experience. As experiment availability is limited for this approach of RALs, novel scheduling strategies are proposed. Each of these contributions has been validated through either simulations, e.g. in case of network architecture and scheduling, or test-bed implementations, in case of the intelligent tools. Three example experiments are discussed along with users' feedback on their experience of creating an experiment and using others’ experimental setup. The focus of the thesis is mainly on the design and hosting of experiments and ensuring user accessibility to them. The main contributions of this thesis are in regards to machine learning and data mining techniques applied to IoT systems in order to realize the P2P RALs system. This research has shown that a P2P architecture of RALs can provide a wide variety of experimental setups in a modular environment with high scalability. It can potentially enhance the user-learning experience while aiding the makers of experiments. It presents new aspects of learning analytics mechanisms to monitor and support users while running experiments, thus lending itself to further research. The proposed mathematical models are also applicable to other Internet of Things applications