Overlay Networks for Edge Management

Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).Edge computing has emerged as a solution to address existing limitations of cloud computing for bandwidth-heavy and time-sensitive applications, by moving (some) computations from bandwidth saturated Cloud infrastructures closer to client devices, where data is effectively produced and consumed. However, existing materializations of the edge computing paradigm take limited advantage of computational and storage power that exists in the edge and between client devices and the cloud. Most of these leverage static hierarchical topologies (e.g., Fog Computing) to pre-process data before sending it to the Cloud, which limits the advantages that can be extracted from the edge computing paradigm. In the past, peer-to-peer systems have sought to tackle the challenges of increasing scalability and availability for very large systems, with a large number of solutions being proposed namely, distributed overlay networks for resource management. In this paper, we argue that the clever adaptation of peer-to-peer solutions can enable novel applications to fully exploit the potential of the edge. In particular, we study the viability of taking advantage of specialized overlay networks in edge environments to enable the management of a large number of computational resources. Contrary to previous proposals, that assume the environment to be composed of mostly homogeneous devices, our proposal embraces existing heterogeneity and exploits the location of computational resources to devise a (partially) self-organizing overlay network that can be exploited both to provide membership information to applications, but also do efficiently disseminate management information across edge devices. We have conducted an experimental evaluation using container-based emulation in an heterogeneous network composed by 100 devices, with results showing that our protocol is able to maximize the bandwidth usage of the system, allowing more data to flow throughout the network, while retaining high robustness to failures.authorsversionpublishe

A Novel Approach to Load Balancing in P2P Overlay Networks for Edge Systems

Edge computing aims at addressing some limitations of cloud computing by bringing computation towards the edge of the system, i.e., closer to the client. There is a panoply of devices that can be integrated into future edge computing platforms, from local datacenters and ISP points of presence, to 5G towers, and even, multiple user devices like smartphones, laptops, and IoT devices. For all of these devices to communicate fruitfully, we need to build systems that enable the seamless interaction and cooperation among these diverse devices. However, creating and maintaining these systems is not trivial since there are numerous types of devices with different capacities. This resource heterogeneity has to be taken into account so that different types of machines contribute to the management of the distributed infrastructure differently, and the operation of the overall system becomes more efficient. In this work, we addressed the challenges identified above by exploring unstructured overlay networks, that have been shown to be possible to manage efficiently and in a fully decentralized way, while being highly robust to failures. To that end, we devised a solution that adapts the number of neighbors of each device (i.e., how many other devices that device knows) according to the capacity of that device and the distribution of capacities of the other devices in the network, as to ensure that the load is fairly distributed between them and, as a consequence, improve the operation of other services atop the unstructured overlay network, for instance, reducing the latencies experienced when broadcasting information. This solution can be easily integrated into most existing peer-to-peer distributed systems, requiring just a slight adaptation to their membership protocol. To show the correction and benefits of our proposal, we evaluated it by comparing it with state of the art decentralized solutions to manage unstructured overlay networks, combining both simulation (to observe the performance of the solution at large scale) and prototype deployments in realistic distributed infrastructures.A computação de periferia visa abordar algumas limitações da computação em nuvem, trazendo a computação para mais perto do cliente. Há uma enorme variedade de dispositivos que podem ser integrados em futuras plataformas de computação de periferia, de data centers locais e pontos de presença de ISPs a torres 5G e até mesmo dispositivos de cliente, como smartphones, laptops e dispositivos IoT. Para que todos esses dispositivos comuniquem de forma proveitosa entre si, precisamos construir sistemas que possibilitem a interação e cooperação eficaz entre eles. No entanto, criar e manter esses sistemas não é trivial, uma vez que existem vários tipos de dispositivos com diferentes capacidades. Essa heterogeneidade de recursos deve ser levada em consideração para que diferentes tipos de máquinas contribuam para o gerenciamento da infraestrutura distribuída de forma distinta e a operação do sistema se torne mais eficiente. Neste trabalho, enfrentámos os desafios identificados acima explorando redes sobrepostas não estruturadas, que se têm mostrado possíveis de gerenciar de forma eficiente e totalmente descentralizada, sendo altamente resistentes a falhas. Para tal, concebemos uma solução que adapta o número de vizinhos de cada dispositivo (ou seja, quantos outros dispositivos aquele dispositivo conhece) de acordo com a sua capacidade e a capacidade dos demais dispositivos da rede, de forma a garantir que a carga seja proporcionalmente distribuída entre eles e, como consequência, reduzindo as latências experienciadas por esses dispositivos. Esta solução pode ser facilmente integrada num sistema distribuído entre-pares existente, exigindo apenas uma ligeira adaptação ao seu protocolo de filiação. Avaliámos a nossa solução comparando-a com outras soluções descentralizadas de última geração, combinando simulação (para observar o desempenho da soluç

Intelligent query processing in P2P networks: semantic issues and routing algorithms

P2P networks have become a commonly used way of disseminating content on the Internet. In this context, constructing efficient and distributed P2P routing algorithms for complex environments that include a huge number of distributed nodes with different computing and network capabilities is a major challenge. In the last years, query routing algorithms have evolved by taking into account different features (provenance, nodes' history, topic similarity, etc.). Such features are usually stored in auxiliary data structures (tables, matrices, etc.), which provide an extra knowledge engineering layer on top of the network, resulting in an added semantic value for specifying algorithms for efficient query routing. This article examines the main existing algorithms for query routing in unstructured P2P networks in which semantic aspects play a major role. A general comparative analysis is included, associated with a taxonomy of P2P networks based on their degree of decentralization and the different approaches adopted to exploit the available semantic aspects.Fil: Nicolini, Ana Lucía. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Instituto de Ciencias e Ingeniería de la Computación. Universidad Nacional del Sur. Departamento de Ciencias e Ingeniería de la Computación. Instituto de Ciencias e Ingeniería de la Computación; ArgentinaFil: Lorenzetti, Carlos Martin. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Instituto de Ciencias e Ingeniería de la Computación. Universidad Nacional del Sur. Departamento de Ciencias e Ingeniería de la Computación. Instituto de Ciencias e Ingeniería de la Computación; ArgentinaFil: Maguitman, Ana Gabriela. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Instituto de Ciencias e Ingeniería de la Computación. Universidad Nacional del Sur. Departamento de Ciencias e Ingeniería de la Computación. Instituto de Ciencias e Ingeniería de la Computación; ArgentinaFil: Chesñevar, Carlos Iván. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Instituto de Ciencias e Ingeniería de la Computación. Universidad Nacional del Sur. Departamento de Ciencias e Ingeniería de la Computación. Instituto de Ciencias e Ingeniería de la Computación; Argentin

階層型ピア・ツー・ピアファイル検索のための負荷管理の研究

In a Peer-to-Peer (P2P) system, multiple interconnected peers or nodes contribute a portion of their resources (e.g., files, disk storage, network bandwidth) in order to inexpensively handle tasks that would normally require powerful servers. Since the emergency of P2P file sharing, load balancing has been considered as a primary concern, as well as other issues such as autonomy, fault tolerance and security. In a process of file search, a heavily loaded peer may incur a long latency or failure in query forwarding or responding. If there are many such peers in a system, it may cause link congestion or path congestion, and consequently affect the performance of overall system. To avoid such situation, some of general techniques used in Web systems such as caching and paging are adopted into P2P systems. However, it is highly insufficient for load balancing since peers often exhibit high heterogeneity and dynamicity in P2P systems. To overcome such a difficulty, the use of super-peers is currently being the most promising approach in optimizing allocation of system load to peers, i.e., it allocates more system load to high capacity and stable super-peers by assigning task of index maintenance and retrieval to them. In this thesis, we focused on two kinds of super-peer based hierarchical architectures of P2P systems, which are distinguished by the organization of super-peers. In each of them, we discussed system load allocation, and proposed novel load balancing algorithms for alleviating load imbalance of super-peers, aiming to decrease average and variation of query response time during index retrieval process. More concretely, in this thesis, our contribution to load management solutions for hierarchical P2P file search are the following: • In Qin’s hierarchical architecture, indices of files held by the user peers in the bottom layer are stored at the super-peers in the middle layer, and the correlation of those two bottom layers is controlled by the central server(s) in the top layer using the notion of tags. In Qin’s system, a heavily loaded super-peer can move excessive load to a lightly loaded super-peer by using the notion of task migration. However, such a task migration approach is not sufficient to balance the load of super-peers if the size of tasks is highly imbalanced. To overcome such an issue, in this thesis, we propose two task migration schemes for this architecture, aiming to ensure an even load distribution over the super-peers. The first scheme controls the load of each task in order to decrease the total cost of task migration. The second scheme directly balances the load over tasks by reordering the priority of tags used in the query forwarding step. The effectiveness of the proposed schemes are evaluated by simulation. The result of simulations indicates that all the schemes can work in coordinate, in alleviating the bottleneck situation of super-peers. • In DHT-based super-peer architecture, indices of files held by the user peers in the lower layer are stored at the DHT connected super-peers in the upper layer. In DHT-based super-peer systems, the skewness of user’s preference regarding keywords contained in multi-keyword query causes query load imbalance of super-peers that combines both routing and response load. Although index replication has a great potential for alleviating this problem, existing schemes did not explicitly address it or incurred high cost. To overcome such an issue, in this thesis, we propose an integrated solution that consists of three replication schemes to alleviate query load imbalance while minimizing the cost. The first scheme is an active index replication in order to decrease routing load in the super-peer layer, and distribute response load of an index among super-peers that stored the replica. The second scheme is a proactive pointer replication that places location information of an index, for reducing maintenance cost between the index and its replicas. The third scheme is a passive index replication that guarantees the maximum query load of super-peers. The result of simulations indicates that the proposed schemes can help alleviating the query load imbalance of super-peers. Moreover, by comparison it was found that our schemes are more cost-effective on placing replicas than other approaches.広島大学(Hiroshima University)博士(工学)Doctor of Engineering in Information Engineeringdoctora

Preliminary specification and design documentation for software components to achieve catallaxy in computational systems

This Report is about the preliminary specifications and design documentation for software components to achieve Catallaxy in computational systems. -- Die Arbeit beschreibt die Spezifikation und das Design von Softwarekomponenten, um das Konzept der Katallaxie in Grid Systemen umzusetzen. Eine Einführung ordnet das Konzept der Katallaxie in bestehende Grid Taxonomien ein und stellt grundlegende Komponenten vor. Anschließend werden diese Komponenten auf ihre Anwendbarkeit in bestehenden Application Layer Netzwerken untersucht.Grid Computing

Ontology-based Search Algorithms over Large-Scale Unstructured Peer-to-Peer Networks

Peer-to-Peer(P2P) systems have emerged as a promising paradigm to structure large scale distributed systems. They provide a robust, scalable and decentralized way to share and publish data.The unstructured P2P systems have gained much popularity in recent years for their wide applicability and simplicity. However efficient resource discovery remains a fundamental challenge for unstructured P2P networks due to the lack of a network structure. To effectively harness the power of unstructured P2P systems, the challenges in distributed knowledge management and information search need to be overcome. Current attempts to solve the problems pertaining to knowledge management and search have focused on simple term based routing indices and keyword search queries. Many P2P resource discovery applications will require more complex query functionality, as users will publish semantically rich data and need efficiently content location algorithms that find target content at moderate cost. Therefore, effective knowledge and data management techniques and search tools for information retrieval are imperative and lasting. In my dissertation, I present a suite of protocols that assist in efficient content location and knowledge management in unstructured Peer-to-Peer overlays. The basis of these schemes is their ability to learn from past peer interactions and increasing their performance with time.My work aims to provide effective and bandwidth-efficient searching and data sharing in unstructured P2P environments. A suite of algorithms which provide peers in unstructured P2P overlays with the state necessary in order to efficiently locate, disseminate and replicate objects is presented. Also, Existing approaches to federated search are adapted and new methods are developed for semantic knowledge representation, resource selection, and knowledge evolution for efficient search in dynamic and distributed P2P network environments. Furthermore,autonomous and decentralized algorithms that reorganizes an unstructured network topology into a one with desired search-enhancing properties are proposed in a network evolution model to facilitate effective and efficient semantic search in dynamic environments

Peer to Peer Information Retrieval: An Overview

Peer-to-peer technology is widely used for file sharing. In the past decade a number of prototype peer-to-peer information retrieval systems have been developed. Unfortunately, none of these have seen widespread real- world adoption and thus, in contrast with file sharing, information retrieval is still dominated by centralised solutions. In this paper we provide an overview of the key challenges for peer-to-peer information retrieval and the work done so far. We want to stimulate and inspire further research to overcome these challenges. This will open the door to the development and large-scale deployment of real-world peer-to-peer information retrieval systems that rival existing centralised client-server solutions in terms of scalability, performance, user satisfaction and freedom

HARD: Hybrid Adaptive Resource Discovery for Jungle Computing

In recent years, Jungle Computing has emerged as a distributed computing paradigm based on simultaneous combination of various hierarchical and distributed computing environments which are composed by large number of heterogeneous resources. In such a computing environment, the resources and the underlying computation and communication infrastructures are highly-hierarchical and heterogeneous. This creates a lot of difficulty and complexity for finding the proper resources in a precise way in order to run a particular job on the system efficiently. This paper proposes Hybrid Adaptive Resource Discovery (HARD), a novel efficient and highly scalable resource-discovery approach which is built upon a virtual hierarchical overlay based on self-organization and self-adaptation of processing resources in the system, where the computing resources are organized into distributed hierarchies according to a proposed hierarchical multi-layered resource description model. The proposed approach supports distributed query processing within and across hierarchical layers by deploying various distributed resource discovery services and functionalities in the system which are implemented using different adapted algorithms and mechanisms in each level of hierarchy. The proposed approach addresses the requirements for resource discovery in Jungle Computing environments such as high-hierarchy, high-heterogeneity, high-scalability and dynamicity. Simulation results show significant scalability and efficiency of the proposed approach over highly heterogeneous, hierarchical and dynamic computing environments

Novel Analytical Modelling-based Simulation of Worm Propagation in Unstructured Peer-to-Peer Networks

Millions of users world-wide are sharing content using Peer-to-Peer (P2P) networks, such as Skype and Bit Torrent. While such new innovations undoubtedly bring benefits, there are nevertheless some associated threats. One of the main hazards is that P2P worms can penetrate the network, even from a single node and then spread rapidly. Understanding the propagation process of such worms has always been a challenge for researchers. Different techniques, such as simulations and analytical models, have been adopted in the literature. While simulations provide results for specific input parameter values, analytical models are rather more general and potentially cover the whole spectrum of given parameter values. Many attempts have been made to model the worm propagation process in P2P networks. However, the reported analytical models to-date have failed to cover the whole spectrum of all relevant parameters and have therefore resulted in high false-positives. This consequently affects the immunization and mitigation strategies that are adopted to cope with an outbreak of worms. The first key contribution of this thesis is the development of a susceptible, exposed, infectious, and Recovered (SEIR) analytical model for the worm propagation process in a P2P network, taking into account different factors such as the configuration diversity of nodes, user behaviour and the infection time-lag. These factors have not been considered in an integrated form previously and have been either ignored or partially addressed in state-of-the-art analytical models. Our proposed SEIR analytical model holistically integrates, for the first time, these key factors in order to capture a more realistic representation of the whole worm propagation process. The second key contribution is the extension of the proposed SEIR model to the mobile M-SEIR model by investigating and incorporating the role of node mobility, the size of the worm and the bandwidth of wireless links in the worm propagation process in mobile P2P networks. The model was designed to be flexible and applicable to both wired and wireless nodes. The third contribution is the exploitation of a promising modelling paradigm, Agent-based Modelling (ABM), in the P2P worm modelling context. Specifically, to exploit the synergies between ABM and P2P, an integrated ABM-Based worm propagation model has been built and trialled in this research for the first time. The introduced model combines the implementation of common, complex P2P protocols, such as Gnutella and GIA, along with the aforementioned analytical models. Moreover, a comparative evaluation between ABM and conventional modelling tools has been carried out, to demonstrate the key benefits of ease of real-time analysis and visualisation. As a fourth contribution, the research was further extended by utilizing the proposed SEIR model to examine and evaluate a real-world data set on one of the most recent worms, namely, the Conficker worm. Verification of the model was achieved using ABM and conventional tools and by then comparing the results on the same data set with those derived from developed benchmark models. Finally, the research concludes that the worm propagation process is to a great extent affected by different factors such as configuration diversity, user-behaviour, the infection time lag and the mobility of nodes. It was found that the infection propagation values derived from state-of-the-art mathematical models are hypothetical and do not actually reflect real-world values. In summary, our comparative research study has shown that infection propagation can be reduced due to the natural immunity against worms that can be provided by a holistic exploitation of the range of factors proposed in this work