Overgrid: A Fully Distributed Demand Response Architecture Based on Overlay Networks

In this paper, we present Overgrid, a fully distributed peer-to-peer (P2P) architecture designed to automatically control and implement distributed demand response (DR) schemes in a community of smart buildings with energy generation and storage capabilities. As overlay networks in communications establish logical links between peers regardless of the physical topology of the network, the Overgrid is able to apply some power balance criteria to its system of buildings, as they belong to a virtual microgrid, regardless of their physical location. We exploit an innovative distributed algorithm, called flow updating, for monitoring the power consumption of the buildings and the number of nodes in the network, proving its applicability in an Overgrid scenario with realistic power profiles and networks of up to 10,000 buildings. To quantify the energy balance capability of Overgrid, we first study the energy characteristics of several types of buildings in our university campus and in an industrial site to accurately provide some reference buildings models. Then, we classify the amount of ``flexible'' energy consumption, i.e., the quota that could be potentially exploited for DR programs. Finally, we validate Overgrid emulating a real P2P network of smart buildings behaving according to our reference models. The experimental results prove the feasibility of our approach

Highly intensive data dissemination in complex networks

This paper presents a study on data dissemination in unstructured Peer-to-Peer (P2P) network overlays. The absence of a structure in unstructured overlays eases the network management, at the cost of non-optimal mechanisms to spread messages in the network. Thus, dissemination schemes must be employed that allow covering a large portion of the network with a high probability (e.g.~gossip based approaches). We identify principal metrics, provide a theoretical model and perform the assessment evaluation using a high performance simulator that is based on a parallel and distributed architecture. A main point of this study is that our simulation model considers implementation technical details, such as the use of caching and Time To Live (TTL) in message dissemination, that are usually neglected in simulations, due to the additional overhead they cause. Outcomes confirm that these technical details have an important influence on the performance of dissemination schemes and that the studied schemes are quite effective to spread information in P2P overlay networks, whatever their topology. Moreover, the practical usage of such dissemination mechanisms requires a fine tuning of many parameters, the choice between different network topologies and the assessment of behaviors such as free riding. All this can be done only using efficient simulation tools to support both the network design phase and, in some cases, at runtime

Statistical structures for internet-scale data management

Efficient query processing in traditional database management systems relies on statistics on base data. For centralized systems, there is a rich body of research results on such statistics, from simple aggregates to more elaborate synopses such as sketches and histograms. For Internet-scale distributed systems, on the other hand, statistics management still poses major challenges. With the work in this paper we aim to endow peer-to-peer data management over structured overlays with the power associated with such statistical information, with emphasis on meeting the scalability challenge. To this end, we first contribute efficient, accurate, and decentralized algorithms that can compute key aggregates such as Count, CountDistinct, Sum, and Average. We show how to construct several types of histograms, such as simple Equi-Width, Average-Shifted Equi-Width, and Equi-Depth histograms. We present a full-fledged open-source implementation of these tools for distributed statistical synopses, and report on a comprehensive experimental performance evaluation, evaluating our contributions in terms of efficiency, accuracy, and scalability

Scalable and Distributed Resource Management Protocols for Cloud and Big Data Clusters

Cloud data centers require an operating system to manage resources and satisfy operational requirements and management objectives. The growth of popularity in cloud services causes the appearance of a new spectrum of services with sophisticated workload and resource management requirements. Also, data centers are growing by addition of various type of hardware to accommodate the ever-increasing requests of users. Nowadays a large percentage of cloud resources are executing data-intensive applications which need continuously changing workload fluctuations and specific resource management. To this end, cluster computing frameworks are shifting towards distributed resource management for better scalability and faster decision making. Such systems benefit from the parallelization of control and are resilient to failures. Throughout this thesis we investigate algorithms, protocols and techniques to address these challenges in large-scale data centers. We introduce a distributed resource management framework which consolidates virtual machine to as few servers as possible to reduce the energy consumption of data center and hence decrease the cost of cloud providers. This framework can characterize the workload of virtual machines and hence handle trade-off energy consumption and Service Level Agreement (SLA) of customers efficiently. The algorithm is highly scalable and requires low maintenance cost with dynamic workloads and it tries to minimize virtual machines migration costs. We also introduce a scalable and distributed probe-based scheduling algorithm for Big data analytics frameworks. This algorithm can efficiently address the problem job heterogeneity in workloads that has appeared after increasing the level of parallelism in jobs. The algorithm is massively scalable and can reduce significantly average job completion times in comparison with the-state of-the-art. Finally, we propose a probabilistic fault-tolerance technique as part of the scheduling algorithm

Self-stabilized fast gossiping algorithms

In this article, we explore the topic of extending aggregate computation in distributed networks with selfstabilizing properties to withstand network dynamics. Existing research suggests that fast gossiping algorithms, based on the properties of order statistics applied to families of exponential random variables, are a viable solution for computing functions of the values stored in the network. We focus on the specific case in which network changes and failures occur in batches with a minimum frequency in the order of the diameter of the network. Our contribution consists in two self-stabilizing mechanisms, allowing fast gossiping algorithms to be applicable to dynamic networks with minor increase in resources usage. The resulting algorithms can be deployed in networks exhibiting churn, node stop-failures and resets, and random topological changes. The theoretical results are verified with simulations on synthetic data, showcasing desirable properties for large-scale network designers such as scalability, lack of single points of failure, and anonymity

Peer to Peer Information Retrieval: An Overview

Peer-to-peer technology is widely used for file sharing. In the past decade a number of prototype peer-to-peer information retrieval systems have been developed. Unfortunately, none of these have seen widespread real- world adoption and thus, in contrast with file sharing, information retrieval is still dominated by centralised solutions. In this paper we provide an overview of the key challenges for peer-to-peer information retrieval and the work done so far. We want to stimulate and inspire further research to overcome these challenges. This will open the door to the development and large-scale deployment of real-world peer-to-peer information retrieval systems that rival existing centralised client-server solutions in terms of scalability, performance, user satisfaction and freedom