Collocation Games and Their Application to Distributed Resource Management

We introduce Collocation Games as the basis of a general framework for modeling, analyzing, and facilitating the interactions between the various stakeholders in distributed systems in general, and in cloud computing environments in particular. Cloud computing enables fixed-capacity (processing, communication, and storage) resources to be offered by infrastructure providers as commodities for sale at a fixed cost in an open marketplace to independent, rational parties (players) interested in setting up their own applications over the Internet. Virtualization technologies enable the partitioning of such fixed-capacity resources so as to allow each player to dynamically acquire appropriate fractions of the resources for unencumbered use. In such a paradigm, the resource management problem reduces to that of partitioning the entire set of applications (players) into subsets, each of which is assigned to fixed-capacity cloud resources. If the infrastructure and the various applications are under a single administrative domain, this partitioning reduces to an optimization problem whose objective is to minimize the overall deployment cost. In a marketplace, in which the infrastructure provider is interested in maximizing its own profit, and in which each player is interested in minimizing its own cost, it should be evident that a global optimization is precisely the wrong framework. Rather, in this paper we use a game-theoretic framework in which the assignment of players to fixed-capacity resources is the outcome of a strategic "Collocation Game". Although we show that determining the existence of an equilibrium for collocation games in general is NP-hard, we present a number of simplified, practically-motivated variants of the collocation game for which we establish convergence to a Nash Equilibrium, and for which we derive convergence and price of anarchy bounds. In addition to these analytical results, we present an experimental evaluation of implementations of some of these variants for cloud infrastructures consisting of a collection of multidimensional resources of homogeneous or heterogeneous capacities. Experimental results using trace-driven simulations and synthetically generated datasets corroborate our analytical results and also illustrate how collocation games offer a feasible distributed resource management alternative for autonomic/self-organizing systems, in which the adoption of a global optimization approach (centralized or distributed) would be neither practical nor justifiable.NSF (CCF-0820138, CSR-0720604, EFRI-0735974, CNS-0524477, CNS-052016, CCR-0635102); Universidad Pontificia Bolivariana; COLCIENCIAS–Instituto Colombiano para el Desarrollo de la Ciencia y la Tecnología "Francisco José de Caldas

LCC : Un réseau de recouvrement multipoint passant à l'échelle

Les propositions récentes de construction de réseaux de recouvrement (overlay) pour la transmission multipoint ont démontré l'importance d'exploiter les informations de la topologie du réseau sous-jacent. Toutefois, ces propositions reposent souvent sur des processus de raffinement incrémentaux pour améliorer les performances du système. Ces approches ne passent pas de ce fait à l'échelle, induisant un surcoût de communication élevé, et nécessitent un temps de convergence important avant d'atteindre une structure stable. Dans cet article, nous proposons un algorithme de localisation qui dirige graduellement les nouveaux venus vers les noeuds les plus proches sans pour cela induire un surcoût élevé. En nous basant sur cette localisation, nous proposons un réseau overlay, nommé "LCC", à la fois robuste, passant à l'échelle et prenant en compte la topologie physique. Nous avons mené des expérimentations réelles sur PlanetLab ainsi que des simulations afin d'évaluer les performances de LCC. Les résultats prouvent d'une part que le processus de localisation nécessite des ressources modestes en termes de délais et de bande passante, et d'autre part que LCC est un overlay efficace pour supporter des applications multipoints à très grande échelle

QoS monitoring in real-time streaming overlays based on lock-free data structures

AbstractPeer-to-peer streaming is a well-known technology for the large-scale distribution of real-time audio/video contents. Delay requirements are very strict in interactive real-time scenarios (such as synchronous distance learning), where playback lag should be of the order of seconds. Playback continuity is another key aspect in these cases: in presence of peer churning and network congestion, a peer-to-peer overlay should quickly rearrange connections among receiving nodes to avoid freezing phenomena that may compromise audio/video understanding. For this reason, we designed a QoS monitoring algorithm that quickly detects broken or congested links: each receiving node is able to independently decide whether it should switch to a secondary sending node, called "fallback node". The architecture takes advantage of a multithreaded design based on lock-free data structures, which improve the performance by avoiding synchronization among threads. We will show the good responsiveness of the proposed approach on machines with different computational capabilities: measured times prove both departures of nodes and QoS degradations are promptly detected and clients can quickly restore a stream reception. According to PSNR and SSIM, two well-known full-reference video quality metrics, QoE remains acceptable on receiving nodes of our resilient overlay also in presence of swap procedures

A decentralized framework for cross administrative domain data sharing

Federation of messaging and storage platforms located in remote datacenters is an essential functionality to share data among geographically distributed platforms. When systems are administered by the same owner data replication reduces data access latency bringing data closer to applications and enables fault tolerance to face disaster recovery of an entire location. When storage platforms are administered by different owners data replication across different administrative domains is essential for enterprise application data integration. Contents and services managed by different software platforms need to be integrated to provide richer contents and services. Clients may need to share subsets of data in order to enable collaborative analysis and service integration. Platforms usually include proprietary federation functionalities and specific APIs to let external software and platforms access their internal data. These different techniques may not be applicable to all environments and networks due to security and technological restrictions. Moreover the federation of dispersed nodes under a decentralized administration scheme is still a research issue. This thesis is a contribution along this research direction as it introduces and describes a framework, called \u201cWideGroups\u201d, directed towards the creation and the management of an automatic federation and integration of widely dispersed platform nodes. It is based on groups to exchange messages among distributed applications located in different remote datacenters. Groups are created and managed using client side programmatic configuration without touching servers. WideGroups enables the extension of the software platform services to nodes belonging to different administrative domains in a wide area network environment. It lets different nodes form ad-hoc overlay networks on-the-fly depending on message destinations located in distinct administrative domains. It supports multiple dynamic overlay networks based on message groups, dynamic discovery of nodes and automatic setup of overlay networks among nodes with no server-side configuration. I designed and implemented platform connectors to integrate the framework as the federation module of Message Oriented Middleware and Key Value Store platforms, which are among the most widespread paradigms supporting data sharing in distributed systems

Improvements in distribution of meteorological data using application layer multicast

The Unidata Program Center is an organization working with the University Center for Atmospheric Research (UCAR), in Colorado. It provides a broad variety of meteorological data, which is used by researchers in many real-world applications. This data is obtained from observation stations and distributed to various universities worldwide, using UnidataâÂÂs own Internet Data Distribution (IDD) system, and software called the Local Data Manager (LDM). The existing solution for data distribution has many limitations, like high end-toend latency of data delivery, increased bandwidth usage at some nodes, poor scalability for future needs and manual intervention for adjusting to changes or faults in the network topology. Since the data is used in so many applications, the impact of these limitations is often substantial. This thesis removes these limitations by suggesting improvements in the IDD system and the LDM. We present new algorithms for constructing an application-layer data distribution network. This distribution network will form the basis of the improved LDM and the IDD system, and will remove most of the limitations given above. Finally, we perform simulations and show that our algorithms achieve better average end-to-end latency as compared to that of the existing solution. We also compare the performance of our algorithms with a randomized solution. We find that for smaller topologies (where the number of nodes in the system are less than 38) the randomized solution constructs efficient distribution networks. However, if the number of nodes in the system increases (more than 38), our solution constructs efficient distribution networks than the randomized solution. We also evaluate the performance of our algorithms as the number of nodes in the system increases and as the number of faults in the system increases. We find that even if the number of faults in the system increases, the average end-to-end latency decreases, thus showing that the distribution topology does not become inefficient

Optimization based methods for solving some problems in telecommunications and the internet

The purpose of this thesis is to develop some new algorithms based on optimization techniques for solving some problems in some areas of telecommunications and the Internet. There are two main parts to this thesis. In the first part we discuss optimization based stochastic and queueing models in telecommunications network corrective maintenance. In the second part we develop optimization based clustering (OBC) algorithms for network evolution and multicast routing. The most typical scenario encountered during mathematical optimization modelling in telecommunications, for example, is to minimize the cost of establishment and maintenance of the networks subject to the performance constraints of the networks and the reliability constraints of the networks as well. Most of these optimization problems are global optimization, that is, they have many local minima and most of these local minima do not provide any useful information for solving these problems. Therefore, the development of effective methods for solving such global optimization problems is important. To run the telecommunications networks with cost-effective network maintenance,we need to establish a practical maintenance model and optimize it. In the first part of the thesis, we solve a known stochastic programming maintenance optimization model with a direct method and then develop some new models. After that we introduce queue programming models in telecommunications network maintenance optimization. The ideas of profit, loss, and penalty will help telecommunications companies have a good view of their maintenance policies and help them improve their service. In the second part of this thesis we propose the use of optimization based clustering (OBC) algorithms to determine level-constrained hierarchical trees for network evolution and multicast routing. This problem is formulated as an optimization problem with a non-smooth, non-convex objective function. Different algorithms are examined for solving this problem. Results of numerical experiments using some artifiicial and real-world databases are reported.Doctor of Philosoph

Constructing efficient self-organising application layer multicast overlays

This thesis investigates efficient techniques to build both low cost (i.e. low resource usage) and low delay ALM trees. We focus on self-organising distributed proposals that use limited information about the underlying physical network, limited coordination between the members, and construct overlays with bounded branching degree subject to the bandwidth constraint of each individual member

Constructing efficient self-organising application layer multicast overlays

EThOS - Electronic Theses Online ServiceGBUnited Kingdo