Coalition Formation and Combinatorial Auctions; Applications to Self-organization and Self-management in Utility Computing

In this paper we propose a two-stage protocol for resource management in a hierarchically organized cloud. The first stage exploits spatial locality for the formation of coalitions of supply agents; the second stage, a combinatorial auction, is based on a modified proxy-based clock algorithm and has two phases, a clock phase and a proxy phase. The clock phase supports price discovery; in the second phase a proxy conducts multiple rounds of a combinatorial auction for the package of services requested by each client. The protocol strikes a balance between low-cost services for cloud clients and a decent profit for the service providers. We also report the results of an empirical investigation of the combinatorial auction stage of the protocol.Comment: 14 page

Resource Management in Large-scale Systems

The focus of this thesis is resource management in large-scale systems. Our primary concerns are energy management and practical principles for self-organization and self-management. The main contributions of our work are: 1. Models. We proposed several models for different aspects of resource management, e.g., energy-aware load balancing and application scaling for the cloud ecosystem, hierarchical architecture model for self-organizing and self-manageable systems and a new cloud delivery model based on auction-driven self-organization approach. 2. Algorithms. We also proposed several different algorithms for the models described above. Algorithms such as coalition formation, combinatorial auctions and clustering algorithm for scale-free organizations of scale-free networks. 3. Evaluation. Eventually we conducted different evaluations for the proposed models and algorithms in order to verify them. All the simulations reported in this thesis had been carried out on different instances and services of Amazon Web Services (AWS). All of these modules will be discussed in detail in the following chapters respectively

A cloud reservation system for big data applications

Emerging Big Data applications increasingly require resources beyond those available from a single server and may be expressed as a complex workflow of many components and dependency relationships-each component potentially requiring its own specific, and perhaps specialized, resources for its execution. Efficiently supporting this type of Big Data application is a challenging resource management problem for existing cloud environments. In response, we propose a two-stage protocol for solving this resource management problem. We exploit spatial locality in the first stage by dynamically forming rack-level coalitions of servers to execute a workflow component. These coalitions only exist for the duration of the execution of their assigned component and are subsequently disbanded, allowing their resources to take part in future coalitions. The second stage creates a package of these coalitions, designed to support all the components in the complete workflow. To minimize the communication and housekeeping overhead needed to form this package of coalitions, the technique of combinatorial auctions is adapted from market-based resource allocation. This technique has a considerably lower overhead for resource aggregation than the traditional hierarchically organized models. We analyze two strategies for coalition formation: the first, history-based uses information from past auctions to pre-form coalitions in anticipation of predicted demand; the second one is a just-in-time-that builds coalitions only when support for specific workflow components is requested

Mechanism design for distributed task and resource allocation among self-interested agents in virtual organizations

The aggregate power of all resources on the Internet is enormous. The Internet can be viewed as a massive virtual organization that holds tremendous amounts of information and resources with different ownerships. However, little is known about how to run this organization efficiently. This dissertation studies the problems of distributed task and resource allocation among self-interested agents in virtual organizations. The developed solutions are not allocation mechanisms that can be imposed by a centralized designer, but decentralized interaction mechanisms that provide incentives to self-interested agents to behave cooperatively. These mechanisms also take computational tractability into consideration due to the inherent complexity of distributed task and resource allocation problems. Targeted allocation mechanisms can achieve global task allocation efficiency in a virtual organization and establish stable resource-sharing communities based on agentsâÃÂàown decisions about whether or not to behave cooperatively. This high level goal requires solving the following problems: synthetic task allocation, decentralized coalition formation and automated multiparty negotiation. For synthetic task allocation, in which each task needs to be accomplished by a virtual team composed of self-interested agents from different real organizations, my approach is to formalize the synthetic task allocation problem as an algorithmic mechanism design optimization problem. I have developed two approximation mechanisms that I prove are incentive compatible for a synthetic task allocation problem. This dissertation also develops a decentralized coalition formation mechanism, which is based on explicit negotiation among self-interested agents. Each agent makes its own decisions about whether or not to join a candidate coalition. The resulting coalitions are stable in the core in terms of coalition rationality. I have applied this mechanism to form resource sharing coalitions in computational grids and buyer coalitions in electronic markets. The developed negotiation mechanism in the decentralized coalition formation mechanism realizes automated multilateral negotiation among self-interested agents who have symmetric authority (i.e., no mediator exists and agents are peers). In combination, the decentralized allocation mechanisms presented in this dissertation lay a foundation for realizing automated resource management in open and scalable virtual organizations

Value Creation through Co-Opetition in Service Networks

Well-defined interfaces and standardization allow for the composition of single Web services into value-added complex services. Such complex Web Services are increasingly traded via agile marketplaces, facilitating flexible recombination of service modules to meet heterogeneous customer demands. In order to coordinate participants, this work introduces a mechanism design approach - the co-opetition mechanism - that is tailored to requirements imposed by a networked and co-opetitive environment

Mechanism design for distributed task and resource allocation among self-interested agents in virtual organizations

The aggregate power of all resources on the Internet is enormous. The Internet can be viewed as a massive virtual organization that holds tremendous amounts of information and resources with different ownerships. However, little is known about how to run this organization efficiently. This dissertation studies the problems of distributed task and resource allocation among self-interested agents in virtual organizations. The developed solutions are not allocation mechanisms that can be imposed by a centralized designer, but decentralized interaction mechanisms that provide incentives to self-interested agents to behave cooperatively. These mechanisms also take computational tractability into consideration due to the inherent complexity of distributed task and resource allocation problems. Targeted allocation mechanisms can achieve global task allocation efficiency in a virtual organization and establish stable resource-sharing communities based on agentsâÃÂàown decisions about whether or not to behave cooperatively. This high level goal requires solving the following problems: synthetic task allocation, decentralized coalition formation and automated multiparty negotiation. For synthetic task allocation, in which each task needs to be accomplished by a virtual team composed of self-interested agents from different real organizations, my approach is to formalize the synthetic task allocation problem as an algorithmic mechanism design optimization problem. I have developed two approximation mechanisms that I prove are incentive compatible for a synthetic task allocation problem. This dissertation also develops a decentralized coalition formation mechanism, which is based on explicit negotiation among self-interested agents. Each agent makes its own decisions about whether or not to join a candidate coalition. The resulting coalitions are stable in the core in terms of coalition rationality. I have applied this mechanism to form resource sharing coalitions in computational grids and buyer coalitions in electronic markets. The developed negotiation mechanism in the decentralized coalition formation mechanism realizes automated multilateral negotiation among self-interested agents who have symmetric authority (i.e., no mediator exists and agents are peers). In combination, the decentralized allocation mechanisms presented in this dissertation lay a foundation for realizing automated resource management in open and scalable virtual organizations