A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce

Nowadays many companies have available large amounts of raw, unstructured data. Among Big Data enabling technologies, a central place is held by the MapReduce framework and, in particular, by its open source implementation, Apache Hadoop. For cost effectiveness considerations, a common approach entails sharing server clusters among multiple users. The underlying infrastructure should provide every user with a fair share of computational resources, ensuring that Service Level Agreements (SLAs) are met and avoiding wastes. In this paper we consider two mathematical programming problems that model the optimal allocation of computational resources in a Hadoop 2.x cluster with the aim to develop new capacity allocation techniques that guarantee better performance in shared data centers. Our goal is to get a substantial reduction of power consumption while respecting the deadlines stated in the SLAs and avoiding penalties associated with job rejections. The core of this approach is a distributed algorithm for runtime capacity allocation, based on Game Theory models and techniques, that mimics the MapReduce dynamics by means of interacting players, namely the central Resource Manager and Class Managers

Coalition Formation and Combinatorial Auctions; Applications to Self-organization and Self-management in Utility Computing

In this paper we propose a two-stage protocol for resource management in a hierarchically organized cloud. The first stage exploits spatial locality for the formation of coalitions of supply agents; the second stage, a combinatorial auction, is based on a modified proxy-based clock algorithm and has two phases, a clock phase and a proxy phase. The clock phase supports price discovery; in the second phase a proxy conducts multiple rounds of a combinatorial auction for the package of services requested by each client. The protocol strikes a balance between low-cost services for cloud clients and a decent profit for the service providers. We also report the results of an empirical investigation of the combinatorial auction stage of the protocol.Comment: 14 page

Wireless Communications in the Era of Big Data

The rapidly growing wave of wireless data service is pushing against the boundary of our communication network's processing power. The pervasive and exponentially increasing data traffic present imminent challenges to all the aspects of the wireless system design, such as spectrum efficiency, computing capabilities and fronthaul/backhaul link capacity. In this article, we discuss the challenges and opportunities in the design of scalable wireless systems to embrace such a "bigdata" era. On one hand, we review the state-of-the-art networking architectures and signal processing techniques adaptable for managing the bigdata traffic in wireless networks. On the other hand, instead of viewing mobile bigdata as a unwanted burden, we introduce methods to capitalize from the vast data traffic, for building a bigdata-aware wireless network with better wireless service quality and new mobile applications. We highlight several promising future research directions for wireless communications in the mobile bigdata era.Comment: This article is accepted and to appear in IEEE Communications Magazin

Optimal association of mobile users to multi-access edge computing resources

Multi-access edge computing (MEC) plays a key role in fifth-generation (5G) networks in bringing cloud functionalities at the edge of the radio access network, in close proximity to mobile users. In this paper we focus on mobile-edge computation offloading, a way to transfer heavy demanding, and latency-critical applications from mobile handsets to close-located MEC servers, in order to reduce latency and/or energy consumption. Our goal is to provide an optimal strategy to associate mobile users to access points (AP) and MEC hosts, while contextually optimizing the allocation of radio and computational resources to each user, with the objective of minimizing the overall user transmit power under latency constraints incorporating both communication and computation times. The overall problem is a mixed-binary problem. To overcome its inherent computational complexity, we propose two alternative strategies: i) a method based on successive convex approximation (SCA) techniques, proven to converge to local optimal solutions; ii) an approach hinging on matching theory, based on formulating the assignment problem as a matching game

A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce

open4noNowadays many companies have available large amounts of raw, unstructured data. Among Big Data enabling technologies, a central place is held by the MapReduce framework and, in particular, by its open source implementation, Apache Hadoop. For cost effectiveness considerations, a common approach entails sharing server clusters among multiple users. The underlying infrastructure should provide every user with a fair share of computational resources, ensuring that service level agreements (SLAs) are met and avoiding wastes. In this paper we consider mathematical models for the optimal allocation of computational resources in a Hadoop 2.x cluster with the aim to develop new capacity allocation techniques that guarantee better performance in shared data centers. Our goal is to get a substantial reduction of power consumption while respecting the deadlines stated in the SLAs and avoiding penalties associated with job rejections. The core of this approach is a distributed algorithm for runtime capacity allocation, based on Game Theory models and techniques, that mimics the MapReduce dynamics by means of interacting players, namely the central Resource Manager and Class Managers.Gianniti, Eugenio; Ardagna, Danilo; Ciavotta, Michele; Passacantando, MauroGianniti, Eugenio; Ardagna, Danilo; Ciavotta, Michele; Passacantando, Maur