MultiGreen: Cost-Minimizing Multi-source Datacenter Power Supply with Online Control

Session 4: Data Center Energy ManagementFulltext of the conference paper in: http://conferences.sigcomm.org/eenergy/2013/papers/p13.pdfFaced by soaring power cost, large footprint of carbon emis- sion and unpredictable power outage, more and more mod- ern Cloud Service Providers (CSPs) begin to mitigate these challenges by equipping their Datacenter Power Supply Sys- tem (DPSS) with multiple sources: (1) smart grid with time- varying electricity prices, (2) uninterrupted power supply (UPS) of finite capacity, and (3) intermittent green or re- newable energy. It remains a significant challenge how to operate among multiple power supply sources in a comple- mentary manner, to deliver reliable energy to datacenter users over time, while minimizing a CSP’s operational cost over the long run. This paper proposes an efficient, online control algorithm for DPSS, called MultiGreen. MultiGreen is based on an innovative two-timescale Lyapunov optimiza- tion technique. Without requiring a priori knowledge of system statistics, MultiGreen allows CSPs to make online decisions on purchasing grid energy at two time scales (in the long-term market and in the real-time market), leveraging renewable energy, and opportunistically charging and dis- charging UPS, in order to fully leverage the available green energy and low electricity prices at times for minimum op- erational cost. Our detailed analysis and trace-driven sim- ulations based on one-month real-world data have demon- strated the optimality (in terms of the tradeoff between min- imization of DPSS operational cost and satisfaction of data- center availability) and stability (performance guarantee in cases of fluctuating energy demand and supply) of Multi- Green

Datacenter management for on-site intermittent and uncertain renewable energy sources

Les technologies de l'information et de la communication sont devenues, au cours des dernières années, un pôle majeur de consommation énergétique avec les conséquences environnementales associées. Dans le même temps, l'émergence du Cloud computing et des grandes plateformes en ligne a causé une augmentation en taille et en nombre des centres de données. Pour réduire leur impact écologique, alimenter ces centres avec des sources d'énergies renouvelables (EnR) apparaît comme une piste de solution. Cependant, certaines EnR telles que les énergies solaires et éoliennes sont liées aux conditions météorologiques, et sont par conséquent intermittentes et incertaines. L'utilisation de batteries ou d'autres dispositifs de stockage est souvent envisagée pour compenser ces variabilités de production. De par leur coût important, économique comme écologique, ainsi que les pertes énergétiques engendrées, l'utilisation de ces dispositifs sans intégration supplémentaire est insuffisante. La consommation électrique d'un centre de données dépend principalement de l'utilisation des ressources de calcul et de communication, qui est déterminée par la charge de travail et les algorithmes d'ordonnancement utilisés. Pour utiliser les EnR efficacement tout en préservant la qualité de service du centre, une gestion coordonnée des ressources informatiques, des sources électriques et du stockage est nécessaire. Il existe une grande diversité de centres de données, ayant différents types de matériel, de charge de travail et d'utilisation. De la même manière, suivant les EnR, les technologies de stockage et les objectifs en termes économiques ou environnementaux, chaque infrastructure électrique est modélisée et gérée différemment des autres. Des travaux existants proposent des méthodes de gestion d'EnR pour des couples bien spécifiques de modèles électriques et informatiques. Cependant, les multiples combinaisons de ces deux parties rendent difficile l'extrapolation de ces approches et de leurs résultats à des infrastructures différentes. Cette thèse explore de nouvelles méthodes pour résoudre ce problème de coordination. Une première contribution reprend un problème d'ordonnancement de tâches en introduisant une abstraction des sources électriques. Un algorithme d'ordonnancement est proposé, prenant les préférences des sources en compte, tout en étant conçu pour être indépendant de leur nature et des objectifs de l'infrastructure électrique. Une seconde contribution étudie le problème de planification de l'énergie d'une manière totalement agnostique des infrastructures considérées. Les ressources informatiques et la gestion de la charge de travail sont encapsulées dans une boîte noire implémentant un ordonnancement sous contrainte de puissance. La même chose s'applique pour le système de gestion des EnR et du stockage, qui agit comme un algorithme d'optimisation d'engagement de sources pour répondre à une demande. Une optimisation coopérative et multiobjectif, basée sur un algorithme évolutionnaire, utilise ces deux boîtes noires afin de trouver les meilleurs compromis entre les objectifs électriques et informatiques. Enfin, une troisième contribution vise les incertitudes de production des EnR pour une infrastructure plus spécifique. En utilisant une formulation en processus de décision markovien (MDP), la structure du problème de décision sous-jacent est étudiée. Pour plusieurs variantes du problème, des méthodes sont proposées afin de trouver les politiques optimales ou des approximations de celles-ci avec une complexité raisonnable.In recent years, information and communication technologies (ICT) became a major energy consumer, with the associated harmful ecological consequences. Indeed, the emergence of Cloud computing and massive Internet companies increased the importance and number of datacenters around the world. In order to mitigate economical and ecological cost, powering datacenters with renewable energy sources (RES) began to appear as a sustainable solution. Some of the commonly used RES, such as solar and wind energies, directly depends on weather conditions. Hence they are both intermittent and partly uncertain. Batteries or other energy storage devices (ESD) are often considered to relieve these issues, but they result in additional energy losses and are too costly to be used alone without more integration. The power consumption of a datacenter is closely tied to the computing resource usage, which in turn depends on its workload and on the algorithms that schedule it. To use RES as efficiently as possible while preserving the quality of service of a datacenter, a coordinated management of computing resources, electrical sources and storage is required. A wide variety of datacenters exists, each with different hardware, workload and purpose. Similarly, each electrical infrastructure is modeled and managed uniquely, depending on the kind of RES used, ESD technologies and operating objectives (cost or environmental impact). Some existing works successfully address this problem by considering a specific couple of electrical and computing models. However, because of this combined diversity, the existing approaches cannot be extrapolated to other infrastructures. This thesis explores novel ways to deal with this coordination problem. A first contribution revisits batch tasks scheduling problem by introducing an abstraction of the power sources. A scheduling algorithm is proposed, taking preferences of electrical sources into account, though designed to be independent from the type of sources and from the goal of the electrical infrastructure (cost, environmental impact, or a mix of both). A second contribution addresses the joint power planning coordination problem in a totally infrastructure-agnostic way. The datacenter computing resources and workload management is considered as a black-box implementing a scheduling under variable power constraint algorithm. The same goes for the electrical sources and storage management system, which acts as a source commitment optimization algorithm. A cooperative multiobjective power planning optimization, based on a multi-objective evolutionary algorithm (MOEA), dialogues with the two black-boxes to find the best trade-offs between electrical and computing internal objectives. Finally, a third contribution focuses on RES production uncertainties in a more specific infrastructure. Based on a Markov Decision Process (MDP) formulation, the structure of the underlying decision problem is studied. For several variants of the problem, tractable methods are proposed to find optimal policies or a bounded approximation

Contributions to energy-aware demand-response systems using SDN and NFV for fog computing

Ever-increasing energy consumption, the depletion of non-renewable resources, the climate impact associated with energy generation, and finite energy-production capacity are important concerns worldwide that drive the urgent creation of new energy management and consumption schemes. In this regard, by leveraging the massive connectivity provided by emerging communications such as the 5G systems, this thesis proposes a long-term sustainable Demand-Response solution for the adaptive and efficient management of available energy consumption for Internet of Things (IoT) infrastructures, in which energy utilization is optimized based on the available supply. In the proposed approach, energy management focuses on consumer devices (e.g., appliances such as a light bulb or a screen). In this regard, by proposing that each consumer device be part of an IoT infrastructure, it is feasible to control its respective consumption. The proposal includes an architecture that uses Network Functions Virtualization (NFV) and Software Defined Networking technologies as enablers to promote the primary use of energy from renewable sources. Associated with architecture, this thesis presents a novel consumption model conditioned on availability in which consumers are part of the management process. To efficiently use the energy from renewable and non-renewable sources, several management strategies are herein proposed, such as the prioritization of the energy supply, workload scheduling using time-shifting capabilities, and quality degradation to decrease- the power demanded by consumers if needed. The adaptive energy management solution is modeled as an Integer Linear Programming, and its complexity has been identified to be NP-Hard. To verify the improvements in energy utilization, an optimal algorithmic solution based on a brute force search has been implemented and evaluated. Because the hardness of the adaptive energy management problem and the non-polynomial growth of its optimal solution, which is limited to energy management for a small number of energy demands (e.g., 10 energy demands) and small values of management mechanisms, several faster suboptimal algorithmic strategies have been proposed and implemented. In this context, at the first stage, we implemented three heuristic strategies: a greedy strategy (GreedyTs), a genetic-algorithm-based solution (GATs), and a dynamic programming approach (DPTs). Then, we incorporated into both the optimal and heuristic strategies a prepartitioning method in which the total set of analyzed services is divided into subsets of smaller size and complexity that are solved iteratively. As a result of the adaptive energy management in this thesis, we present eight strategies, one timal and seven heuristic, that when deployed in communications infrastructures such as the NFV domain, seek the best possible scheduling of demands, which lead to efficient energy utilization. The performance of the algorithmic strategies has been validated through extensive simulations in several scenarios, demonstrating improvements in energy consumption and the processing of energy demands. Additionally, the simulation results revealed that the heuristic approaches produce high-quality solutions close to the optimal while executing among two and seven orders of magnitude faster and with applicability to scenarios with thousands and hundreds of thousands of energy demands. This thesis also explores possible application scenarios of both the proposed architecture for adaptive energy management and algorithmic strategies. In this regard, we present some examples, including adaptive energy management in-home systems and 5G networks slicing, energy-aware management solutions for unmanned aerial vehicles, also known as drones, and applicability for the efficient allocation of spectrum in flex-grid optical networks. Finally, this thesis presents open research problems and discusses other application scenarios and future work.El constante aumento del consumo de energía, el agotamiento de los recursos no renovables, el impacto climático asociado con la generación de energía y la capacidad finita de producción de energía son preocupaciones importantes en todo el mundo que impulsan la creación urgente de nuevos esquemas de consumo y gestión de energía. Al aprovechar la conectividad masiva que brindan las comunicaciones emergentes como los sistemas 5G, esta tesis propone una solución de Respuesta a la Demanda sostenible a largo plazo para la gestión adaptativa y eficiente del consumo de energía disponible para las infraestructuras de Internet of Things (IoT), en el que se optimiza la utilización de la energía en función del suministro disponible. En el enfoque propuesto, la gestión de la energía se centra en los dispositivos de consumo (por ejemplo, electrodomésticos). En este sentido, al proponer que cada dispositivo de consumo sea parte de una infraestructura IoT, es factible controlar su respectivo consumo. La propuesta incluye una arquitectura que utiliza tecnologías de Network Functions Virtualization (NFV) y Software Defined Networking como habilitadores para promover el uso principal de energía de fuentes renovables. Asociada a la arquitectura, esta tesis presenta un modelo de consumo condicionado a la disponibilidad en el que los consumidores son parte del proceso de gestión. Para utilizar eficientemente la energía de fuentes renovables y no renovables, se proponen varias estrategias de gestión, como la priorización del suministro de energía, la programación de la carga de trabajo utilizando capacidades de cambio de tiempo y la degradación de la calidad para disminuir la potencia demandada. La solución de gestión de energía adaptativa se modela como un problema de programación lineal entera con complejidad NP-Hard. Para verificar las mejoras en la utilización de energía, se ha implementado y evaluado una solución algorítmica óptima basada en una búsqueda de fuerza bruta. Debido a la dureza del problema de gestión de energía adaptativa y el crecimiento no polinomial de su solución óptima, que se limita a la gestión de energía para un pequeño número de demandas de energía (por ejemplo, 10 demandas) y pequeños valores de los mecanismos de gestión, varias estrategias algorítmicas subóptimos más rápidos se han propuesto. En este contexto, en la primera etapa, implementamos tres estrategias heurísticas: una estrategia codiciosa (GreedyTs), una solución basada en algoritmos genéticos (GATs) y un enfoque de programación dinámica (DPTs). Luego, incorporamos tanto en la estrategia óptima como en la- heurística un método de prepartición en el que el conjunto total de servicios analizados se divide en subconjuntos de menor tamaño y complejidad que se resuelven iterativamente. Como resultado de la gestión adaptativa de la energía en esta tesis, presentamos ocho estrategias, una óptima y siete heurísticas, que cuando se despliegan en infraestructuras de comunicaciones como el dominio NFV, buscan la mejor programación posible de las demandas, que conduzcan a un uso eficiente de la energía. El desempeño de las estrategias algorítmicas ha sido validado a través de extensas simulaciones en varios escenarios, demostrando mejoras en el consumo de energía y el procesamiento de las demandas de energía. Los resultados de la simulación revelaron que los enfoques heurísticos producen soluciones de alta calidad cercanas a las óptimas mientras se ejecutan entre dos y siete órdenes de magnitud más rápido y con aplicabilidad a escenarios con miles y cientos de miles de demandas de energía. Esta tesis también explora posibles escenarios de aplicación tanto de la arquitectura propuesta para la gestión adaptativa de la energía como de las estrategias algorítmicas. En este sentido, presentamos algunos ejemplos, que incluyen sistemas de gestión de energía adaptativa en el hogar, en 5G networkPostprint (published version

A manifesto for future generation cloud computing: research directions for the next decade

The Cloud computing paradigm has revolutionised the computer science horizon during the past decade and has enabled the emergence of computing as the fifth utility. It has captured significant attention of academia, industries, and government bodies. Now, it has emerged as the backbone of modern economy by offering subscription-based services anytime, anywhere following a pay-as-you-go model. This has instigated (1) shorter establishment times for start-ups, (2) creation of scalable global enterprise applications, (3) better cost-to-value associativity for scientific and high performance computing applications, and (4) different invocation/execution models for pervasive and ubiquitous applications. The recent technological developments and paradigms such as serverless computing, software-defined networking, Internet of Things, and processing at network edge are creating new opportunities for Cloud computing. However, they are also posing several new challenges and creating the need for new approaches and research strategies, as well as the re-evaluation of the models that were developed to address issues such as scalability, elasticity, reliability, security, sustainability, and application models. The proposed manifesto addresses them by identifying the major open challenges in Cloud computing, emerging trends, and impact areas. It then offers research directions for the next decade, thus helping in the realisation of Future Generation Cloud Computing