Improving energy efficiency of virtualized datacenters

Nowadays, many organizations choose to increasingly implement the cloud computing approach. More specifically, as customers, these organizations are outsourcing the management of their physical infrastructure to data centers (or cloud computing platforms). Energy consumption is a primary concern for datacenter (DC) management. Its cost represents about 80% of the total cost of ownership and it is estimated that in 2020, the US DCs alone will spend about $13 billion on energy bills. Generally, the datacenter servers are manufactured in such a way that they achieve high energy efficiency at high utilizations. Thereby for a low cost per computation all datacenter servers should push the utilization as high as possible. In order to fight the historically low utilization, cloud computing adopted server virtualization. The latter allows a physical server to execute multiple virtual servers (called virtual machines) in an isolated way. With virtualization, the cloud provider can pack (consolidate) the entire set of virtual machines (VMs) on a small set of physical servers and thereby, reduce the number of active servers. Even so, the datacenter servers rarely reach utilizations higher than 50% which means that they operate with sets of longterm unused resources (called 'holes'). My first contribution is a cloud management system that dynamically splits/fusions VMs such that they can better fill the holes. This solution is effective only for elastic applications, i.e. applications that can be executed and reconfigured over an arbitrary number of VMs. However the datacenter resource fragmentation stems from a more fundamental problem. Over time, cloud applications demand more and more memory but the physical servers provide more an more CPU. In nowadays datacenters, the two resources are strongly coupled since they are bounded to a physical sever. My second contribution is a practical way to decouple the CPU-memory tuple that can simply be applied to a commodity server. Thereby, the two resources can vary independently, depending on their demand. My third and my forth contribution show a practical system which exploit the second contribution. The underutilization observed on physical servers is also true for virtual machines. It has been shown that VMs consume only a small fraction of the allocated resources because the cloud customers are not able to correctly estimate the resource amount necessary for their applications. My third contribution is a system that estimates the memory consumption (i.e. the working set size) of a VM, with low overhead and high accuracy. Thereby, we can now consolidate the VMs based on their working set size (not the booked memory). However, the drawback of this approach is the risk of memory starvation. If one or multiple VMs have an sharp increase in memory demand, the physical server may run out of memory. This event is undesirable because the cloud platform is unable to provide the client with the booked memory. My fourth contribution is a system that allows a VM to use remote memory provided by a different rack server. Thereby, in the case of a peak memory demand, my system allows the VM to allocate memory on a remote physical server

Améliorer l'efficacité énergétique des datacenters virtualisés

De nos jours, de nombreuses entreprises choisissent de plus en plus d'adopter le cloud computing. Plus précisément, en tant que clients, elles externalisent la gestion de leur infrastructure physique vers des centres de données (ou plateformes de cloud computing). La consommation d'énergie est une préoccupation majeure pour la gestion des centres de données (datacenter, DC). Son impact financier représente environ 80% du coût total de possession et l'on estime qu'en 2020, les DCs américains dépenseront à eux seuls environ 13 milliards de dollars en factures énergétiques. Généralement, les serveurs de centres de données sont conçus de manière à atteindre une grande efficacité énergétique pour des utilisations élevées. Pour diminuer le coût de calcul, les serveurs de centre de données devraient maximiser leur utilisation. Afin de lutter contre l'utilisation historiquement faible des serveurs, le cloud computing a adopté la virtualisation des serveurs. Cette dernière permet à un serveur physique d'exécuter plusieurs serveurs virtuels (appelés machines virtuelles) de manière isolée. Avec la virtualisation, le fournisseur de cloud peut regrouper (consolider) l'ensemble des machines virtuelles (VM) sur un ensemble réduit de serveurs physiques et ainsi réduire le nombre de serveurs actifs. Même ainsi, les serveurs de centres de données atteignent rarement des utilisations supérieures à 50%, ce qui signifie qu'ils fonctionnent avec des ensembles de ressources majoritairement inutilisées (appelés «trous»). Ma première contribution est un système de gestion de cloud qui divise ou fusionne dynamiquement les machines virtuelles de sorte à ce qu'elles puissent mieux remplir les trous. Cette solution n'est efficace que pour des applications élastiques, c'est-à-dire des applications qui peuvent être exécutées et reconfigurées sur un nombre arbitraire de machines virtuelles. Cependant, la fragmentation des ressources provient d'un problème plus fondamental. On observe que les applications cloud demandent de plus en plus de mémoire, tandis que les serveurs physiques fournissent plus de CPU. Dans les DC actuels, les deux ressources sont fortement couplées puisqu'elles sont liées à un serveur physique. Ma deuxième contribution est un moyen pratique de découpler la paire CPU-mémoire, qui peut être simplement appliquée à n'importe quel serveur. Ainsi, les deux ressources peuvent varier indépendamment, en fonction de leur demande. Ma troisième et ma quatrième contribution montrent un système pratique qui exploite la deuxième contribution. La sous-utilisation observée sur les serveurs physiques existe également pour les machines virtuelles. Il a été démontré que les machines virtuelles ne consomment qu'une petite fraction des ressources allouées car les clients du cloud ne sont pas en mesure d'estimer correctement la quantité de ressources nécessaire à leurs applications. Ma troisième contribution est un système qui estime la consommation de mémoire (c'est-à-dire la taille du working set) d'une machine virtuelle, avec un surcoût faible et une grande précision. Ainsi, nous pouvons maintenant consolider les machines virtuelles en fonction de la taille de leur working set (plutôt que leur mémoire réservée). Cependant, l'inconvénient de cette approche est le risque de famine de mémoire. Si une ou plusieurs machines virtuelles ont une forte augmentation de la demande en mémoire, le serveur physique peut manquer de mémoire. Cette situation n'est pas souhaitable, car la plate-forme cloud est incapable de fournir au client la mémoire qu'il a payée. Finalement, ma quatrième contribution est un système qui permet à une machine virtuelle d'utiliser la mémoire distante fournie par un autre serveur du rack. Ainsi, dans le cas d'un pic de la demande en mémoire, mon système permet à la VM d'allouer de la mémoire sur un serveur physique distant.Nowadays, many organizations choose to increasingly implement the cloud computing approach. More specifically, as customers, these organizations are outsourcing the management of their physical infrastructure to data centers (or cloud computing platforms). Energy consumption is a primary concern for datacenter (DC) management. Its cost represents about 80% of the total cost of ownership and it is estimated that in 2020, the US DCs alone will spend about 13 billion dollars on energy bills. Generally, the datacenter servers are manufactured in such a way that they achieve high energy efficiency at high utilizations. Thereby for a low cost per computation all datacenter servers should push the utilization as high as possible. In order to fight the historically low utilization, cloud computing adopted server virtualization. The latter allows a physical server to execute multiple virtual servers (called virtual machines) in an isolated way. With virtualization, the cloud provider can pack (consolidate) the entire set of virtual machines (VMs) on a small set of physical servers and thereby, reduce the number of active servers. Even so, the datacenter servers rarely reach utilizations higher than 50% which means that they operate with sets of longterm unused resources (called 'holes'). My first contribution is a cloud management system that dynamically splits/fusions VMs such that they can better fill the holes. This solution is effective only for elastic applications, i.e. applications that can be executed and reconfigured over an arbitrary number of VMs. However the datacenter resource fragmentation stems from a more fundamental problem. Over time, cloud applications demand more and more memory but the physical servers provide more an more CPU. In nowadays datacenters, the two resources are strongly coupled since they are bounded to a physical sever. My second contribution is a practical way to decouple the CPU-memory tuple that can simply be applied to a commodity server. Thereby, the two resources can vary independently, depending on their demand. My third and my forth contribution show a practical system which exploit the second contribution. The underutilization observed on physical servers is also true for virtual machines. It has been shown that VMs consume only a small fraction of the allocated resources because the cloud customers are not able to correctly estimate the resource amount necessary for their applications. My third contribution is a system that estimates the memory consumption (i.e. the working set size) of a VM, with low overhead and high accuracy. Thereby, we can now consolidate the VMs based on their working set size (not the booked memory). However, the drawback of this approach is the risk of memory starvation. If one or multiple VMs have an sharp increase in memory demand, the physical server may run out of memory. This event is undesirable because the cloud platform is unable to provide the client with the booked memory. My fourth contribution is a system that allows a VM to use remote memory provided by a different rack server. Thereby, in the case of a peak memory demand, my system allows the VM to allocate memory on a remote physical server

The lock holder and the lock waiter pre-emption problems: nip them in the bud using informed spinlocks (I-Spinlocks)

International audienceIn native Linux systems, spinlock's implementation relies on the assumption that both the lock holder thread and lock waiter threads cannot be preempted. However, in a virtualized environment, these threads are scheduled on top of virtual CPUs (vCPU) that can be preempted by the hypervisor at any time, thus forcing lock waiter threads on other vCPUs to busy wait and to waste CPU cycles. This leads to the well-known Lock Holder Preemption (LHP) and Lock Waiter Preemption (LWP) issues. In this paper, we propose I-Spinlock (for Informed Spinlock), a new spinlock implementation for virtualized environments. Its main principle is to only allow a thread to acquire a lock if and only if the remaining time-slice of its vCPU is sufficient to enter and leave the critical section. This is possible if the spinlock primitive is aware (informed) of its time-to-preemption (by the hypervisor). We implemented I-Spinlock in the Xen virtualization system. We show that our solution is compliant with both para-virtual and hardware virtualization modes. We performed extensive performance evaluations with various reference benchmarks and compared our solution to previous solutions. The evaluations demonstrate that I-Spinlock outperforms other solutions, and more significantly when the number of core increases

The lock holder and the lock waiter pre-emption problems: nip them in the bud using informed spinlocks (I-Spinlocks)

International audienceIn native Linux systems, spinlock's implementation relies on the assumption that both the lock holder thread and lock waiter threads cannot be preempted. However, in a virtualized environment, these threads are scheduled on top of virtual CPUs (vCPU) that can be preempted by the hypervisor at any time, thus forcing lock waiter threads on other vCPUs to busy wait and to waste CPU cycles. This leads to the well-known Lock Holder Preemption (LHP) and Lock Waiter Preemption (LWP) issues. In this paper, we propose I-Spinlock (for Informed Spinlock), a new spinlock implementation for virtualized environments. Its main principle is to only allow a thread to acquire a lock if and only if the remaining time-slice of its vCPU is sufficient to enter and leave the critical section. This is possible if the spinlock primitive is aware (informed) of its time-to-preemption (by the hypervisor). We implemented I-Spinlock in the Xen virtualization system. We show that our solution is compliant with both para-virtual and hardware virtualization modes. We performed extensive performance evaluations with various reference benchmarks and compared our solution to previous solutions. The evaluations demonstrate that I-Spinlock outperforms other solutions, and more significantly when the number of core increases

Crowdsensing Mobile Content and Context Data: Lessons Learned in the Wild

International audienceThis paper discusses the design and development efforts made to collect data using an opportunistic crowdsensing mobile application. Relevant issues are underlined, and solutions proposed within the CHIST-ERA Macaco project for the specifics of collecting fine-grained content and context data are highlighted. Global statistics on the data gathered for over a year of collection show its quality: Macaco data provides a longterm and fine-grained sampling of the user behavior and network usage that is relevant to model and analyse for future content and context-aware networking developments

Mitigating performance unpredictability in the IaaS using the Kyoto principle

International audiencePerformance isolation is enforced in the cloud by setting to each virtual machine (VM) a given fraction of each resource type (physical memory, processor, and IO bandwidth). However, microarchitectural-level resources such as processor's caches cannot be divided and allocated to VMs: they are globally shared among all VMs which compete for their use, leading to cache contention. Therefore, performance isolation and predictability are compromised. This situation is devastating for HPC applications. In this paper, we propose a software solution (called Kyoto) to this issue, inspired by the polluters pay principle. A VM is said to pollute the cache if it provokes significant cache replacements which impact the performance of other VMs. Henceforth, using the Kyoto system, the provider can encourage HPC cloud users to book pollution permits for their VMs. We have implemented Kyoto in several virtualization systems including both general purpose systems (Xen and KVM) and specialized HPC systems (Pisces)

StopGap: Elastic VMs to enhance server consolidation

International audienceVirtualized cloud infrastructures (also known as IaaS platforms) generally rely on a server consolidation system to pack virtual machines (VMs) on as few servers as possible. However, an important limitation of consolidation is not addressed by such systems. Because the managed VMs may be of various sizes (small, medium, large, etc.), VM packing may be obstructed when VMs do not fit available spaces. This phenomenon leaves servers with a set of unused resources (‘holes’). It is similar to memory fragmentation, a well‐known problem in operating system domain. In this paper, we propose a solution which consists in resizing VMs so that they can fit with holes. This operation leads to the management of what we call elastic VMs and requires cooperation between the application level and the IaaS level, because it impacts management at both levels. To this end, we propose a new resource negotiation and allocation model in the IaaS, called HRNM. We demonstrate HRNM's applicability through the implementation of a prototype compatible with two main IaaS managers (OpenStack and OpenNebula). By performing thorough experiments with SPECvirt_sc2010 (a reference benchmark for server consolidation), we show that the impact of HRNM on customer's application is negligible. Finally, using Google data center traces, we show an improvement of about 62.5% for the traditional consolidation engines

Swift birth and quick death: enabling fast parallel guest boot and destruction in the Xen hypervisor

ISBN : 978-1-4503-4948-2International audienceThe ability to quickly set up and tear down a virtual machine is critical for today's cloud elasticity, as well as in numerous other scenarios: guest migration/consolidation, event-driven invocation of micro-services, dynamically adaptive unikernel-based applications, micro-reboots for security or stability, etc. In this paper, we focus on the process of setting up/freeing the hypervisor and host control layer data structures at boot/destruction time, showing that it does not scale in current virtualization solutions. In addition to the direct overhead of long VM set-up/destruction times, we demonstrate by experimentation the indirect costs on real world auto scaling systems. Focusing on the popular Xen hypervisor, we identify three critical issues hindering the scalability of the boot and destruction processes: serialized boot, unscalable interactions with the Xenstore at guest creation time, and remote NUMA memory scrubbing at destruction time. For each of these issues we present the design and implementation of a solution in the Xen infrastructure: parallel boot with fine-grained locking, caching of Xenstore data, and local NUMA scrubbing. We evaluate these solutions using micro-benchmarks, macro-benchmarks, and real world datacenter traces. Results show that our work improves the current Xen implementation by a significant factor, for example macro-benchmarks indicate a speedup of more than 4X in high-load scenarios

Working set size estimation techniques in virtualized environments: One size does not fit all

Energy consumption is a primary concern for datacenters? management. Numerous datacenters are relying on virtualization, as it provides flexible resource management means such as virtual machine (VM) checkpoint/restart, migration and consolidation. However, one of the main hindrances to server consolidation is physical memory. In nowadays cloud, memory is generally statically allocated to VMs and wasted if not used. Techniques (such as ballooning) were introduced for dynamically reclaiming memory from VMs, such that only the needed memory is provisioned to each VM. However, the challenge is to precisely monitor the needed memory, i.e., the working set of each VM. In this paper, we thoroughly review the main techniques that were proposed for monitoring the working set of VMs. Additionally, we have implemented the main techniques in the Xen hypervisor and we have defined different metrics in order to evaluate their efficiency. Based on the evaluation results, we propose Badis, a system which combines several of the existing solutions, using the right solution at the right time. We also propose a consolidation extension which leverages Badis in order to pack the VMs based on the working set size and not the booked memory. The implementation of all techniques, our proposed system, and the benchmarks we have used are publicly available in order to support further research in this domain

Fine-Grained Isolation for Scalable, Dynamic, Multi-tenant Edge Clouds

International audienc