A survey of general-purpose experiment management tools for distributed systems

International audienceIn the field of large-scale distributed systems, experimentation is particularly difficult. The studied systems are complex, often nondeterministic and unreliable, software is plagued with bugs, whereas the experiment workflows are unclear and hard to reproduce. These obstacles led many independent researchers to design tools to control their experiments, boost productivity and improve quality of scientific results. Despite much research in the domain of distributed systems experiment management, the current fragmentation of efforts asks for a general analysis. We therefore propose to build a framework to uncover missing functionality of these tools, enable meaningful comparisons be-tween them and find recommendations for future improvements and research. The contribution in this paper is twofold. First, we provide an extensive list of features offered by general-purpose experiment management tools dedicated to distributed systems research on real platforms. We then use it to assess existing solutions and compare them, outlining possible future paths for improvements

Batsim: a Realistic Language-Independent Resources and Jobs Management Systems Simulator

International audienceAs large scale computation systems are growing to exascale, Resources and Jobs Management Systems (RJMS) need to evolve to manage this scale modification. However, their study is problematic since they are critical production systems, where experimenting is extremely costly due to downtime and energy costs. Meanwhile, many scheduling algorithms emerging from theoretical studies have not been transferred to production tools for lack of realistic experimental validation. To tackle these problems we propose Batsim, an extendable, language-independent and scalable RJMS simulator. It allows researchers and engineers to test and compare any scheduling algorithm, using a simple event-based communication interface, which allows different levels of realism. In this paper we show that Batsim's behaviour matches the one of the real RJMS OAR. Our evaluation process was made with reproducibility in mind and all the experiment material is freely available

Performance analysis and models for collocated VMs running on multi-core physical machines

Next generation high performance computers will massively use virtualization as a way to share hardware resources between multiple applications and to provide flexible mechanisms for fault tolerance and energy optimisation. In this context, understanding the performance behavior of virtual machines and the interference between them is a major scientific challenge that will allow a more efficient usage of resources. The first step is to characterize CPU usage sharing and to propose a performance model for virtual machines. Nonetheless, focusing on the sharing of a single CPU core is no more possible as next generation high performance machines will contain a large number of cores. Moreover, as these cores share micro-architectural resources e.g. caches, using a single core performance model is not sufficient as inter-core interference can happen. Finally, to be able to use such a model in large scale infrastructures as Clouds or high performance computers, the model must be lightweight to simulate the behavior of tens of thousands physical machines hosting hundreds of thousands virtual machines (VMs). In this paper, we present an in-depth analysis of the performance of collocated VMs. By running our experiments on the Grid'5000 testbed, we were able to evaluate 2 processor families for a total of 6 different processor models. We have systematically explored the effect of collocation by testing all the different VCPU to CPU mapping while taking into account micro-architectural components (shared caches and NUMA nodes). We also explored the effect of multi-core virtual machines. Based on these experiments, we evaluate 8 lightweight performance models and observe that the virtual machine performance can be accurately predicted using a model that takes into account the number of VMs on the core and on the related NUMA node (with less than 8% error). Finally, we validate our models on several processors and on both single and multi-(virtual)-cores VM. Using this model, we can increase the accuracy of the virtualization layer of the general purpose distributed system simulator SimGrid and improve it's usability to simulate (HPC) Clouds. These results may also be used to improve VM placement algorithms.La prochaine génération d'ordinateur haute performance (HPC) utilisera massivement la virtualisation comme un moyen pour partager les ressources matérielles entre plusieurs applications et également pour fournir des mécanismes flexibles pour la tolérance aux fautes ainsi que l'optimisation énergétique. Dans ce contexte, comprendre comment se comporte la performance des machines virtuelles et les interférences entre elles est un défi scientifique majeur qui permettra d'aller vers un utilisation plus efficace des ressources. La première étape est la caractérisation du partage de processeur et d'en proposer un modèle de performance pour les machines virtuelles. Mais, se concentrer sur le partage d'un processeur unique n'est plus possible. En effet, les prochaines générations d'ordinateur haute performance contiendront un très large ensemble de coeurs. De plus, comme ces coeurs partages des ressources micro-architecturales, e.g. caches, utilisé un modéle de performance d'un coeur unique n'est pas suffisant car il ne capturerai pas les interferences entre coeurs. Finalement, pour pouvoir utiliser un tel modèle à l'échelle d'infrastructures large échelle tel que les Clouds ou les ordinateurs HPC, le modèle doit être légé pour pouvoir simuler des dizaines de milliers de machines physiques hébergeant des centaines de milliers de machines virtuelles. Dans ce papier, nous présentons une analyse en profondeur des performances de machines virtuelles colocalisées i.e. s'exécutant sur la même machine physique. En exécutant nos expérimentations sur le banc d'essai Grid'5000, nous avons pu évaluer 2 familles de processeurs pour un total de 6 différents modèles de processeurs. Nous avons exploré systématiquement l'effet de la colocalisation en testant tous les différentes placements VCPU vers CPU tout en prenant en compte les composants micro-architecturaux (caches partagés et NUMA nodes). Nous avons également exploré l'effet des machines virtuelles qui ont plusieurs coeurs. En se basant sur ces expérimentations, nous avons évalué 8 modèles légés de performance et observé que la performance d'une machine virtuelle peut être précisément prédite en utilisant un modèle qui prend en compte le nombre de machines virtuelles sur le coeur et le nombre de machines virtuelles sur la NUMA Node (avec un taux d'erreur inférieur à 8%). Finalement, nous avons évalué nos modèles sur plusieurs processeurs et sur des machines virtuelles avec un ou plusieurs processeurs virtuels. En utilisant ce modèle, nous pourrons accroître la précision de la couche de virtualisation du simulateur générique pour les systèmes distribués, SimGrid mais également sa capacité à simuler des Clouds (haute performance). Nos résultats peuvent également être utilisés pour améliorer les algorithmes de placement de machines virtuelles

Batsim: a Realistic Language-Independent Resources and Jobs Management Systems Simulator

International audienceAs large scale computation systems are growing to exascale, Resources and Jobs Management Systems (RJMS) need to evolve to manage this scale modification. However, their study is problematic since they are critical production systems, where experimenting is extremely costly due to downtime and energy costs. Meanwhile, many scheduling algorithms emerging from theoretical studies have not been transferred to production tools for lack of realistic experimental validation. To tackle these problems we propose Batsim, an extendable, language-independent and scalable RJMS simulator. It allows researchers and engineers to test and compare any scheduling algorithm, using a simple event-based communication interface, which allows different levels of realism. In this paper we show that Batsim's behaviour matches the one of the real RJMS OAR. Our evaluation process was made with reproducibility in mind and all the experiment material is freely available

In-Transit Molecular Dynamics Analysis with Apache Flink

International audienceIn this paper, an on-line parallel analytics framework is proposed to process and store in transit all the data being generated by a Molecular Dynamics (MD) simulation run using staging nodes in the same cluster executing the simulation. The implementation and deployment of such a parallel workflow with standard HPC tools, managing problems such as data partitioning and load balancing, can be a hard task for scientists. In this paper we propose to leverage Apache Flink, a scalable stream processing engine from the Big Data domain, in this HPC context. Flink enables to program analyses within a simple window based map/reduce model, while the runtime takes care of the deployment, load balancing and fault tolerance. We build a complete in transit analytics workflow, connecting an MD simulation to Apache Flink and to a distributed database, Apache HBase, to persist all the desired data. To demonstrate the expressivity of this programming model and its suitability for HPC scientific environments, two common analytics in the MD field have been implemented. We assessed the performance of this framework, concluding that it can handle simulations of sizes used in the literature while providing an effective and versatile tool for scientists to easily incorporate on-line parallel analytics in their current workflows

Contribution à la convergence d'infrastructure entre le calcul haute performance et le traitement de données à large échelle

The amount of produced data, either in the scientific community or the commercialworld, is constantly growing. The field of Big Data has emerged to handle largeamounts of data on distributed computing infrastructures. High-Performance Computing (HPC) infrastructures are traditionally used for the execution of computeintensive workloads. However, the HPC community is also facing an increasingneed to process large amounts of data derived from high definition sensors andlarge physics apparati. The convergence of the two fields -HPC and Big Data- iscurrently taking place. In fact, the HPC community already uses Big Data tools,which are not always integrated correctly, especially at the level of the file systemand the Resource and Job Management System (RJMS).In order to understand how we can leverage HPC clusters for Big Data usage, andwhat are the challenges for the HPC infrastructures, we have studied multipleaspects of the convergence: We initially provide a survey on the software provisioning methods, with a focus on data-intensive applications. We contribute a newRJMS collaboration technique called BeBiDa which is based on 50 lines of codewhereas similar solutions use at least 1000 times more. We evaluate this mechanism on real conditions and in simulated environment with our simulator Batsim.Furthermore, we provide extensions to Batsim to support I/O, and showcase thedevelopments of a generic file system model along with a Big Data applicationmodel. This allows us to complement BeBiDa real conditions experiments withsimulations while enabling us to study file system dimensioning and trade-offs.All the experiments and analysis of this work have been done with reproducibilityin mind. Based on this experience, we propose to integrate the developmentworkflow and data analysis in the reproducibility mindset, and give feedback onour experiences with a list of best practices.RésuméLa quantité de données produites, que ce soit dans la communauté scientifiqueou commerciale, est en croissance constante. Le domaine du Big Data a émergéface au traitement de grandes quantités de données sur les infrastructures informatiques distribuées. Les infrastructures de calcul haute performance (HPC) sont traditionnellement utilisées pour l’exécution de charges de travail intensives en calcul. Cependant, la communauté HPC fait également face à un nombre croissant debesoin de traitement de grandes quantités de données dérivées de capteurs hautedéfinition et de grands appareils physique. La convergence des deux domaines-HPC et Big Data- est en cours. En fait, la communauté HPC utilise déjà des outilsBig Data, qui ne sont pas toujours correctement intégrés, en particulier au niveaudu système de fichiers ainsi que du système de gestion des ressources (RJMS).Afin de comprendre comment nous pouvons tirer parti des clusters HPC pourl’utilisation du Big Data, et quels sont les défis pour les infrastructures HPC, nousavons étudié plusieurs aspects de la convergence: nous avons d’abord proposé uneétude sur les méthodes de provisionnement logiciel, en mettant l’accent sur lesapplications utilisant beaucoup de données. Nous contribuons a l’état de l’art avecune nouvelle technique de collaboration entre RJMS appelée BeBiDa basée sur 50lignes de code alors que des solutions similaires en utilisent au moins 1000 fois plus.Nous évaluons ce mécanisme en conditions réelles et en environnement simuléavec notre simulateur Batsim. En outre, nous fournissons des extensions à Batsimpour prendre en charge les entrées/sorties et présentons le développements d’unmodèle de système de fichiers générique accompagné d’un modèle d’applicationBig Data. Cela nous permet de compléter les expériences en conditions réellesde BeBiDa en simulation tout en étudiant le dimensionnement et les différentscompromis autours des systèmes de fichiers.Toutes les expériences et analyses de ce travail ont été effectuées avec la reproductibilité à l’esprit. Sur la base de cette expérience, nous proposons d’intégrerle flux de travail du développement et de l’analyse des données dans l’esprit dela reproductibilité, et de donner un retour sur nos expériences avec une liste debonnes pratiques