Performance characterization of black boxes with self-controlled load injection for simulation-based sizing

International audienceSizing and capacity planning are key issues that must be addressed by anyone wanting to ensure a distributed system will sustain an expected workload. Solutions typically consist in either benchmarking,or modeling and simulating the target system. However, full-scale benchmarking may be too costly and almost impossible, while the granularity of modeling is often limited by the huge complexity and the lack of information about the system. To extract a model for this kind of system, we propose a methodology that combines both solutions by first identifying a middle-grain model made of interconnected black boxes, and then to separately characterize the performance and resource consumption of these black boxes. Then, we present two important issues : saturation and stability, that are key to system capacity characterization. To experiment our methodology, we propose a component-based supporting architecture, introducing control theory issues in a general approach to autonomic computing infrastructures

Model-Based Performance Anticipation in Multi-tier Autonomic Systems: Methodology and Experiments

http://www.thinkmind.org/download.php?articleid=netser_v3_n34_2010_3International audienceThis paper advocates for the introduction of perfor- mance awareness in autonomic systems. Our goal is to introduce performance prediction of a possible target configuration when a self-* feature is planning a system reconfiguration. We propose a global and partially automated process based on queues and queuing networks modelling. This process includes decomposing a distributed application into black boxes, identifying the queue model for each black box and assembling these models into a queuing network according to the candidate target configuration. Finally, performance prediction is performed either through simulation or analysis. This paper sketches the global process and focuses on the black box model identification step. This step is automated thanks to a load testing platform enhanced with a workload control loop. Model identification is based on statistical tests. The identified models are then used in performance prediction of autonomic system configurations. This paper describes the whole process through a practical experiment with a multi-tier application

Model-Based Performance Anticipation in Multi-tier Autonomic Systems: Methodology and Experiments

http://www.thinkmind.org/download.php?articleid=netser_v3_n34_2010_3International audienceThis paper advocates for the introduction of perfor- mance awareness in autonomic systems. Our goal is to introduce performance prediction of a possible target configuration when a self-* feature is planning a system reconfiguration. We propose a global and partially automated process based on queues and queuing networks modelling. This process includes decomposing a distributed application into black boxes, identifying the queue model for each black box and assembling these models into a queuing network according to the candidate target configuration. Finally, performance prediction is performed either through simulation or analysis. This paper sketches the global process and focuses on the black box model identification step. This step is automated thanks to a load testing platform enhanced with a workload control loop. Model identification is based on statistical tests. The identified models are then used in performance prediction of autonomic system configurations. This paper describes the whole process through a practical experiment with a multi-tier application

Reliable Self-Deployment of Cloud Applications

International audienceCloud applications consist of a set of interconnected software elements distributed over several virtual machines, themselves hosted on remote physical servers. Most existing solutions for deploying such applications require human intervention to configure parts of the system, do not respect functional dependencies among elements that must be respected when starting them, and do not handle virtual machine failures that can occur when deploying an application. This paper presents a self-deployment protocol that was designed to automatically configure a set of software elements to be deployed on different virtual machines. This protocol works in a decentralized way, i.e., there is no need for a centralized server. It also starts the software elements in a certain order, respecting important architectural invariants. This protocol supports virtual machine and network failures, and always succeeds in deploying an application when faced with a finite number of failures. Designing such highly parallel management protocols is difficult, therefore formal modeling techniques and verification tools were used for validation purposes. The protocol was implemented in Java and was used to deploy industrial applications

Self-scalable Benchmarking as a Service with Automatic Saturation Detection

Part 4: ServicesInternational audienceSoftware applications providers have always been required to perform load testing prior to launching new applications. This crucial test phase is expensive in human and hardware terms, and the solutions generally used would benefit from further development. In particular, designing an appropriate load profile to stress an application is difficult and must be done carefully to avoid skewed testing. In addition, static testing platforms are exceedingly complex to set up. New opportunities to ease load testing solutions are becoming available thanks to cloud computing. This paper describes a Benchmark-as-a-Service platform based on: (i) intelligent generation of traffic to the benched application without inducing thrashing (avoiding predefined load profiles), (ii) a virtualized and self-scalable load injection system. This platform was found to reduce the cost of testing by 50% compared to more commonly used solutions. It was experimented on the reference JEE benchmark RUBiS. This involved detecting bottleneck tiers

Building a generics stress tool for server performance and behaviour analysis

We present the Tester, a 100% Java benchmarking tool designed to accept a custom representation of a load schedule and reproduce it using distributed client machines in order to test a service. Our implementation is strongly based on a set of interfaces and pluggable implementations allowing for custom extensions. This project was realized at the Conrad Zuse Institut in Berlin, in a CoreGrid Research Exchange Program. One of the features of our application is the addition of the ZIB's Sminer Project Preferences library, a hierarchical data structure extracted from an XML configuration file, allowing for multiple customisation possibilities

Introduction de fonctionnalités d'auto-optimisation dans une architecture de selfbenchmarking

Le Benchmarking des systèmes client-serveur implique des infrastructures techniques réparties complexes, dont la gestion nécessite une approche autonomique. Cette gestion s'appuie sur une suite d'étapes, observation, analyse et rétroaction, qui correspond au principe d'une boucle de contrôle autonome. Des travaux antérieurs dans le domaine du test de performances ont montré comment introduire des fonctionnalités de test autonome par le biais d'une injection de charge auto-régulée. L'objectif de cette thèse est de suivre cette démarche de calcul autonome (autonomic computing) en y introduisant des fonctionnalités d'optimisation autonome. On peut ainsi obtenir automatiquement des résultats de benchmarks fiables et comparables, mettant en oeuvre l'ensemble des étapes de self-benchmarking. Notre contribution est double. D'une part, nous proposons un algorithme original pour l'optimisation dans un contexte de test de performance, qui vise à diminuer le nombre de solutions potentielles à tester, moyennant une hypothèse sur la forme de la fonction qui lie la valeur des paramètres à la performance mesurée. Cet algorithme est indépendant du système à optimiser. Il manipule des paramètres entiers, dont les valeurs sont comprises dans un intervalle donné, avec une granularité de valeur donnée. D'autre part, nous montrons une approche architecturale à composants et une organisation du benchmark automatique en plusieurs boucles de contrôle autonomes (détection de saturation, injection de charge, calcul d'optimisation), coordonnées de manière faiblement couplée via un mode de communication asynchrone de type publication-souscription. Complétant un canevas logiciel à composants pour l'injection de charge auto-régulée, nous y ajoutons des composants pour reparamétrer et redémarrer automatiquement le système à optimiser.Deux séries d'expérimentations ont été menées pour valider notre dispositif d'auto-optimisation. La première série concerne une application web de type achat en ligne, déployée sur un serveur d'application JavaEE. La seconde série concerne une application à trois tiers effectifs (WEB, métier (EJB JOnAS) et base de données) clusterSample. Les trois tiers sont sur des machines physiques distinctes.Benchmarking client-server systems involves complex, distributed technical infrastructures, whose management deserves an autonomic approach. It also relies on observation, analysis and feedback steps that closely matches the autonomic control loop principle. While previous works in performance testing have shown how to introduce autonomic load testing features through self-regulated load injection, the goal of this thesis is to follow this approach of autonomic computing to introduce self-optimization features in this architecture to obtain reliable and comparable benchmark results, and to achieve the fully principle of Self-benchmarking.Our contribution is twofold. From the algorithmic point of view, we propose an original optimization algorithm in the context of performance testing. This algorithm is divided into two parts. The first one concerns the overall level, i.e. the control of the performance index evolution, based on global parameters setting of the system. The second part concerns the search for the optimum when only one parameter is modified. From the software architecture point of view, we complete the Fractal component-based architecture, containing several autonomic control loops (saturation, injection, optimization computing) and we implement the coordination principle between these loops by asynchronous messages according to the publish-subscribe communication paradigm. To apply a given parameters setting on the system under test, we introduced new components Configurators to support the setting of parameters before starting the test process. It may also be necessary to restart all or part of the system to optimize to ensure that the new setting is effectively taken into account. We introduced components Starters to cover this need in a specific way for each system.To validate our self-optimization framework, two types of campaigns have been conducted onto the servers of Orange Labs in Meylan and the servers of the LISTIC Laboratory of the University of Savoie in Polytech Annecy-Chambéry (Annecy le Vieux). The first one is a WEB online shopping application deployed on a Java EE application server JonAS. The second one is a three-tiers application (WEB, business (EJB JOnAS) and data base) clusterSample. The three tiers are in three separate machines.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF

Contrôle des applications fondé sur la qualité de service pour les plate-formes logicielles dématérialisées (Cloud)

Cloud computing is a new computing model. Infrastructure, application and data are moved from local machines to internet and provided as services. Cloud users, such as application owners, can greatly save budgets from the elasticity feature, which refers to the “pay as you go” and on-demand characteristics, of cloud service. The goal of this thesis is to manage the Quality of Service (QoS) for applications running in cloud environments Cloud services provide application owners with great flexibility to assign “suitable” amount of resources according to the changing needs, for example caused by fluctuating request rate. “Suitable” or not needs to be clearly documented in Service Level Agreements (SLA) if this resource demanding task is hosted in a third party, such as a Platform as a Service (PaaS) provider. In this thesis, we propose and formally describe PSLA, which is a SLA description language for PaaS. PSLA is based on WS-Agreement, which is extendable and widely accepted as a SLA description language. Before signing the SLA contract, negotiations are unavoidable. During negotiations, the PaaS provider needs to evaluate if the SLA drafts are feasible or not. These evaluations are based on the analysis of the behavior of the application deployed in the cloud infrastructure, for instance throughput of served requests, response time, etc. Therefore, application dependent analysis, such as benchmark, is needed. Benchmarks are relatively costly and precise feasibility study usually imply large amount of benchmarks. In this thesis, we propose a benchmark based SLA feasibility study method to evaluate whether or not a SLA expressed in PSLA, including QoS targets, resource constraints, cost constraints and workload constraints can be achieved. This method makes tradeoff between the accuracy of a SLA feasibility study and benchmark costs. The intermediate of this benchmark based feasibility study process will be used as the workload-resource mapping model of our runtime control method. When application is running in a cloud infrastructure, the scalability feature of cloud infrastructures allows us to allocate and release resources according to changing needs. These resource provisioning activities are named runtime control. We propose the Runtime Control method based onSchedule, REactive and PROactive methods (RCSREPRO). Changing needs are mainly caused by the fluctuating workload for majority of the applications running in the cloud. The detailed workload information, for example the request arrival rates at scheduled points in time, is difficult to be known before running the application. Moreover, workload information listed in PSLA is too rough to give a fitted resource provisioning schedule before runtime. Therefore, runtime control decisions are needed to be performed in real time. Since resource provisioning actions usually require several minutes, RCSREPRO performs a proactive runtime control which means that it predicts future needs and assign resources in advance to have them ready when they are needed. Hence, prediction of the workload and workload-resource mapping are two problems involved in proactive runtime control. The workload-resource mapping model, which is initially derived from benchmarks in SLA feasibility study is continuously improved in a feedback way at runtime, increasing the accuracy of the control.To sum up, we contribute with three aspects to the QoS management of application running in the cloud: creation of PSLA, a PaaS level SLA description language; proposal of a benchmark based SLA feasibility study method; proposal of a runtime control method, RCSREPRO, to ensure the SLA when the application is running. The work described in this thesis is motivated and funded by the FSN OpenCloudware project (www.opencloudware.org).Le « Cloud computing » est un nouveau modèle de systèmes de calcul. L’infrastructure, les applications et les données sont déplacées de machines localisées sur des systèmes dématérialisés accédés sous forme de service via Internet. Le modèle « coût à l’utilisation » permet des économies de coût en modifiant la configuration à l’exécution (élasticité). L’objectif de cette thèse est de contribuer à la gestion de la Qualité de Service (QdS) des applications s’exécutant dans le Cloud. Les services Cloud prétendent fournir une flexibilité importante dans l’attribution des ressources de calcul tenant compte des variations perçues, telles qu’une fluctuation de la charge. Les capacités de variation doivent être précisément exprimées dans un contrat (le Service Level Agreement, SLA) lorsque l’application est hébergée par un fournisseur de Plateform as a Service (PaaS). Dans cette thèse, nous proposons et nous décrivons formellement le langage de description de SLA PSLA. PSLA est fondé sur WS-Agreement qui est lui-même un langage extensible de description de SLA. Des négociations préalables à la signature du SLA sont indispensables, pendant lesquelles le fournisseur de PaaS doit évaluer la faisabilité des termes du contrat. Cette évaluation, par exemple le temps de réponse, le débit maximal de requêtes servies, etc, est fondée sur une analyse du comportement de l’application déployée dans l’infrastructure cloud. Une analyse du comportement de l’application est donc nécessaire et habituellement assurée par des tests (benchmarks). Ces tests sont relativement coûteux et une étude précise de faisabilité demande en général de nombreux tests. Dans cette thèse, nous proposons une méthode d’étude de faisabilité concernant les critères de performance, à partir d’une proposition de SLA exprimée en PSLA. Cette méthode est un compromis entre la précision d’une étude exhaustive de faisabilité et les coûts de tests. Les résultats de cette étude constituent le modèle initial de la correspondance charge entrante-allocation de ressources utilisée à l’exécution. Le contrôle à l’exécution (runtime control) d’une application gère l’allocation de ressources en fonction des besoins, s’appuyant en particulier sur les capacités de passage à l’échelle (scalability) des infrastructures de cloud. Nous proposons RCSREPRO (Runtime Control method based on Schedule, REactive and PROactive methods), une méthode de contrôle à l’exécution fondée sur la planification et des contrôles réactifs et prédictifs. Les besoins d’adaptation à l’exécution sont essentiellement dus à une variation de la charge soumise à l’application, variations difficiles à estimer avant exécution et seulement grossièrement décrites dans le SLA. Il est donc nécessaire de reporter à l’exécution les décisions d’adaptation et d’y évaluer les possibles variations de charge. Comme les actions de modification des ressources attribuées peuvent prendre plusieurs minutes, RCSREPRO réalise un contrôle prédictif fondée sur l’analyse de charge et la correspondance indicateurs de performance-ressources attribuées, initialement définie via des tests. Cette correspondance est améliorée en permanence à l’exécution. En résumé, les contributions de cette thèse sont la proposition de langage PSLA pour décrire les SLA ; une proposition de méthode pour l’étude de faisabilité d’un SLA ; une proposition de méthode (RCSREPRO) de contrôle à l’exécution de l’application pour garantir le SLA. Les travaux de cette thèse s’inscrivent dans le contexte du projet FSN OpenCloudware (www.opencloudware.org) et ont été financés en partie par celui-ci

Déploiement d'applications patrimoniales en environnements de type informatique dans le nuage

L'objectif de cette thèse est d'offrir une solution de bout en bout permettant de décrire et de déployer de façon fiable une application distribuée dans un environnement virtualisé. Ceci passe par la définition d'un formalisme permettant de décrirer une application ainsi que son environnement d'exécution, puis de fournir les outils capable d'interpéter ce formalisme pour déployer (installer, instancier et configurer) l'application sur une plate-forme de type cloud computing.Cloud computing aims to cut down on the outlay and operational expenses involved in setting up and running applications. To do this, an application is split into a set of virtualized hardware and software resources. This virtualized application can be autonomously managed, making it responsive to the dynamic changes affecting its running environment. This is referred to as Application Life-cycle Management (ALM). In cloud computing, ALM is a growing but immature market, with many offers claiming to significantly improve productivity. However, all these solutions are faced with a major restriction: the duality between the level of autonomy they offer and the type of applications they can handle. To address this, this thesis focuses on managing the initial deployment of an application to demonstrate that the duality is artificial. The main contributions of this work are presented in a platform named VAMP (Virtual Applications Management Platform). VAMP can deploy any legacy application distributed in the cloud, in an autonomous, generic and reliable way. It consists of: a component-based model to describe the elements making up an application and their projection on the running infrastructure, as well as the dependencies binding them in the applicative architecture; an asynchronous, distributed and reliable protocol for self-configuration and self-activation of the application; mechanisms ensuring the reliability of the VAMP system itself. Beyond implementing the solution, the most critical aspects of running VAMP have been formally verified using model checking tools. A validation step was also used to demonstrate the genericity of the proposal through various real-life implementations.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF

Proceedings of the First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014): Porto, Portugal

Proceedings of: First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014). Porto (Portugal), August 27-28, 2014