Validation goals and metrics: project deliverable D7.3.2

The document at hand describes the second and final iteration of validation goals and metrics of the CACTOS software components. This deliverable D7.3.2 updates the first iteration (D7.3.1 Validation Goals and Metrics) and takes recent changes in tooling and a new use case scenario, but also results from the first validation cycle (D7.4.1 Validation and Result Analysis) into account. Global validation goals have been derived in (D7.3.1 Validation Goals and Metrics) based on scenario requirements and objectives from the description of work (DoW). Further and more detailed validation goals and metrics is presented in a split view, first from the CACTOS scenarios, and second from the CACTOS tools perspective. The scenarios are namely Business Analytics (by Flexiant), Scientific Computing (by University of Ulm), and Enterprise Applications (by PlayGen). The tools considered are CACTOS’ three main components CactoScale, CactoOpt and CactoSim. This document outlines at least one validation scenario per tool. A validation scenario is declared as successful when all of its acceptance criteria are met. The validation scenarios described in the document at hand for business analytics are #BC.1 NODE LOAD DISTRIBUTION and #BC.2 FAULT TOLERANCE. Both will be mostly validated on the Flexiant testbed. The validation scenarios for scientific computation are #SC.1 Running Molpro in a virtualised environment, #SC.2 Deploying Molpro through CACTOS, #SC.3 Monitoring Molpro instances, #SC.4 Mine Molpro traces, #SC.5 Prediction of execution time and execution phases, #SC.6 On-line Phase Detection of Running Applications, #SC.7 Failure Detection and Snapshots, #SC.8 Application Restart from Snapshots in Case of Failures, #SC.9 Concurrent Usage of Resources, #SC.10 Phase-aware scheduling, #SC.11 Detect Lack of Resources, #SC.12 Simulate Spare Resources, #SC.13 Simulate Change of Resource Allocation, and #SC.14 Control Power State of Physical Resources. The validation scenarios for enterprise application are Use Case I – Initial System Setup, Use Case II – User-Driven Optimisation, Use Case III – Automatic Optimisation, and Use Case IV – Scaling. The validation for CactoSim is delivered in (D6.5 Final results from optimization algorithms validation and experimentation) and hence not listed here. CactoOpt defines in the document at hand the validation scenarios Periodic Optimisation, Event driven Optimisation, and Previous Optimisation Plan in Progress. CactoScale defines the validation scenarios Cloud Platform Data Collection, Infrastructure Model Generation, and Offline Log Analysis. Concluding, the set of validation scenarios are the fundamental guide for the second and final validation of the CACTOS software in (D7.4.2 Validation and Result Analysis) at the end of the project. The validation scenarios together cover the global validation goals and requirements, requested by the use case scenarios and the DoW’s objectives

Preliminary results from optimisation models validation and experimentation: project deliverable D6.2

Since the arrival of cloud computing, a significant amount of research has been and continues to be carried out towards the creation of efficient optimisation strategies for meeting certain optimisation goals such as energy efficiency, resource consolidation or performance improvement within virtualised data centres. However, investigating whether specific optimisation algorithms can achieve the desired function in a production environment, and investigating how well they operate are quite complex tasks. Untested optimisation rules typically cannot be directly deployed in the production system, instead requiring manual test-bed experiments. This technique can be prohibitively costly, time consuming and cannot always account for scale and other constraints. This work presents a design-time optimisation evaluation solution based on discrete event simulation for cloud computing. By using a simulation toolkit (CactoSim) coupled with a runtime optimisation toolkit (CactoOpt), a cloud architect is able to create a direct replica model of the data centre production environment and then run simulations which take into account optimisation strategies. Results produced by such simulations can be used to estimate the optimisation algorithm performance under various conditions. In order to test the CactoSim and CactoOpt integration concept, a validation process has been performed on two different scenarios. The first scenario investigates the VM placement algorithm performance within a simulated testbed when admitting new VMs into the system. The second scenario analyses consolidation optimisation strategy impact on resource utilisation, with the objective being to free up nodes towards the goal of energy saving. This deliverable represents the initial part of two iterative pieces of work

Integrated data collection and analysis frameworks: project deliverable D4.4

CactoScale is a CACTOS component which provides monitoring and data analysis functionality. This deliverable presents the integration of the algorithmic data analysis framework and the data collection tool in a production-mode setup. For the setup we utilised the testbed provided by the industrial partner Flexiant and we examined CactoScale under the workload provided by DataPlay application from Playgen. In this deliverable we measure the performance of CactoScale when this is integrated with Flexiant Cloud Orchestrator (FCO) and the University of Ulm (UULM) testbeds under the impact of Playgen’s enterprise application (DataPlay) workload. We assess the scaling capability of the monitoring framework by measuring the overall latency under increasing numbers of VMs on the platform’s nodes. Furthermore, we assess any overhead induced by CactoScale on the integrated use case of Playgen when deployed on FCO. CactoScale also provides data analysis functionality to CACTOS. We demonstrate the performance of the analysis framework when integrated on the FCO-testbed and the UULM testbed. We use data collected from monitoring the deployed DataPlay application to perform correlation analysis using a Lightweight Anomaly Detection Tool (LADT). LADT utilises data correlation analysis to indicate any potential anomalies on a cloud compute node. To experiment on anomaly detection analysis we employ the DICE-fault-injection tool which allows fault injection on the Flexiant’s platform. We use DataPlay’s workload measurements to further estimate the scalability of the analytics tool by comparing the performance when different numbers of CPU cores are used for the analysis. Further CactoScale is integrated with the rest of the CACTOS ecosystem through the development of a prediction tool which is used by CactoOpt and CactoSim to enable cloud operators to make informed choices for deploying workloads. This integration has been evaluated using a computational quantum chemistry tool named MOLPRO. The evaluation considers stress testing the monitoring infrastructure both on single and multiple nodes. The experimental evaluations highlight that CactoScale offers scalable and low overhead monitoring and data analysis in a cloud environment

Preliminary results from optimisation models validation and experimentation: project deliverable D6.2

Since the arrival of cloud computing, a significant amount of research has been and continues to be carried out towards the creation of efficient optimisation strategies for meeting certain optimisation goals such as energy efficiency, resource consolidation or performance improvement within virtualised data centres. However, investigating whether specific optimisation algorithms can achieve the desired function in a production environment, and investigating how well they operate are quite complex tasks. Untested optimisation rules typically cannot be directly deployed in the production system, instead requiring manual test-bed experiments. This technique can be prohibitively costly, time consuming and cannot always account for scale and other constraints. This work presents a design-time optimisation evaluation solution based on discrete event simulation for cloud computing. By using a simulation toolkit (CactoSim) coupled with a runtime optimisation toolkit (CactoOpt), a cloud architect is able to create a direct replica model of the data centre production environment and then run simulations which take into account optimisation strategies. Results produced by such simulations can be used to estimate the optimisation algorithm performance under various conditions. In order to test the CactoSim and CactoOpt integration concept, a validation process has been performed on two different scenarios. The first scenario investigates the VM placement algorithm performance within a simulated testbed when admitting new VMs into the system. The second scenario analyses consolidation optimisation strategy impact on resource utilisation, with the objective being to free up nodes towards the goal of energy saving. This deliverable represents the initial part of two iterative pieces of work

CactoSim simulation framework final prototype: accompanying document for project deliverable D6.4

This deliverable provides supporting documentation for the software deliverable D6.4, the third release of the CactoSim simulation framework. It includes user manual supporting information for installation, usage, features, and data collection using the CactoSim toolkit, as well as technical information for developers with description of the architecture and internal interfaces. Through the use of CactoSim and this supporting documentation, the user is provided with a full concept of what is CactoSim able to produce, the main set of features, simulation of energy consumption, VMs, applications, and data centre elements. Simulations can be carried out allowing for energy conscious decisions for cloud operators. A full integration with other CACTOS tools (CactoOpt and CactoScale) makes CactoSim an indispensable tool for predicting and improving the usage of the data centre in a non-invasive way. To this end, this deliverable specifically presents (i) a utilisation guide or user manual of CactoSim towards easing users the installation process, (ii) the main list of features described from an utilisation perspective, (iii) a full integration between CactoSim and other CACTOS tools, and (iv) a description of the internals of CactoSim with an architecture and class diagrams in order to ease the extension of CactoSim for developers