Dynamic QoS optimization architecture for cloud-based DDDAS

Cloud computing urges the need for novel on-demand approaches, where the Quality of Service (QoS) requirements of cloud-based services can dynamically and adaptively evolve at runtime as Service Level Agreement (SLA) and environment changes. Given the unpredictable, dynamic and on-demand nature of the cloud, it would be unrealistic to assume that optimal QoS can be achieved at design time. As a result, there is an increasing need for dynamic and self- adaptive QoS optimization solutions to respond to dynamic changes in SLA and the environment. In this context, we posit that the challenge of self-adaptive QoS optimization encompasses two dynamics, which are related to QoS sensitivity and conflicting objectives at runtime. We propose novel design of a dynamic data-driven architecture for optimizing QoS influenced by those dynamics. The architecture leverages on DDDAS primitives by employing distributed simulations and symbiotic feedback loops, to dynamically adapt decision making metaheuristics, which optimizes for QoS tradeoffs in cloud-based systems. We use a scenario to exemplify and evaluate the approach

Energy and Performance: Management of Virtual Machines: Provisioning, Placement, and Consolidation

Cloud computing is a new computing paradigm that offers scalable storage and compute resources to users on demand through Internet. Public cloud providers operate large-scale data centers around the world to handle a large number of users request. However, data centers consume an immense amount of electrical energy that can lead to high operating costs and carbon emissions. One of the most common and effective method in order to reduce energy consumption is Dynamic Virtual Machines Consolidation (DVMC) enabled by the virtualization technology. DVMC dynamically consolidates Virtual Machines (VMs) into the minimum number of active servers and then switches the idle servers into a power-saving mode to save energy. However, maintaining the desired level of Quality-of-Service (QoS) between data centers and their users is critical for satisfying users’ expectations concerning performance. Therefore, the main challenge is to minimize the data center energy consumption while maintaining the required QoS. This thesis address this challenge by presenting novel DVMC approaches to reduce the energy consumption of data centers and improve resource utilization under workload independent quality of service constraints. These approaches can be divided into three main categories: heuristic, meta-heuristic and machine learning. Our first contribution is a heuristic algorithm for solving the DVMC problem. The algorithm uses a linear regression-based prediction model to detect over-loaded servers based on the historical utilization data. Then it migrates some VMs from the over-loaded servers to avoid further performance degradations. Moreover, our algorithm consolidates VMs on fewer number of server for energy saving. The second and third contributions are two novel DVMC algorithms based on the Reinforcement Learning (RL) approach. RL is interesting for highly adaptive and autonomous management in dynamic environments. For this reason, we use RL to solve two main sub-problems in VM consolidation. The first sub-problem is the server power mode detection (sleep or active). The second sub-problem is to find an effective solution for server status detection (overloaded or non-overloaded). The fourth contribution of this thesis is an online optimization meta-heuristic algorithm called Ant Colony System-based Placement Optimization (ACS-PO). ACS is a suitable approach for VM consolidation due to the ease of parallelization, that it is close to the optimal solution, and its polynomial worst-case time complexity. The simulation results show that ACS-PO provides substantial improvement over other heuristic algorithms in reducing energy consumption, the number of VM migrations, and performance degradations. Our fifth contribution is a Hierarchical VM management (HiVM) architecture based on a three-tier data center topology which is very common use in data centers. HiVM has the ability to scale across many thousands of servers with energy efficiency. Our sixth contribution is a Utilization Prediction-aware Best Fit Decreasing (UP-BFD) algorithm. UP-BFD can avoid SLA violations and needless migrations by taking into consideration the current and predicted future resource requirements for allocation, consolidation, and placement of VMs. Finally, the seventh and the last contribution is a novel Self-Adaptive Resource Management System (SARMS) in data centers. To achieve scalability, SARMS uses a hierarchical architecture that is partially inspired from HiVM. Moreover, SARMS provides self-adaptive ability for resource management by dynamically adjusting the utilization thresholds for each server in data centers.Siirretty Doriast

An efficient resource sharing technique for multi-tenant databases

Multi-tenancy is one of the key components of cloud computing environment. Multi-tenant database system in SaaS (Software as a Service) has gained a lot of attention in academics, research and business arena. These database systems provide scalability and economic benefits for both cloud service providers and customers(organizations/companies referred as tenants) by sharing same resources and infrastructure in isolation of shared databases, network and computing resources with Service level agreement (SLA) compliances. In a multitenant scenario, active tenants compete for resources in order to access the database. If one tenant blocks up the resources, the performance of all the other tenants may be restricted and a fair sharing of the resources may be compromised. The performance of tenants must not be affected by resource-intensive activities and volatile workloads of other tenants. Moreover, the prime goal of providers is to accomplish low cost of operation, satisfying specific schemas/SLAs of each tenant. Consequently, there is a need to design and develop effective and dynamic resource sharing algorithms which can handle above mentioned issues. This work presents a model embracing a query classification and worker sorting technique to efficiently share I/O, CPU and Memory thus enhancing dynamic resource sharing and improvising the utilization of idle instances proficiently. The model is referred as Multi-Tenant Dynamic Resource Scheduling Model (MTDRSM) .The MTDRSM support workload execution of different benchmark such as TPC-C(Transaction Processing Performance Council), YCSB(The Yahoo! Cloud Serving Benchmark)etc. and on different database such as MySQL, Oracle, H2 database etc. Experiments are conducted for different benchmarks with and without SLA compliances to evaluate the performance of MTDRSM in terms of latency and throughput achieved. The experiments show significant performance improvement over existing Mute Bench model in terms of latency and throughput

Advanced Signal Processing Techniques Applied to Power Systems Control and Analysis

The work published in this book is related to the application of advanced signal processing in smart grids, including power quality, data management, stability and economic management in presence of renewable energy sources, energy storage systems, and electric vehicles. The distinct architecture of smart grids has prompted investigations into the use of advanced algorithms combined with signal processing methods to provide optimal results. The presented applications are focused on data management with cloud computing, power quality assessment, photovoltaic power plant control, and electrical vehicle charge stations, all supported by modern AI-based optimization methods

Optimisation of a water company’s waste pumping asset base with a focus on energy reduction

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonWater companies use a significant quantity of electricity for the operation of their clean and wastewater assets. Rising energy prices have led to higher energy bills within the water companies, which has increased operating costs. Thus, improvements in demand side energy management are needed to increase efficiency and reduce costs, which forms the premise for this research project. Thames Water Utilities Ltd has identified that improvements in demand side energy management is required and is currently researching various methods to reduce energy consumption. One initiative included the upgrade of a variety of site telemetry assets. By deploying these new telemetry assets, Thames Water Utilities Ltd are more able to liberate the asset data and as such, be able to make informed decisions on how better to control and optimise the target sites, which is where this research project has seen further opportunities. This enhanced telemetry and SCADA infrastructure will enable successful research to further develop an intelligent integrated system that tackles pump scheduling and process control with the emphasis on energy management. The use of modern techniques, such as artificial intelligence, to optimise the network operation is gradually gaining traction. The balance between implementing new technology (with the benefits it may bring) and reluctance to change from the incumbent operating model will always provide challenges in the technology adoption agenda. The main work of this research project included the physical surveying of a wastewater hydraulic catchment, inclusive of all wet well dimensions, lidar overlays, and pump electrical power characteristics. These survey results where then able to be programmed by the research into the company’s' hydraulic model to enable a higher degree of accuracy in the modelling, as well as enabling electrical power as a measurable output. From here, the model was then able to be optimised, focussing on electrical energy as an output variable for reduction. The research concluded that electrical energy consumption over time can be reduced using the aforementioned strategies and as such recommends further work to move from the model environment to physical architecture. It does so with the key message that risk tolerances on water levels must be pre-agreed with hydraulic specialists prior to deployment