A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.Comment: accepted by 37th IEEE International Conference on Distributed Computing (ICDCS 2017

Real-time multi-resource allocation via a structured policy table

Mobile Edge Cloud endures limited computational resources as compared to back-end cloud. Semi-Markov Decision Process (SMDP) based Multi-Resource Allocation (MRA) work [6] introduces optimal resource allocation for mobile requests in the resource constrained edge cloud environments. In this study, we scale existing SMDP MRA work for real-world scenarios. First, we structure the policy tables in a two dimensional matrix such that columns represent states of the system and rows for the actions. Second, we propose an index based search technique over structured policy tables. Simulation results demonstrate that our approach outperforms the legacy method and retrieves an optimal action from the policy tables in the order of microseconds, which meets the delay criteria of real-time applications in edge cloud based systems.11th International Conference on Intelligent Networking and Collaborative Systems(INCoS 2019), September 5-7, 2019, Oita, Japa

Optimizing simultaneous autoscaling for serverless cloud computing

This paper explores resource allocation in serverless cloud computing platforms and proposes an optimization approach for autoscaling systems. Serverless computing relieves users from resource management tasks, enabling focus on application functions. However, dynamic resource allocation and function replication based on changing loads remain crucial. Typically, autoscalers in these platforms utilize threshold-based mechanisms to adjust function replicas independently. We model applications as interconnected graphs of functions, where requests probabilistically traverse the graph, triggering associated function execution. Our objective is to develop a control policy that optimally allocates resources on servers, minimizing failed requests and response time in reaction to load changes. Using a fluid approximation model and Separated Continuous Linear Programming (SCLP), we derive an optimal control policy that determines the number of resources per replica and the required number of replicas over time. We evaluate our approach using a simulation framework built with Python and simpy. Comparing against threshold-based autoscaling, our approach demonstrates significant improvements in average response times and failed requests, ranging from 15% to over 300% in most cases. We also explore the impact of system and workload parameters on performance, providing insights into the behavior of our optimization approach under different conditions. Overall, our study contributes to advancing resource allocation strategies, enhancing efficiency and reliability in serverless cloud computing platforms

EIPSIM: Modeling Secure IP Address Allocation at Cloud Scale

Public clouds provide impressive capability through resource sharing. However, recent works have shown that the reuse of IP addresses can allow adversaries to exploit the latent configurations left by previous tenants. In this work, we perform a comprehensive analysis of the effect of cloud IP address allocation on exploitation of latent configuration. We first develop a statistical model of cloud tenant behavior and latent configuration based on literature and deployed systems. Through these, we analyze IP allocation policies under existing and novel threat models. Our resulting framework, EIPSim, simulates our models in representative public cloud scenarios, evaluating adversarial objectives against pool policies. In response to our stronger proposed threat model, we also propose IP scan segmentation, an IP allocation policy that protects the IP pool against adversarial scanning even when an adversary is not limited by number of cloud tenants. Our evaluation shows that IP scan segmentation reduces latent configuration exploitability by 97.1% compared to policies proposed in literature and 99.8% compared to those currently deployed by cloud providers. Finally, we evaluate our statistical assumptions by analyzing real allocation and configuration data, showing that results generalize to deployed cloud workloads. In this way, we show that principled analysis of cloud IP address allocation can lead to substantial security gains for tenants and their users

A Hybrid Optimization Algorithm for Efficient Virtual Machine Migration and Task Scheduling Using a Cloud-Based Adaptive Multi-Agent Deep Deterministic Policy Gradient Technique

This To achieve optimal system performance in the quickly developing field of cloud computing, efficient resource management—which includes accurate job scheduling and optimized Virtual Machine (VM) migration—is essential. The Adaptive Multi-Agent System with Deep Deterministic Policy Gradient (AMS-DDPG) Algorithm is used in this study to propose a cutting-edge hybrid optimization algorithm for effective virtual machine migration and task scheduling. An sophisticated combination of the War Strategy Optimization (WSO) and Rat Swarm Optimizer (RSO) algorithms, the Iterative Concept of War and Rat Swarm (ICWRS) algorithm is the foundation of this technique. Notably, ICWRS optimizes the system with an amazing 93% accuracy, especially for load balancing, job scheduling, and virtual machine migration. The VM migration and task scheduling flexibility and efficiency are greatly improved by the AMS-DDPG technology, which uses a powerful combination of deterministic policy gradient and deep reinforcement learning. By assuring the best possible resource allocation, the Adaptive Multi-Agent System method enhances decision-making even more. Performance in cloud-based virtualized systems is significantly enhanced by our hybrid method, which combines deep learning and multi-agent coordination. Extensive tests that include a detailed comparison with conventional techniques verify the effectiveness of the suggested strategy. As a consequence, our hybrid optimization approach is successful. The findings show significant improvements in system efficiency, shorter job completion times, and optimum resource utilization. Cloud-based systems have unrealized potential for synergistic optimization, as shown by the integration of ICWRS inside the AMS-DDPG framework. Enabling a high-performing and sustainable cloud computing infrastructure that can adapt to the changing needs of modern computing paradigms is made possible by this strategic resource allocation, which is attained via careful computational utilization

FEDRESOURCE: Federated Learning Based Resource Allocation in Modern Wireless Networks

Deep reinforcement learning can effectively deal with resource allocation (RA) in wireless networks. However, more complex networks can have slower learning speeds, and a lack of network adaptability requires new policies to be learned for newly introduced systems. To address these issues, a novel federated learning-based resource allocation (FEDRESOURCE) has been proposed in this paper which efficiently performs RA in wireless networks. The proposed FEDRESOURCE technique uses federated learning (FL) which is a ML technique that shares the DRL-based RA model between distributed systems and a cloud server to describe a policy. The regularized local loss that occurs in the network will be reduced by using a butterfly optimization technique, which increases the convergence of the FL algorithm. The suggested FL framework speeds up policy learning and allows for adoption by employing deep learning and the optimization technique. Experiments were conducted using a Python-based simulator and detailed numerical results for the wireless RA sub-problems. The theoretical results of the novel FEDRESOURCE algorithm have been validated in terms of transmission power, convergence of algorithm, throughput, and cost. The proposed FEDRESOURCE technique achieves maximum transmit power up to 27%, 55%, and 68% energy efficiency compared to Scheduling policy, Asynchronous FL framework, and Heterogeneous computation schemes respectively. The proposed FEDRESOURCE technique can increase discrimination accuracy by 1.7%, 1.2%, and 0.78% compared to the scheduling policy framework, Asynchronous FL framework, and Heterogeneous computation schemes respectively

Workload Prediction for Efficient Performance Isolation and System Reliability

In large-scaled and distributed systems, like multi-tier storage systems and cloud data centers, resource sharing among workloads brings multiple benefits while introducing many performance challenges. The key to effective workload multiplexing is accurate workload prediction. This thesis focuses on how to capture the salient characteristics of the real-world workloads to develop workload prediction methods and to drive scheduling and resource allocation policies, in order to achieve efficient and in-time resource isolation among applications. For a multi-tier storage system, high-priority user work is often multiplexed with low-priority background work. This brings the challenge of how to strike a balance between maintaining the user performance and maximizing the amount of finished background work. In this thesis, we propose two resource isolation policies based on different workload prediction methods: one is a Markovian model-based and the other is a neural networks-based. These policies aim at, via workload prediction, discovering the opportune time to schedule background work with minimum impact on user performance. Trace-driven simulations verify the efficiency of the two pro- posed resource isolation policies. The Markovian model-based policy successfully schedules the background work at the appropriate periods with small impact on the user performance. The neural networks-based policy adaptively schedules user and background work, resulting in meeting both performance requirements consistently. This thesis also proposes an accurate while efficient neural networks-based pre- diction method for data center usage series, called PRACTISE. Different from the traditional neural networks for time series prediction, PRACTISE selects the most informative features from the past observations of the time series itself. Testing on a large set of usage series in production data centers illustrates the accuracy (e.g., prediction error) and efficiency (e.g., time cost) of PRACTISE. The superiority of the usage prediction also allows a proactive resource management in the highly virtualized cloud data centers. In this thesis, we analyze on the performance tickets in the cloud data centers, and propose an active sizing algorithm, named ATM, that predicts the usage workloads and re-allocates capacity to work- loads to avoid VM performance tickets. Moreover, driven by cheap prediction of usage tails, we also present TailGuard in this thesis, which dynamically clones VMs among co-located boxes, in order to efficiently reduce the performance violations of physical boxes in cloud data centers

Optimization of Elastic Cloud Brokerage Mechanisms for Future Telecommunication Service Environments

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Cloud computing mechanisms and cloud-based services are currently revolutionizing Web as well as telecommunication service platforms and service offerings. Apart from providing infrastructures, platforms and software as a service, mechanism for dynamic allocation of compute and storage resources on-demand, commonly termed as “elastic cloud computing” account for the most important cloud computing functionalities. Resource elasticity allows not only for efficient internal compute and storage resource consumption, but also, through so called hybrid cloud computing mechanisms, for dynamic utilization of external resources on-demand. This capability is especially useful in order to cost-efficiently cope with peakworkloads, allowing service providers to significantly reduce usually required over-provisioned service infrastructures, allowing for “pay-per-use” cost models. With a steadily growing number of cloud providers and with the proliferation of unified cloud computing interfaces, service providers are given free choice of flexibly selecting and utilizing cloud resources from different cloud providers. Cloud brokering systems allow for dynamic selection and utilization of cloud computing resources based on functional (e.g. QoS, SLA, energy consumption) as well as nonfunctional criteria (e.g. costs). The presented work focuses on enhanced cloud brokering mechanisms for telecommunication service platforms, enabling quality telecommunication service assurance, still optimizing cloud resources consumption, i.e. saving costs and energy. Furthermore this work shows that by combining cloud brokering mechanisms with standardized telecommunication service brokering mechanisms an even greater benefit for telecommunication service providers can be achieved as this enables an even better cost-efficiency since different user segments can seamlessly be served by allocating different cloud resources to them in a policy-driven manner