A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.Comment: accepted by 37th IEEE International Conference on Distributed Computing (ICDCS 2017

Dynamics and hysteresis in square lattice artificial spin-ice

Dynamical effects under geometrical frustration are considered in a model for artificial spin ice on a square lattice in two dimensions. Each island of the spin ice has a three-component Heisenberg-like dipole moment subject to shape anisotropies that influence its direction. The model has real dynamics, including rotation of the magnetic degrees of freedom, going beyond the Ising-type models of spin ice. The dynamics is studied using a Langevin equation solved via a second order Heun algorithm. Thermodynamic properties such as the specific heat are presented for different couplings. A peak in specific heat is related to a type of melting-like phase transition present in the model. Hysteresis in an applied magnetic field is calculated for model parameters where the system is able to reach thermodynamic equilibrium.Comment: Revised versio

Time-Varying Input and State Delay Compensation for Uncertain Nonlinear Systems

A robust controller is developed for uncertain, second-order nonlinear systems subject to simultaneous unknown, time-varying state delays and known, time-varying input delays in addition to additive, sufficiently smooth disturbances. An integral term composed of previous control values facilitates a delay-free open-loop error system and the development of the feedback control structure. A stability analysis based on Lyapunov-Krasovskii (LK) functionals guarantees uniformly ultimately bounded tracking under the assumption that the delays are bounded and slowly varying