Towards a Holistic Controller: Reinforcement Learning for Data Center Control

The increased use of cloud and other large scale datacenter IT services and the associated power usage has put the spotlight on more energy-efficient datacenter management. In this paper, a simple model was developed to represent the heat rejection system and energy usage in a small DC setup. The model was then controlled by a reinforcement learning agent that handles both the load balancing of the IT workload, as well as cooling system setpoints.The main contribution is the holistic approach to datacenter control where both facility metrics, IT hardware metric and cloud service logs are used as inputs. The application of reinforcement learning in the proposed holistic setup is feasible and achieves results that outperform standard algorithms. The paper presents both the simplified DC model and the reinforcement learning agent in detail and discusses how this work can be extended towards a richer datacenter model

Machine Learning in SME: An Empirical Study on Enablers and Success Factors

Machine learning (ML) techniques are rapidly evolving, both in academia and practice. However, enterprises show different maturity levels in successfully implementing ML techniques. Thus, we review the state of adoption of ML in enterprises. We find that ML technologies are being increasingly adopted in enterprises, but that small and medium-size enterprises (SME) are struggling with the introduction in comparison to larger enterprises. In order to identify enablers and success factors we conduct a qualitative empirical study with 18 companies in different industries. The results show that especially SME fail to apply ML technologies due to insufficient ML knowhow. However, partners and appropriate tools can compensate this lack of resources. We discuss approaches to bridge the gap for SME

ContainerGym: A Real-World Reinforcement Learning Benchmark for Resource Allocation

We present ContainerGym, a benchmark for reinforcement learning inspired by a real-world industrial resource allocation task. The proposed benchmark encodes a range of challenges commonly encountered in real-world sequential decision making problems, such as uncertainty. It can be configured to instantiate problems of varying degrees of difficulty, e.g., in terms of variable dimensionality. Our benchmark differs from other reinforcement learning benchmarks, including the ones aiming to encode real-world difficulties, in that it is directly derived from a real-world industrial problem, which underwent minimal simplification and streamlining. It is sufficiently versatile to evaluate reinforcement learning algorithms on any real-world problem that fits our resource allocation framework. We provide results of standard baseline methods. Going beyond the usual training reward curves, our results and the statistical tools used to interpret them allow to highlight interesting limitations of well-known deep reinforcement learning algorithms, namely PPO, TRPO and DQN