Discovering Linear Models of Grid Workload

Despite extensive research focused on enabling QoS for grid users through economic and intelligent resource provisioning, no consensus has emerged on the most promising strategies. On top of intrinsically challenging problems, the complexity and size of data has so far drastically limited the number of comparative experiments. An alternative to experimenting on real, large, and complex data, is to look for well-founded and parsimonious representations. The goal of this paper is to answer a set of preliminary questions, which may help steering the design of those along feasible paths: is it possible to exhibit consistent models of the grid workload? If such models do exist, which classes of models are more appropriate, considering both simplicity and descriptive power? How can we actually discover such models? And finally, how can we assess the quality of these models on a statistically rigorous basis? Our main contributions are twofold. First we found that grid workload models can consistently be discovered from the real data, and that limiting the range of models to piecewise linear time series models is sufficiently powerful. Second, we presents a bootstrapping strategy for building more robust models from the limited samples at hand. This study is based on exhaustive information representative of a significant fraction of e-science computing activity in Europe

Implementing FCFS and SJF for finding the need of Reinforcement Learning in Cloud Environment

Cloud has grown significantly and has become a popular serviceoriented paradigm offering users a variety of services. The end-user submits requests to the cloud in the form of tasks with the expectation that they will be executed at the best possible lowest time, cost and without any errors. On the other hand, the cloud executes these tasks on the Virtual Machines (VM) by using resource scheduling algorithms. The cloud performance is directly dependent on how the resources are managed and allocated for executing the tasks. The main aim of this research paper is to compare the behaviour of cloud resource scheduling algorithms: First Come First Serve (FCFS) and Shortest Job First (SJF) by processing high-sized tasks. This research paper is broadly divided into four phases: the first phase includes an experiment conducted by processing approximately 80 thousand tasks from the Alibaba task event dataset using the resource scheduling algorithms: FCFS and SJF on the cloud VMs under different circumstances; the second phase includes the experimental results; the third phase includes a empirical analysis of the behaviour of resource scheduling algorithms; the last phase includes the proposed need of Reinforcement Learning (RL) to improve cloud resource scheduling and its overall performance

Design and optimisation of a (FA)Q-learning-based HTTP adaptive streaming client

In recent years, HTTP (Hypertext Transfer Protocol) adaptive streaming (HAS) has become the de facto standard for adaptive video streaming services. A HAS video consists of multiple segments, encoded at multiple quality levels. State-of-the-art HAS clients employ deterministic heuristics to dynamically adapt the requested quality level based on the perceived network conditions. Current HAS client heuristics are, however, hardwired to fit specific network configurations, making them less flexible to fit a vast range of settings. In this article, a (frequency adjusted) Q-learning HAS client is proposed. In contrast to existing heuristics, the proposed HAS client dynamically learns the optimal behaviour corresponding to the current network environment in order to optimise the quality of experience. Furthermore, the client has been optimised both in terms of global performance and convergence speed. Thorough evaluations show that the proposed client can outperform deterministic algorithms by 11-18% in terms of mean opinion score in a wide range of network configurations

Multi-objective reinforcement learning for responsive grids

The original publication is available at www.springerlink.comInternational audienceGrids organize resource sharing, a fundamental requirement of large scientific collaborations. Seamless integration of grids into everyday use requires responsiveness, which can be provided by elastic Clouds, in the Infrastructure as a Service (IaaS) paradigm. This paper proposes a model-free resource provisioning strategy supporting both requirements. Provisioning is modeled as a continuous action-state space, multi-objective reinforcement learning (RL) problem, under realistic hypotheses; simple utility functions capture the high level goals of users, administrators, and shareholders. The model-free approach falls under the general program of autonomic computing, where the incremental learning of the value function associated with the RL model provides the so-called feedback loop. The RL model includes an approximation of the value function through an Echo State Network. Experimental validation on a real data-set from the EGEE grid shows that introducing a moderate level of elasticity is critical to ensure a high level of user satisfaction

Grid Differentiated Services: a Reinforcement Learning Approach

International audienceLarge scale production grids are a major case for autonomic computing. Following the classical definition of Kephart, an autonomic computing system should optimize its own behavior in accordance with high level guidance from humans. This central tenet of this paper is that the combination of utility functions and reinforcement learning (RL) can provide a general and efficient method for dynamically allocating grid resources in order to optimize the satisfaction of both endusers and participating institutions. The flexibility of an RLbased system allows to model the state of the grid, the jobs to be scheduled, and the high-level objectives of the various actors on the grid. RL-based scheduling can seamlessly adapt its decisions to changes in the distributions of inter-arrival time, QoS requirements, and resource availability. Moreover, it requires minimal prior knowledge about the target environment, including user requests and infrastructure. Our experimental results, both on a synthetic workload and a real trace, show that RL is not only a realistic alternative to empirical scheduler design, but is able to outperform them

The judgment of forseti: economic utility for dynamic heap sizing of multiple runtimes

We introduce the Forseti system, which is a principled approach for holistic memory management. It permits a sysadmin to specify the total physical memory resource that may be shared between all concurrent virtual machines on a physical node. Forseti models the heap size versus application throughput for each virtual machine, and seeks to maximize the combined throughput of the set of VMs based on concepts from economic utility theory. We evaluate the Forseti system using a standard Java managed runtime, i.e. OpenJDK. Our results demonstrate that Forseti enables dramatic reductions (up to 5x) in heap footprint without compromising application execution times