ATOM: model-driven autoscaling for microservices

Microservices based architectures are increasinglywidespread in the cloud software industry. Still, there is ashortage of auto-scaling methods designed to leverage the uniquefeatures of these architectures, such as the ability to indepen-dently scale a subset of microservices, as well as the ease ofmonitoring their state and reciprocal calls.We propose to address this shortage with ATOM, a model-driven autoscaling controller for microservices. ATOM instanti-ates and solves at run-time a layered queueing network model ofthe application. Computational optimization is used to dynami-cally control the number of replicas for each microservice and itsassociated container CPU share, overall achieving a fine-grainedcontrol of the application capacity at run-time.Experimental results indicate that for heavy workloads ATOMoffers around 30%-37% higher throughput than baseline model-agnostic controllers based on simple static rules. We also find thatmodel-driven reasoning reduces the number of actions needed toscale the system as it reduces the number of bottleneck shiftsthat we observe with model-agnostic controllers

Model-based Resource Management for Fine-grained Services

Brief Biography: Alim Ul Gias is currently a Research Associate at the Centre for Parallel Computing (CPC), University of Westminster. He completed his PhD from Imperial College London in 2022. Before starting his PhD, Alim was a lecturer at Institute of Information Technology (IIT), University of Dhaka (DU). He completed his bachelor's and master's program from the same institute. His current research focuses on different Quality of Service (QoS) aspects of cloud-native applications e.g., microservices. In particular, he aims to address the performance and resource management challenges concenrining the microservices architecture

Self-Learning Cloud Controllers: Fuzzy Q-Learning for Knowledge Evolution

Cloud controllers aim at responding to application demands by automatically scaling the compute resources at runtime to meet performance guarantees and minimize resource costs. Existing cloud controllers often resort to scaling strategies that are codified as a set of adaptation rules. However, for a cloud provider, applications running on top of the cloud infrastructure are more or less black-boxes, making it difficult at design time to define optimal or pre-emptive adaptation rules. Thus, the burden of taking adaptation decisions often is delegated to the cloud application. Yet, in most cases, application developers in turn have limited knowledge of the cloud infrastructure. In this paper, we propose learning adaptation rules during runtime. To this end, we introduce FQL4KE, a self-learning fuzzy cloud controller. In particular, FQL4KE learns and modifies fuzzy rules at runtime. The benefit is that for designing cloud controllers, we do not have to rely solely on precise design-time knowledge, which may be difficult to acquire. FQL4KE empowers users to specify cloud controllers by simply adjusting weights representing priorities in system goals instead of specifying complex adaptation rules. The applicability of FQL4KE has been experimentally assessed as part of the cloud application framework ElasticBench. The experimental results indicate that FQL4KE outperforms our previously developed fuzzy controller without learning mechanisms and the native Azure auto-scaling

A Reliable and Cost-Efficient Auto-Scaling System for Web Applications Using Heterogeneous Spot Instances

Cloud providers sell their idle capacity on markets through an auction-like mechanism to increase their return on investment. The instances sold in this way are called spot instances. In spite that spot instances are usually 90% cheaper than on-demand instances, they can be terminated by provider when their bidding prices are lower than market prices. Thus, they are largely used to provision fault-tolerant applications only. In this paper, we explore how to utilize spot instances to provision web applications, which are usually considered availability-critical. The idea is to take advantage of differences in price among various types of spot instances to reach both high availability and significant cost saving. We first propose a fault-tolerant model for web applications provisioned by spot instances. Based on that, we devise novel auto-scaling polices for hourly billed cloud markets. We implemented the proposed model and policies both on a simulation testbed for repeatable validation and Amazon EC2. The experiments on the simulation testbed and the real platform against the benchmarks show that the proposed approach can greatly reduce resource cost and still achieve satisfactory Quality of Service (QoS) in terms of response time and availability

Towards a Cloud Native Big Data Platform using MiCADO

In the big data era, creating self-managing scalable platforms for running big data applications is a fundamental task. Such self-managing and self-healing platforms involve a proper reaction to hardware (e.g., cluster nodes) and software (e.g., big data tools) failures, besides a dynamic resizing of the allocated resources based on overload and underload situations and scaling policies. The distributed and stateful nature of big data platforms (e.g., Hadoop-based cluster) makes the management of these platforms a challenging task. This paper aims to design and implement a scalable cloud native Hadoop-based big data platform using MiCADO, an open-source, and a highly customisable multi-cloud orchestration and auto-scaling framework for Docker containers, orchestrated by Kubernetes. The proposed MiCADO-based big data platform automates the deployment and enables an automatic horizontal scaling (in and out) of the underlying cloud infrastructure. The empirical evaluation of the MiCADO-based big data platform demonstrates how easy, efficient, and fast it is to deploy and undeploy Hadoop clusters of different sizes. Additionally, it shows how the platform can automatically be scaled based on user-defined policies (such as CPU-based scaling)

Proactive-Reactive Global Scaling, with Analytics

International audienceIn this work, we focus on by-design global scaling, a technique that, given a functional specification of a microservice architecture, or-chestrates the scaling of all its components, avoiding cascading slowdowns typical of uncoordinated, mainstream autoscaling. State-of-the-art by-design global scaling adopts a reactive approach to traffic fluctuations, undergoing inefficiencies due to the reaction overhead. Here, we tacklethis problem by proposing a proactive version of by-design global scaling able to anticipate future scaling actions. We provide four contributionsin this direction: i) a platform able to host both reactive and proactive global scaling; ii) a proactive implementation based on data analytics; iii)a hybrid solution that mixes reactive and proactive scaling; iv) use cases and empirical benchmarks, obtained through our platform, that comparereactive, proactive, and hybrid global scaling performance. From our comparison, proactive global scaling consistently outperforms reactive,while the hybrid solution is the best-performing one