3,352 research outputs found
A Reliable and Cost-Efficient Auto-Scaling System for Web Applications Using Heterogeneous Spot Instances
Cloud providers sell their idle capacity on markets through an auction-like
mechanism to increase their return on investment. The instances sold in this
way are called spot instances. In spite that spot instances are usually 90%
cheaper than on-demand instances, they can be terminated by provider when their
bidding prices are lower than market prices. Thus, they are largely used to
provision fault-tolerant applications only. In this paper, we explore how to
utilize spot instances to provision web applications, which are usually
considered availability-critical. The idea is to take advantage of differences
in price among various types of spot instances to reach both high availability
and significant cost saving. We first propose a fault-tolerant model for web
applications provisioned by spot instances. Based on that, we devise novel
auto-scaling polices for hourly billed cloud markets. We implemented the
proposed model and policies both on a simulation testbed for repeatable
validation and Amazon EC2. The experiments on the simulation testbed and the
real platform against the benchmarks show that the proposed approach can
greatly reduce resource cost and still achieve satisfactory Quality of Service
(QoS) in terms of response time and availability
Multi-Level ML Based Burst-Aware Autoscaling for SLO Assurance and Cost Efficiency
Autoscaling is a technology to automatically scale the resources provided to
their applications without human intervention to guarantee runtime Quality of
Service (QoS) while saving costs. However, user-facing cloud applications serve
dynamic workloads that often exhibit variable and contain bursts, posing
challenges to autoscaling for maintaining QoS within Service-Level Objectives
(SLOs). Conservative strategies risk over-provisioning, while aggressive ones
may cause SLO violations, making it more challenging to design effective
autoscaling. This paper introduces BAScaler, a Burst-Aware Autoscaling
framework for containerized cloud services or applications under complex
workloads, combining multi-level machine learning (ML) techniques to mitigate
SLO violations while saving costs. BAScaler incorporates a novel
prediction-based burst detection mechanism that distinguishes between
predictable periodic workload spikes and actual bursts. When bursts are
detected, BAScaler appropriately overestimates them and allocates resources
accordingly to address the rapid growth in resource demand. On the other hand,
BAScaler employs reinforcement learning to rectify potential inaccuracies in
resource estimation, enabling more precise resource allocation during
non-bursts. Experiments across ten real-world workloads demonstrate BAScaler's
effectiveness, achieving a 57% average reduction in SLO violations and cutting
resource costs by 10% compared to other prominent methods
Cloud Computing Systems Exploration over Workload Prediction Factor in Distributed Applications
This paper highlights the different techniques of workload prediction in cloud computing. Cloud computing resources have a special kind of arrangement in which resources are made available on demand to the customers. Today, most of the organizations are using cloud computing that results in reduction of the operational cost. Cloud computing also reduces the overhead of any organization due to implementation of many hardware and software platforms. These services are being provided by cloud provider on the basis of pay per use. There are lots of cloud service providers in the modern era. In this competitive era, every cloud provider works to provide better services to the customer. To fulfill the customer?s requirements, dynamic provisioning can serve the purpose in cloud system where resources can be released and allocated on later stage as per needs. That?s why resource scaling becomes a great challenge for the cloud providers. There are many approaches to scale the number of instances of any resource. Two main approaches namely: proactive and reactive are used in cloud systems. Reactive approach reacts at later stage while proactive approach predicts resources in advance. Cloud provider needs to predict the number of resources in advance that an application is intended to use. Historical data and patterns can be used for the workload prediction. The benefit of the proactive approach lies in advance number of instances of a resource available for the future use. This results in improved performance for the cloud systems
Burst-aware predictive autoscaling for containerized microservices
Autoscaling methods are used for cloud-hosted applications to dynamically scale the allocated resources for guaranteeing Quality-of-Service (QoS). The public-facing application serves dynamic workloads, which contain bursts and pose challenges for autoscaling methods to ensure application performance. Existing State-of-the-art autoscaling methods are burst-oblivious to determine and provision the appropriate resources. For dynamic workloads, it is hard to detect and handle bursts online for maintaining application performance. In this article, we propose a novel burst-aware autoscaling method which detects burst in dynamic workloads using workload forecasting, resource prediction, and scaling decision making while minimizing response time service-level objectives (SLO) violations. We evaluated our approach through a trace-driven simulation, using multiple synthetic and realistic bursty workloads for containerized microservices, improving performance when comparing against existing state-of-the-art autoscaling methods. Such experiments show an increase of Ă— 1.09 in total processed requests, a reduction of Ă— 5.17 for SLO violations, and an increase of Ă— 0.767 cost as compared to the baseline method.This work was partially supported by the European Research Council (ERC) under the EU Horizon 2020 programme (GA 639595), the Spanish Ministry of Economy, Industry and Competitiveness (TIN2015-65316-P and IJCI2016-27485) and the Generalitat de Catalunya (2014-SGR-1051).Peer ReviewedPostprint (author's final draft
OptScaler: A Hybrid Proactive-Reactive Framework for Robust Autoscaling in the Cloud
Autoscaling is a vital mechanism in cloud computing that supports the
autonomous adjustment of computing resources under dynamic workloads. A primary
goal of autoscaling is to stabilize resource utilization at a desirable level,
thus reconciling the need for resource-saving with the satisfaction of Service
Level Objectives (SLOs). Existing proactive autoscaling methods anticipate the
future workload and scale the resources in advance, whereas the reliability may
suffer from prediction deviations arising from the frequent fluctuations and
noise of cloud workloads; reactive methods rely on real-time system feedback,
while the hysteretic nature of reactive methods could cause violations of the
rigorous SLOs. To this end, this paper presents OptScaler, a hybrid autoscaling
framework that integrates the power of both proactive and reactive methods for
regulating CPU utilization. Specifically, the proactive module of OptScaler
consists of a sophisticated workload prediction model and an optimization
model, where the former provides reliable inputs to the latter for making
optimal scaling decisions. The reactive module provides a self-tuning estimator
of CPU utilization to the optimization model. We embed Model Predictive Control
(MPC) mechanism and robust optimization techniques into the optimization model
to further enhance its reliability. Numerical results have demonstrated the
superiority of both the workload prediction model and the hybrid framework of
OptScaler in the scenario of online services compared to prevalent reactive,
proactive, or hybrid autoscalers. OptScaler has been successfully deployed at
Alipay, supporting the autoscaling of applets in the world-leading payment
platform
- …