1 research outputs found
Performance-Feedback Autoscaling with Budget Constraints for Cloud-based Workloads of Workflows
The growing popularity of workflows in the cloud domain promoted the
development of sophisticated autoscaling policies that allow automatic
allocation and deallocation of resources. However, many state-of-the-art
autoscaling policies for workflows are mostly plan-based or designed for
batches (ensembles) of workflows. This reduces their flexibility when dealing
with workloads of workflows, as the workloads are often subject to
unpredictable resource demand fluctuations. Moreover, autoscaling in clouds
almost always imposes budget constraints that should be satisfied. The
budget-aware autoscalers for workflows usually require task runtime estimates
to be provided beforehand, which is not always possible when dealing with
workloads due to their dynamic nature. To address these issues, we propose a
novel Performance-Feedback Autoscaler (PFA) that is budget-aware and does not
require task runtime estimates for its operation. Instead, it uses the
performance-feedback loop that monitors the average throughput on each resource
type. We implement PFA in the popular Apache Airflow workflow management
system, and compare the performance of our autoscaler with other two
state-of-the-art autoscalers, and with the optimal solution obtained with the
Mixed Integer Programming approach. Our results show that PFA outperforms other
considered online autoscalers, as it effectively minimizes the average job
slowdown by up to 47% while still satisfying the budget constraints. Moreover,
PFA shows by up to 76% lower average runtime than the competitors.Comment: Technical Repor