Prediction based scaling in a distributed stream processing cluster

2020 Spring.Includes bibliographical references.Proliferation of IoT sensors and applications have enabled us to monitor and analyze scientific and social phenomena with continuously arriving voluminous data. To provide real-time processing capabilities over streaming data, distributed stream processing engines (DSPEs) such as Apache STORM and Apache FLINK have been widely deployed. These frameworks support computations over large-scale, high frequency streaming data. However, current on-demand auto-scaling features in these systems may result in an inefficient resource utilization which is closely related to cost effectiveness in popular cloud-based computing environments. We propose ARSTREAM, an auto-scaling computing environment that manages fluctuating throughputs for data from sensor networks, while ensuring efficient resource utilization. We have built an Artificial Neural Network model for predicting data processing queues and this model captures non-linear relationships between data arrival rates, resource utilization, and the size of data processing queue. If a bottleneck is predicted, ARSTREAM scales-out the current cluster automatically for current jobs without halting them at the user level. In addition, ARSTREAM incorporates threshold-based re-balancing to minimize data loss during extreme peak traffic that could not be predicted by our model. Our empirical benchmarks show that ARSTREAM forecasts data processing queue sizes with RMSE of 0.0429 when tested on real-time data

Wide area network autoscaling for cloud applications

Modern cloud orchestrators like Kubernetes provide a versatile and robust way to host applications at scale. One of their key features is autoscaling, that automatically adjusts cloud resources (compute, memory, storage) in order to dynamically adapt to the demands of the application. However, the scope of cloud autoscaling is limited to the datacenter hosting the cloud and it doesn't apply uniformly to the allocation of network resources. In I/O-constrained or data-in-motion use cases this can lead to severe performance degradation for the application. For example, when the load on a cloud service increases and the Wide Area Network (WAN) connecting the datacenter to the Internet becomes saturated, the application experiences an increase in delay and loss. In many cases this is dealt by overprovisioning network capacity, which introduces significant additional costs and inefficiencies. On the other hand, thanks to the concept of "Network as Code", the WAN today exposes a programmable set ofAPIs that can be used to dynamically allocate and deallocate capacity on-demand. In this paper we propose extending the concept of cloud autoscaling into the network to address this limitation. This way, applications running in the cloud can communicate their networking requirements, like bandwidth or traffic profile, to an SDN controller or Network as a Service (NaaS) platform. Moreover, we aim to define the concepts of vertical and horizontal autoscaling applied to networking. We present a prototype that automatically allocates bandwidth in the underlay of an SD-WAN, according to the requirements of the applications hosted in Kubernetes. Finally, we discuss open research challenges

Investigating into Cloud Resource Management Mechanisms

Driven by the rapid growth of the demand for efficient and economical computational power, cloud computing has led the world into a new era. It delivers computing resources as services, whereby shared resources are provided to cloud users over the network in order to offer dynamic flexible resource provisioning for reliable and guaranteed services by using pay-as-you-use pricing model. Since multiple cloud users can request cloud resources simultaneously, cloud resource management mechanisms must operate in an efficient manner to satisfy demand of cloud users. Therefore, investigating cloud resource management mechanisms to achieve cloud resource efficiency is one of key elements that benefits both cloud providers and users. In this thesis, we present cloud resource management mechanisms for two different cloud infrastructures, i.e. virtual machine-based (VM-based) and application-based infrastructure. The VM-based infrastructure is an infrastructure that provides multi-tenancy for cloud users at VM-level, i.e. each cloud user directly controls their VMs in the cloud environment. The application-based infrastructure provides multi-tenancy at application level, in the other word, each cloud user directly control their applications in the cloud environment. For the VM-based infrastructure, we introduce two heuristics metrics to capture multi-dimensional characteristics of logical machines. By using a multivariate probabilistic model, we develop an algorithm to improve resource utilisation for the VM-based infrastructure. We then designed and implemented an application-based infrastructure called Elastic Application Container system (EAC system) to support multi-tenant cloud use. Based on the characteristics of the application-based and the VM-based infrastructure, we developed auto-scaling algorithms that can automatically scale cloud resources in the EAC system. In general, the cloud resource management mechanisms proposed in this thesis aims to investigate resource management mechanisms for cloud resource utilisation in the VM-based infrastructure and to provide suitable cloud resource provisioning mechanisms for the application-based infrastructure.Imperial Users Onl

An Intelligent model for supporting Edge Migration for Virtual Function Chains in Next Generation Internet of Things

The developments on next generation IoT sensing devices, with the advances on their low power computational capabilities and high speed networking has led to the introduction of the edge computing paradigm. Within an edge cloud environment, services may generate and consume data locally, without involving cloud computing infrastructures. Aiming to tackle the low computational resources of the IoT nodes, Virtual-Function-Chain has been proposed as an intelligent distribution model for exploiting the maximum of the computational power at the edge, thus enabling the support of demanding services. An intelligent migration model with the capacity to support Virtual-Function-Chains is introduced in this work. According to this model, migration at the edge can support individual features of a Virtual-Function-Chain. First, auto-healing can be implemented with cold migrations, if a Virtual Function fails unexpectedly. Second, a Quality of Service monitoring model can trigger live migrations, aiming to avoid edge devices overload. The evaluation studies of the proposed model revealed that it has the capacity to increase the robustness of an edge-based service on low-powered IoT devices. Finally, comparison with similar frameworks, like Kubernetes, showed that the migration model can effectively react on edge network fluctuations