Orchestration and management of application functions over virtualized cloud infrastructures

Next-generation networks are expected to provide higher data rates and ultra-low latency in support of demanding applications, such as virtual and augmented reality, robots and drones, etc. To meet these stringent requirements of applications, edge computing constitutes a central piece of the solution architecture wherein functional components of an application can be deployed over the edge network to reduce bandwidth demand over the core network while providing ultra-low latency communication to users. In this thesis, we provide solutions to resource orchestration and management for applications over a virtualized client-edge-server infrastructure. We first investigate the problem of optimal placement of pipelines of application functions (virtual service chains) and the steering of traffic through them, over a multi-technology edge network model consisting of both wired and wireless millimeter-wave (mmWave) links. This problem is NP-hard. We provide a comprehensive “microscopic” binary integer program to model the system, along with a heuristic that is one order of magnitude faster than optimally solving the problem. Extensive evaluations demonstrate the benefits of orchestrating virtual service chains (by distributing them over the edge network) compared to a baseline “middlebox” approach in terms of overall admissible virtual capacity. Next, we look at the problem of finding the optimal configuration parameters, such as memory and CPU, for application functions running as serverless functions, i.e. they run in stateless compute containers that are event-driven, ephemeral, and fully managed by the cloud provider. While serverless computing is a relatively simpler computing model, configuring such parameters correctly while minimizing cost and meeting delay constraints is not trivial. To solve this problem, we present a framework that uses Bayesian Optimization to find the optimal configuration for serverless functions. The framework uses statistical learning techniques to intelligently collect samples with the goal of predicting the cost and execution time of a serverless function across unseen configuration values. Our framework uses the predicted cost and execution time to select the “best” configuration parameters for running a single or a chain of serverless functions (service chains). Evaluations on a commercial cloud provider and a wide range of simulated distributed cloud environments confirm the efficacy of our approach.2021-02-10T00:00:00

Serving deep learning models in a serverless platform

Serverless computing has emerged as a compelling paradigm for the development and deployment of a wide range of event based cloud applications. At the same time, cloud providers and enterprise companies are heavily adopting machine learning and Artificial Intelligence to either differentiate themselves, or provide their customers with value added services. In this work we evaluate the suitability of a serverless computing environment for the inferencing of large neural network models. Our experimental evaluations are executed on the AWS Lambda environment using the MxNet deep learning framework. Our experimental results show that while the inferencing latency can be within an acceptable range, longer delays due to cold starts can skew the latency distribution and hence risk violating more stringent SLAs

Next Generation Cloud Computing: New Trends and Research Directions

The landscape of cloud computing has significantly changed over the last decade. Not only have more providers and service offerings crowded the space, but also cloud infrastructure that was traditionally limited to single provider data centers is now evolving. In this paper, we firstly discuss the changing cloud infrastructure and consider the use of infrastructure from multiple providers and the benefit of decentralising computing away from data centers. These trends have resulted in the need for a variety of new computing architectures that will be offered by future cloud infrastructure. These architectures are anticipated to impact areas, such as connecting people and devices, data-intensive computing, the service space and self-learning systems. Finally, we lay out a roadmap of challenges that will need to be addressed for realising the potential of next generation cloud systems.Comment: Accepted to Future Generation Computer Systems, 07 September 201