Impact of User Patience on Auto-Scaling Resource Capacity for Cloud Services

International audienceAn important feature of most cloud computing solutions is auto-scaling, an operation that enables dynamic changes on resource capacity. Auto-scaling algorithms generally take into account aspects such as system load and response time to determine when and by how much a resource pool capacity should be extended or shrunk. In this article, we propose a scheduling algorithm and auto-scaling triggering strategies that explore user patience, a metric that estimates the perception end-users have from the Quality of Service (QoS) delivered by a service provider based on the ratio between expected and actual response times for each request. The proposed strategies help reduce costs with resource allocation while maintaining perceived QoS at adequate levels. Results show reductions on resource-hour consumption by up to approximately 9% compared to traditional approaches

Resource provisioning in Science Clouds: Requirements and challenges

Cloud computing has permeated into the information technology industry in the last few years, and it is emerging nowadays in scientific environments. Science user communities are demanding a broad range of computing power to satisfy the needs of high-performance applications, such as local clusters, high-performance computing systems, and computing grids. Different workloads are needed from different computational models, and the cloud is already considered as a promising paradigm. The scheduling and allocation of resources is always a challenging matter in any form of computation and clouds are not an exception. Science applications have unique features that differentiate their workloads, hence, their requirements have to be taken into consideration to be fulfilled when building a Science Cloud. This paper will discuss what are the main scheduling and resource allocation challenges for any Infrastructure as a Service provider supporting scientific applications

A case study of proactive auto-scaling for an ecommerce workload

Preliminary data obtained from a partnership between the Federal University of Campina Grande and an ecommerce company indicates that some applications have issues when dealing with variable demand. This happens because a delay in scaling resources leads to performance degradation and, in literature, is a matter usually treated by improving the auto-scaling. To better understand the current state-of-the-art on this subject, we re-evaluate an auto-scaling algorithm proposed in the literature, in the context of ecommerce, using a long-term real workload. Experimental results show that our proactive approach is able to achieve an accuracy of up to 94 percent and led the auto-scaling to a better performance than the reactive approach currently used by the ecommerce company

On the Impact of Advance Reservations for Energy-Aware Provisioning of Bare-Metal Cloud Resources

International audienceThis work investigates factors that can impact the elasticity of bare-metal resources. We analyse data from a real bare-metal deployment system to build a deployment time model, which is used to evaluate how provisioning time impacts the reservation of bare-metal resources. Climate/Blazar, a reservation framework designed for OpenStack, is discussed. Simulation results show that reservations can help reduce the time to deliver a provisioned cluster to its customer while achieving energy savings similar to those of strategies that switch-off idle resources

Users’ time preference based stochastic resource allocation in cloud spot market: cloud provider’s perspective

Cloud Computing spot markets have enabled the users to make use of the spare computing capacities of the cloud providers at a relatively cheaper price which in turn has given the providers such as Amazon and Google an opportunity to earn extra money by auctioning-off the underutilized resources. However, resource availability is a problem in the spot market owing to spot-price fluctuations. Ignoring the customer’s preference is one of the potential reasons behind this. In this paper, we propose a time preference (value of service at different points of time) based stochastic integer linear programming model to allocate the cloud resources among the cloud users with a view to maximizing the revenue of cloud providers from the spot-market

Achieving Continuous Delivery of Immutable Containerized Microservices with Mesos/Marathon

In the recent years, DevOps methodologies have been introduced to extend the traditional agile principles which have brought up on us a paradigm shift in migrating applications towards a cloud-native architecture. Today, microservices, containers, and Continuous Integration/Continuous Delivery have become critical to any organization’s transformation journey towards developing lean artifacts and dealing with the growing demand of pushing new features, iterating rapidly to keep the customers happy. Traditionally, applications have been packaged and delivered in virtual machines. But, with the adoption of microservices architectures, containerized applications are becoming the standard way to deploy services to production. Thanks to container orchestration tools like Marathon, containers can now be deployed and monitored at scale with ease. Microservices and Containers along with Container Orchestration tools disrupt and redefine DevOps, especially the delivery pipeline. This Master’s thesis project focuses on deploying highly scalable microservices packed as immutable containers onto a Mesos cluster using a container orchestrating framework called Marathon. This is achieved by implementing a CI/CD pipeline and bringing in to play some of the greatest and latest practices and tools like Docker, Terraform, Jenkins, Consul, Vault, Prometheus, etc. The thesis is aimed to showcase why we need to design systems around microservices architecture, packaging cloud-native applications into containers, service discovery and many other latest trends within the DevOps realm that contribute to the continuous delivery pipeline. At BetterDoctor Inc., it is observed that this project improved the avg. release cycle, increased team members’ productivity and collaboration, reduced infrastructure costs and deployment failure rates. With the CD pipeline in place along with container orchestration tools it has been observed that the organisation could achieve Hyperscale computing as and when business demands

Scalable invoice-based B2B payments with microservices

Paying by invoice has several advantages for businesses over conventional payment methods such as Debit/Credit cards. An invoice not only allows a buyer to make the purchase on credit but also contains the tax information for each purchased item. Businesses need to save this information in their financial records and report it to the authorities. The core challenge in an invoice-based payment method is the ability to make an accurate credit decision for a given purchase. Such a credit decision requires information about the buying company such as their credit rating. The company information is gathered in real time from different third-party sources. In this context, Enterpay Oy provides an invoiced-based B2B payment solution and is growing its payment service to European countries. In order to support this expansion, Enterpay needs to develop new capabilities such as the ability to detect fraudulent purchases. These new features require the application architecture to be flexible in terms of technology. For example, different components of the service should be built with the most suited programming language, libraries, and frameworks. The goal of this thesis is to enable efficient scaling and high availability for Enterpay's payment service. Thus, we have migrated from a monolithic a microservice-based architecture. This transition allows us to choose the best suited technology for the business case of the given microservice. We extracted various modules from the original monolithic application, which have different scalability criteria. We built these modules as Docker containers, which run as independent microservices. We used Kubernetes as the container orchestration framework and deployed the microservice in Amazon Web Services (AWS). Finally, we conducted experiments to measure the performance of the service with the new architecture. We found that this architecture not only scales faster but also recovers from instance failures quicker than the previous solution. Additionally, we noticed that the average response time of service request is similar in both architectures. Finally, we observed that new microservices can be built using different technology stack and deployed conveniently in the same Kubernetes cluster

An efficient cloud scheduler design supporting preemptible instances

Maximizing resource utilization by performing an efficient resource provisioning is a key factor for any cloud provider: commercial actors can maximize their revenues, whereas scientific and non-commercial providers can maximize their infrastructure utilization. Traditionally, batch systems have allowed data centers to fill their resources as much as possible by using backfilling and similar techniques. However, in an IaaS cloud, where virtual machines are supposed to live indefinitely, or at least as long as the user is able to pay for them, these policies are not easily implementable. In this work we present a new scheduling algorithm for IaaS providers that is able to support preemptible instances, that can be stopped by higher priority requests without introducing large modifications in the current cloud schedulers. This scheduler enables the implementation of new cloud usage and payment models that allow more efficient usage of the resources and potential new revenue sources for commercial providers. We also study the correctness and the performace overhead of the proposed scheduler agains existing solutions

A highly-available and scalable microservice architecture for access management

Access management is a key aspect of providing secure services and applications in information technology. Ensuring secure access is particularly challenging in a cloud environment wherein resources are scaled dynamically. In fact keeping track of dynamic cloud instances and administering access to them requires careful coordination and mechanisms to ensure reliable operations. PrivX is a commercial offering from SSH Communications and Security Oyj that automatically scans and keeps track of the cloud instances and manages access to them. PrivX is currently built on the microservices approach, wherein the application is structured as a collection of loosely coupled services. However, PrivX requires external modules and with specific capabilities to ensure high availability. Moreover, complex scripts are required to monitor the whole system. The goal of this thesis is to make PrivX highly-available and scalable by using a container orchestration framework. To this end, we first conduct a detailed study of mostly widely used container orchestration frameworks: Kubernetes, Docker Swarm and Nomad. We then select Kubernetes based on a feature evaluation relevant to the considered scenario. We package the individual components of PrivX, including its database, into Docker containers and deploy them on a Kubernetes cluster. We also build a prototype system to demonstrate how microservices can be managed on a Kubernetes cluster. Additionally, an auto scaling tool is created to scale specific services based on predefined rules. Finally, we evaluate the service recovery time for each of the services in PrivX, both in the RPM deployment model and the prototype Kubernetes deployment model. We find that there is no significant difference in service recovery time between the two models. However, Kubernetes ensured high availability of the services. We find that Kubernetes is the preferred mode for deploying PrivX and it makes PrivX highly available and scalable