FaaSdom: A Benchmark Suite for Serverless Computing

Serverless computing has become a major trend among cloud providers. With serverless computing, developers fully delegate the task of managing the servers, dynamically allocating the required resources, as well as handling availability and fault-tolerance matters to the cloud provider. In doing so, developers can solely focus on the application logic of their software, which is then deployed and completely managed in the cloud. Despite its increasing popularity, not much is known regarding the actual system performance achievable on the currently available serverless platforms. Specifically, it is cumbersome to benchmark such systems in a language- or runtime-independent manner. Instead, one must resort to a full application deployment, to later take informed decisions on the most convenient solution along several dimensions, including performance and economic costs. FaaSdom is a modular architecture and proof-of-concept implementation of a benchmark suite for serverless computing platforms. It currently supports the current mainstream serverless cloud providers (i.e., AWS, Azure, Google, IBM), a large set of benchmark tests and a variety of implementation languages. The suite fully automatizes the deployment, execution and clean-up of such tests, providing insights (including historical) on the performance observed by serverless applications. FaaSdom also integrates a model to estimate budget costs for deployments across the supported providers. FaaSdom is open-source and available at https://github.com/bschitter/benchmark-suite-serverless-computing.Comment: ACM DEBS'2

Reconstructable Software Appliances with Kameleon

International audienceA software appliance builder bundles together an application with its needed middleware and an operating system to allow easy deployment on Infrastructure as a Service (IaaS) providers. These builders have the potential to address a key need in our community: the ability to reproduce an experiment. This paper reports the experiences on developing a software appliance builder called Kameleon that leverages popular and well tested tools. Kameleon simplifies the creation of complex software appliances that are targeted at research on operating systems, HPC and distributed computing. It does so by proposing a highly modular description format that encourages collaboration and reuse of procedures. Moreover, it provides debugging mechanisms for improving experimenter's productivity. To justify that our appliance builder stands above others, we compare it with the most known tools used by developers and researchers to automate the construction of software environments for virtual machines and IaaS infrastructures

Evaluation of Serverless Computing Frameworks Based on Kubernetes

Recent advancements in virtualization and software architectures have led to the birth of the new paradigm of serverless computing. Serverless computing, also known as function-as-a-service, allows developers to deploy functions as computing units without worrying about the underlying infrastructure. Moreover, no resources are allocated or billed until a function is invoked. Thus, the major benefits of serverless computing are reduced developer concern about infrastructure, reduced time to market and lower cost. Currently, serverless computing is generally available through various public cloud service providers. However, there are certain bottlenecks on public cloud platforms, such as vendor lock-in, computation restrictions and regulatory restrictions. Thus, there is a growing interest to implement serverless computing on a private infrastructure. One of the preferred ways of implementing serverless computing is through the use of containers. A container-based solution allows to utilize features of existing orchestration frameworks, such as Kubernetes. This thesis discusses the implementation of serverless computing on Kubernetes. To this end, we carry out a feature evaluation of four open source serverless computing frameworks, namely Kubeless, OpenFaaS, Fission and OpenWhisk. Based on predefined criteria, we select Kubeless, Fission and OpenFaaS for further evaluation. First, we describe the developer experience on each framework. Next, we compare three different modes in which OpenFaaS functions are executed: HTTP, serializing and streaming. We evaluate the response time of function invocation and ease of monitoring and management of logs. We find that HTTP mode is the preferred mode for OpenFaaS. Finally, we evaluate the performance of the considered frameworks under different workloads. We find that Kubeless has the best performance among the three frameworks, both in terms of response time and the ratio of successful responses