1 research outputs found
Overbooking Microservices in the Cloud
We consider the problem of scheduling serverless-computing instances such as
Amazon Lambda functions, or scheduling microservices within (privately held)
virtual machines (VMs). Instead of a quota per tenant/customer, we assume
demand for Lambda functions is modulated by token-bucket mechanisms per tenant.
Such quotas are due to, e.g., limited resources (as in a fog/edge-cloud
context) or to prevent excessive unauthorized invocation of numerous instances
by malware. Based on an upper bound on the stationary number of active "Lambda
servers" considering the execution-time distribution of Lambda functions, we
describe an approach that the cloud could use to overbook Lambda functions for
improved utilization of IT resources. An earlier bound for a single service
tier is extended to multiple service tiers. For the context of scheduling
microservices in a private setting, the framework could be used to determine
the required VM resources for a token-bucket constrained workload stream.
Finally, we note that the looser Markov inequality may be useful in settings
where the job service times are dependent