Provisioning multi-tier cloud applications using statistical bounds on sojourn time

In this paper we present a simple and effective approach for re-source provisioning to achieve a percentile bound on the end to end response time of a multi-tier application. We, at first, model the multi-tier application as an open tandem network of M/G/1-PS queues and develop a method that produces a near optimal appli-cation configuration, i.e, number of servers at each tier, to meet the percentile bound in a homogeneous server environment – using a single type of server. We then extend our solution to a K-server case and our technique demonstrates a good accuracy, independent of the variability of service-times. Our approach demonstrates a provisioning error of no more than 3 % compared to a 140 % worst case provisioning error obtained by techniques based on anM/M/1-FCFS queue model. In addition, we extend our approach to han-dle a heterogenous server environment, i.e., with multiple types of servers. We find that fewer high-capacity servers are preferable for high percentile provisioning. Finally, we extend our approach to account for the rental cost of each server-type and compute a cost efficient application configuration with savings of over 80%. We demonstrate the applicability of our approach in a real world sys-tem by employing it to provision the two tiers of the java implemen-tation of TPC-W – a multi-tier transactional web benchmark that represents an e-commerce web application, i.e. an online book-store