D-SPACE4Cloud: A Design Tool for Big Data Applications

The last years have seen a steep rise in data generation worldwide, with the development and widespread adoption of several software projects targeting the Big Data paradigm. Many companies currently engage in Big Data analytics as part of their core business activities, nonetheless there are no tools and techniques to support the design of the underlying hardware configuration backing such systems. In particular, the focus in this report is set on Cloud deployed clusters, which represent a cost-effective alternative to on premises installations. We propose a novel tool implementing a battery of optimization and prediction techniques integrated so as to efficiently assess several alternative resource configurations, in order to determine the minimum cost cluster deployment satisfying QoS constraints. Further, the experimental campaign conducted on real systems shows the validity and relevance of the proposed method

AWS PredSpot: Machine Learning for Predicting the Price of Spot Instances in AWS Cloud

Elastic Cloud Compute (EC2) is one of the most well-known services provided by Amazon for provisioning cloud computing resources, also known as instances. Besides the classical on-demand scheme, where users purchase compute capacity at a fixed cost, EC2 supports so-called spot instances, which are offered following a bidding scheme, where users can save up to 90% of the cost of the on-demand instance. EC2 spot instances can be a useful alternative for attaining an important reduction in infrastructure cost, but designing bidding policies can be a difficult task, since bidding under their cost will either prevent users from provisioning instances or losing those that they already own. Towards this extent, accurate forecasting of spot instance prices can be of an outstanding interest for designing working bidding policies. In this paper, we propose the use of different machine learning techniques to estimate the future price of EC2 spot instances. These include linear, ridge and lasso regressions, multilayer perceptrons, K-nearest neighbors, extra trees and random forests. The obtained performance varies significantly between instances types, and root mean squared errors ranges between values very close to zero up to values over 60 in some of the most expensive instances. Still, we can see that for most of the instances, forecasting performance is remarkably good, encouraging further research in this field of study

Price forecasting for spot instances in Cloud computing

[EN] Big data applications usually need to rent a large number of virtual machines from Cloud computing providers. As a result of the policies employed by Cloud providers, the prices of spot virtual machine instances behavior stochastically. Spot prices (prices of spot instances) fluctuate greatly or have multiple regimes. Choosing virtual machines according to trends in prices is helpful in decreasing the resource rental cost. Existing price prediction methods are unable to accurately predict prices in these environments. As a result, a dynamic-ARIMA and two markov regime-switching autoregressive model based forecasting methods have been developed in this paper. Experimental results show that the proposals are better than the existing MonthAR in most scenarios. (C) 2017 Elsevier B.V. All rights reserved.The authors would like to thank the reviewers for their constructive and useful comments. This work is supported by the National Natural Science Foundation of China (Grant No. 61602243 and No. 61572127), the Natural Science Foundation of Jiangsu Province (Grant No. BK20160846), Jiangsu Key Laboratory of Image and Video Understanding for Social Safety (Grant No. 30916014107). Ruben Ruiz is partially supported by the Spanish Ministry of Economy and Competitiveness, under the project "SCHEYARD" (No. DPI2015-65895-R) financed by FEDER funds.Cai, Z.; Li, X.; Ruiz García, R.; Li, Q. (2018). Price forecasting for spot instances in Cloud computing. Future Generation Computer Systems. 79:38-53. https://doi.org/10.1016/j.future.2017.09.038S38537