Resource boxing: Converting realistic cloud task utilization patterns for theoretical scheduling

Scheduling is a core component within distributed systems to determine optimal allocation of tasks within servers. This is challenging within modern Cloud computing systems - comprising millions of tasks executing in thousands of heterogeneous servers. Theoretical scheduling is capable of providing complete and sophisticated algorithms towards a single objective function. However, Cloud computing systems pursue multiple and oftentimes conflicting objectives towards provisioning high levels of performance, availability, reliability and energy-efficiency. As a result, theoretical scheduling for Cloud computing is performed by simplifying assumptions for applicability. This is especially true for task utilization patterns, which fluctuate in practice yet are modelled as piecewise constant in theoretical scheduling models. While there exists work for modelling dynamic Cloud task patterns for evaluating applied scheduling, such models are incompatible with the inputs needed for theoretical scheduling - which require such patterns to be represented as boxes. Presently there exist no methods capable of accurately converting real task patterns derived from empirical data into boxes. This results in a significant gap towards theoreticians understanding and proposing algorithms derived from realistic assumptions towards enhanced Cloud scheduling. This work proposes resource boxing - an approach for automated conversion of realistic task patterns in Cloud computing directly into box-inputs for theoretical scheduling. We propose four resource conversion algorithms capable of accurately representing real task utilization patterns in the form of scheduling boxes. Algorithms were evaluated using production Cloud trace data, demonstrating a difference between real utilization and scheduling boxes less than 5%. We also provide an application for how resource boxing can be exploited to directly translate research from the applied community into the theoretical community

A Blockchain-based Security-Oriented Framework for Cloud Federation

Cloud federations have been formed to share the services, prompt and support cooperation, as well as interoperability among their already deployed cloud systems. However, the creation and management of the cloud federations lead to various security issues such as confidentially, integrity and availability of the data. Despite the access control policies in place, an attacker may compromise the communication channel processing the access requests and the decisions between the access control systems and the members(users) and vice-versa. In cloud federation, the rating of the services offered by different cloud members becomes integral to providing the users with the best quality services. Hence, we propose an innovative blockchain- based framework that on the one hand permits secure communication between the members of the federation and the access control systems, while on the other hand provides the quality services to the members by considering the service constraints imposed by them

The Contemporary Affirmation of Taxonomy and Recent Literature on Workflow Scheduling and Management in Cloud Computing

The Cloud computing systemspreferred over the traditional forms of computing such as grid computing, utility computing, autonomic computing is attributed forits ease of access to computing, for its QoS preferences, SLA2019;s conformity, security and performance offered with minimal supervision. A cloud workflow schedule when designed efficiently achieves optimalre source sage, balance of workloads, deadline specific execution, cost control according to budget specifications, efficient consumption of energy etc. to meet the performance requirements of today2019; svast scientific and business requirements. The businesses requirements under recent technologies like pervasive computing are motivating the technology of cloud computing for further advancements. In this paper we discuss some of the important literature published on cloud workflow scheduling

Optimizing Cloud-Service Performance: Efficient Resource Provisioning Via Optimal Workload Allocation

Cloud computing is being widely accepted and utilized in the business world. From the perspective of businesses utilizing the cloud, it is critical to meet their customers\u27 requirements by achieving service-level-objectives. Hence, the ability to accurately characterize and optimize cloud-service performance is of great importance. In this dissertation, a stochastic multi-tenant framework is proposed to model the service of customer requests in a cloud infrastructure composed of heterogeneous virtual machines (VMs). The proposed framework addresses the critical concepts and characteristics in the cloud, including virtualization, multi-tenancy, heterogeneity of VMs, VM isolation for the purpose of security and/or performance guarantee and the stochastic response time of a customer request. Two cloud-service performance metrics are mathematically characterized, namely the percentile of the stochastic response time and the mean of the stochastic response time of a customer request. Based upon the proposed multi-tenant framework, a workload-allocation algorithm, termed max-min-cloud algorithm, is then devised to optimize the performance of the cloud service. A rigorous optimality proof of the max-min-cloud algorithm is given when the stochastic response time of a customer request assumed exponentially distributed. Furthermore, extensive Monte-Carlo simulations are conducted to validate the optimality of the max-min-cloud algorithm by comparing with other two workload-allocation algorithms under various scenarios. Next, the resource provisioning problem in the cloud is studied in light of the max-min-cloud algorithm. In particular, an efficient resource-provisioning strategy, termed the MPC strategy, is proposed for serving dynamically arriving customer requests. The efficacy of the MPC strategy is verified through two practical cases when the arrival of the customer requests is predictable and unpredictable, respectively. As an extension of the max-min-cloud algorithm, we further devise the max-load-first algorithm to deal with the VM placement problem in the cloud. MC simulation results show that the max-load-first VM-placement algorithm outperforms the other two heuristic algorithms in terms of reducing the mean of stochastic completion time of a group of arbitrary customers\u27 requests. Simulation results also provide insight on how the initial loads of servers affect the performance of the cloud system. In summary, the findings in this dissertation work can be of great benefit to both service providers (namely business owners) and cloud providers. For business owners, the max-min-cloud workload-allocation algorithm and the MPC resource-provisioning strategy together can be used help them build a better understanding of how much virtual resources in the cloud they may need to meet customers\u27 expectations subject to cost constraints. For cloud providers, the max-load-first VM-placement algorithm can be used to optimize the computational performance of the service by appropriately utilizing the physical machines and efficiently placing the VMs in their cloud infrastructures

Refining The Estimation Of The Available Bandwidth In Inter-cloud Links For Task Scheduling

In hybrid clouds, the available bandwidth in inter-cloud links is quite variable. Overestimating the available bandwidth on theses channels at scheduling time can enlarge the makespan and cause deadline misses. In this paper, we propose a procedure for deflating the estimated available bandwidth used as input to cloud schedulers since schedulers are not usually designed to cope with inaccurate information on available bandwidth. The procedure is based on a multiple linear regression procedure which utilizes historical information of previous executions of workflows. Results showed that the proposed procedure can increase the number of valid schedules without increasing the makespan and cost estimations, regardless the variability in the available bandwidth during the execution of an application workflow.11271132Zhang, Q., Cheng, L., Boutaba, R., Cloud computing: State-of-the-art and research challenges (2010) Journal of Internet Services and Applications, 1 (1), pp. 7-18Bittencourt, L.F., Madeira, E.R.M., Da Fonseca, N.L.S., Scheduling in hybrid clouds (2012) IEEE Communications Magazine, 50 (9), pp. 42-47Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M., Workflows for e-Science (2007) Scientific Workflows for Grids, , SpringerBittencourt, L.F., Madeira, E.R.M., Da Fonseca, N.L.S., Impact of communication uncertainties on workflow scheduling in hybrid clouds (2012) IEEE Global Communications Conference (IEEE GLOBECOM), , Anaheim, USA, decemberBatista, D.M., Chaves, L.J., Fonseca, N.L., Ziviani, A., Performance analysis of available bandwidth estimation tools for grid networks (2010) Journal of Supercomputing, 53 (1), pp. 103-121. , JulRahman, M., Li, X., Palit, H., Hybrid heuristic for scheduling data analytics workflow applications in hybrid cloud environment (2011) IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum (IPDPSW), pp. 966-974Bittencourt, L.F., Madeira, E.R.M., HCOC: A cost optimization algorithm for workflow scheduling in hybrid clouds (2011) Journal of Internet Services and Applications, 2 (3), pp. 207-227. , DecVecchiola, C., Calheiros, R.N., Karunamoorthy, D., Buyya, R., Deadline-driven provisioning of resources for scientific applications in hybrid clouds with aneka (2012) Future Gener. Comput. Syst., 28 (1), pp. 58-65. , janAllen, G., Angulo, D., Foster, I., Lanfermann, G., Liu, C., Radke, T., Seidel, E., Shalf, J., The cactus worm: Experiments with dynamic resource discovery and allocation in a grid environment (2001) Journal of High Performance Computing Applications, 15, p. 2001Sakellariou, R., Zhao, H., A low-cost rescheduling policy for efficient mapping of workflows on grid systems (2004) Scientific Programming, 12 (4), pp. 253-262. , DecBatista, D.M., Da Fonseca, N.L.S., Miyazawa, F.K., Granelli, F., Self-adjustment of resource allocation for grid applications (2008) Computer Networks, 52 (9), pp. 1762-1781. , JunBatista, D.M., Da Fonseca, N.L.S., Robust scheduler for grid networks under uncertainties of both application demands and resource availability (2011) Computer Networks, 55 (1), pp. 3-19Wang, G., Ng, T.S.E., The impact of virtualization on network performance of amazon EC2 data center (2010) 29th Conference on Information Communications, pp. 1163-1171Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Katz, D.S., Pegasus: A framework for mapping complex scientific workflows onto distributed systems (2005) Scientific Programming Journal, 13 (3), pp. 219-237Ramakrishnan, A., Singh, G., Zhao, H., Deelman, E., Sakellariou, R., Vahi, K., Blackburn, K., Samidi, M., Scheduling dataintensive workflows onto storage-constrained distributed resources (2007) IEEE International Symposium on Cluster Computing and the Grid, pp. 401-409Sanghrajka, S., Mahajan, N., Sion, R., Cloud performance benchmark series-network performance: Amazon EC2 (2011) Stony Brook University, Tech. Rep.Sanghrajka, S., Sion, R., Cloud performance benchmark series-network performance: Rackspace com (2011) Stony Brook University, Tech. Rep.Casanova, H., Legrand, A., Zagorodnov, D., Berman, F., Heuristics for scheduling parameter sweep applications in grid environments (2000) Heterogeneous Computing Workshop, pp. 349-36