A Framework for Approximate Optimization of BoT Application Deployment in Hybrid Cloud Environment

We adopt a systematic approach to investigate the efficiency of near-optimal deployment of large-scale CPU-intensive Bag-of-Task applications running on cloud resources with the non-proportional cost to performance ratios. Our analytical solutions perform in both known and unknown running time of the given application. It tries to optimize users' utility by choosing the most desirable tradeoff between the make-span and the total incurred expense. We propose a schema to provide a near-optimal deployment of BoT application regarding users' preferences. Our approach is to provide user with a set of Pareto-optimal solutions, and then she may select one of the possible scheduling points based on her internal utility function. Our framework can cope with uncertainty in the tasks' execution time using two methods, too. First, an estimation method based on a Monte Carlo sampling called AA algorithm is presented. It uses the minimum possible number of sampling to predict the average task running time. Second, assuming that we have access to some code analyzer, code profiling or estimation tools, a hybrid method to evaluate the accuracy of each estimation tool in certain interval times for improving resource allocation decision has been presented. We propose approximate deployment strategies that run on hybrid cloud. In essence, proposed strategies first determine either an estimated or an exact optimal schema based on the information provided from users' side and environmental parameters. Then, we exploit dynamic methods to assign tasks to resources to reach an optimal schema as close as possible by using two methods. A fast yet simple method based on First Fit Decreasing algorithm, and a more complex approach based on the approximation solution of the transformed problem into a subset sum problem. Extensive experiment results conducted on a hybrid cloud platform confirm that our framework can deliver a near optimal solution respecting user's utility function

A queueing theory approach to Pareto-optimal bags-of-tasks scheduling on clouds

Cloud hosting services offer computing resources which can scale along with the needs of users. When access to data is limited by the network capacity this scalability also becomes limited. To investigate the impact of this limitation we focus on bags{of{tasks where task data is stored outside the cloud and has to be transferred across the network before task execution can commence. The existing bags-of-tasks estimation tools are not able to provide accurate estimates in such a case. We introduce a queuing{network inspired model which successfully models the limited network resources. Based on the Mean{Value Analysis of this model we derive an efficient procedure that results with an estimate of the makespan and the executions costs for a given configuration of cloud virtual machines. We compare the calculated Pareto set with measurements performed in a number of experiments for real-world bags-of-tasks and validate the proposed model and the accuracy of the estimated configurations

Scientific Workflow Scheduling for Cloud Computing Environments

The scheduling of workflow applications consists of assigning their tasks to computer resources to fulfill a final goal such as minimizing total workflow execution time. For this reason, workflow scheduling plays a crucial role in efficiently running experiments. Workflows often have many discrete tasks and the number of different task distributions possible and consequent time required to evaluate each configuration quickly becomes prohibitively large. A proper solution to the scheduling problem requires the analysis of tasks and resources, production of an accurate environment model and, most importantly, the adaptation of optimization techniques. This study is a major step toward solving the scheduling problem by not only addressing these issues but also optimizing the runtime and reducing monetary cost, two of the most important variables. This study proposes three scheduling algorithms capable of answering key issues to solve the scheduling problem. Firstly, it unveils BaRRS, a scheduling solution that exploits parallelism and optimizes runtime and monetary cost. Secondly, it proposes GA-ETI, a scheduler capable of returning the number of resources that a given workflow requires for execution. Finally, it describes PSO-DS, a scheduler based on particle swarm optimization to efficiently schedule large workflows. To test the algorithms, five well-known benchmarks are selected that represent different scientific applications. The experiments found the novel algorithms solutions substantially improve efficiency, reducing makespan by 11% to 78%. The proposed frameworks open a path for building a complete system that encompasses the capabilities of a workflow manager, scheduler, and a cloud resource broker in order to offer scientists a single tool to run computationally intensive applications

Evolutionary algorithm-based multi-objective task scheduling optimization model in cloud environments

© 2015, Springer Science+Business Media New York. Optimizing task scheduling in a distributed heterogeneous computing environment, which is a nonlinear multi-objective NP-hard problem, plays a critical role in decreasing service response time and cost, and boosting Quality of Service (QoS). This paper, considers four conflicting objectives, namely minimizing task transfer time, task execution cost, power consumption, and task queue length, to develop a comprehensive multi-objective optimization model for task scheduling. This model reduces costs from both the customer and provider perspectives by considering execution and power cost. We evaluate our model by applying two multi-objective evolutionary algorithms, namely Multi-Objective Particle Swarm Optimization (MOPSO) and Multi-Objective Genetic Algorithm (MOGA). To implement the proposed model, we extend the Cloudsim toolkit by using MOPSO and MOGA as its task scheduling algorithms which determine the optimal task arrangement among VMs. The simulation results show that the proposed multi-objective model finds optimal trade-off solutions amongst the four conflicting objectives, which significantly reduces the job response time and makespan. This model not only increases QoS but also decreases the cost to providers. From our experimentation results, we find that MOPSO is a faster and more accurate evolutionary algorithm than MOGA for solving such problems

Improved Hybrid Algorithm Approach based Load Balancing Technique in Cloud Computing

The routine life of modern citizens is completely dominated by the computer aided services The computer aided services depends on information and communication technologies The success behind this cloud computing are data centers with virtualization technology equipped with fastest internet and the wide acceptance of the users due to its affordable price to the common people Effective services can be provided to the end user only when proper scheduling of tasks are done in peak hours when heterogeneous collection of requests are coming to the data cente

A Multimedia Cloud Computing Model for Combinatorial Virtual Machine Placement

Cloud computing, which allows users to access subscription-based services on a pay-as-you-go basis, has recently transformed IT departments. Today, a variety of media services are offered through the Internet owing to the development of multimedia cloud computing, which is based on cloud computing. However, as multimedia cloud computing spreads, it has a negative influence on greenhouse gas emissions due to its high energy consumption and raises expenses for cloud users. Therefore, while still providing consumers with the resources they require and maintaining a high level of service, multimedia cloud service providers should make every effort to consume as little energy as possible. This proposal proposes residual usage-aware (RUA) and performance-aware (PA) methods for virtual machine placement. To save energy, find a suitable host to switch off. These two techniques were merged and applied to cloud data centers in order to complete the VM consolidation process. The outcomes of the simulation demonstrate a trade-off between energy consumption and SLA violations. Additionally, during VM deployment, it can manage shifting workloads to prevent host overload, dramatically lowering SLA breaches

A survey and taxonomy of resource optimisation for executing Bag-of-Task applications on public clouds

Cloud computing has been widely adopted due to the flexibility in resource provisioning and on-demand pricing models. Entire clusters of Virtual Machines (VMs) can be dynamically provisioned to meet the computational demands of users. However, from a user’s perspective, it is still challenging to utilise cloud resources efficiently. This is because an overwhelmingly wide variety of resource types with different prices and significant performance variations are available. This paper presents a survey and taxonomy of existing research in optimising the execution of Bag-of-Task applications on cloud resources. A BoT application consists of multiple independent tasks, each of which can be executed by a VM in any order; these applications are widely used by both the scientific communities and commercial organisations. The objectives of this survey are as follows: (i) to provide the reader with a concise understanding of existing research on optimising the execution of BoT applications on the cloud, (ii) to define a taxonomy that categorises current frameworks to compare and contrast them, and (iii) to present current trends and future research directions in the area.PostprintPeer reviewe

Resource Management and Scheduling for Big Data Applications in Cloud Computing Environments

This chapter presents software architectures of the big data processing platforms. It will provide an in-depth knowledge on resource management techniques involved while deploying big data processing systems on cloud environment. It starts from the very basics and gradually introduce the core components of resource management which we have divided in multiple layers. It covers the state-of-art practices and researches done in SLA-based resource management with a specific focus on the job scheduling mechanisms.Comment: 27 pages, 9 figure