Parallel Differential Evolution approach for Cloud workflow placements under simultaneous optimization of multiple objectives

International audienceThe recent rapid expansion of Cloud computing facilities triggers an attendant challenge to facility providers and users for methods for optimal placement of workflows on distributed resources, under the often-contradictory impulses of minimizing makespan, energy consumption, and other metrics. Evolutionary Optimization techniques that from theoretical principles are guaranteed to provide globally optimum solutions, are among the most powerful tools to achieve such optimal placements. Multi-Objective Evolutionary algorithms by design work upon contradictory objectives, gradually evolving across generations towards a converged Pareto front representing optimal decision variables – in this case the mapping of tasks to resources on clusters. However the computation time taken by such algorithms for convergence makes them prohibitive for real time placements because of the adverse impact on makespan. This work describes parallelization, on the same cluster, of a Multi-Objective Differential Evolution method (NSDE-2) for optimization of workflow placement, and the attendant speedups that bring the implicit accuracy of the method into the realm of practical utility. Experimental validation is performed on a real-life testbed using diverse Cloud traces. The solutions under different scheduling policies demonstrate significant reduction in energy consumption with some improvement in makespan

A Survey on Scheduling the Task in Fog Computing Environment

With the rapid increase in the Internet of Things (IoT), the amount of data produced and processed is also increased. Cloud Computing facilitates the storage, processing, and analysis of data as needed. However, cloud computing devices are located far away from the IoT devices. Fog computing has emerged as a small cloud computing paradigm that is near to the edge devices and handles the task very efficiently. Fog nodes have a small storage capability than the cloud node but it is designed and deployed near to the edge device so that request must be accessed efficiently and executes in time. In this survey paper we have investigated and analysed the main challenges and issues raised in scheduling the task in fog computing environment. To the best of our knowledge there is no comprehensive survey paper on challenges in task scheduling of fog computing paradigm. In this survey paper research is conducted from 2018 to 2021 and most of the paper selection is done from 2020-2021. Moreover, this survey paper organizes the task scheduling approaches and technically plans the identified challenges and issues. Based on the identified issues, we have highlighted the future work directions in the field of task scheduling in fog computing environment

Scientific Workflow Scheduling for Cloud Computing Environments

The scheduling of workflow applications consists of assigning their tasks to computer resources to fulfill a final goal such as minimizing total workflow execution time. For this reason, workflow scheduling plays a crucial role in efficiently running experiments. Workflows often have many discrete tasks and the number of different task distributions possible and consequent time required to evaluate each configuration quickly becomes prohibitively large. A proper solution to the scheduling problem requires the analysis of tasks and resources, production of an accurate environment model and, most importantly, the adaptation of optimization techniques. This study is a major step toward solving the scheduling problem by not only addressing these issues but also optimizing the runtime and reducing monetary cost, two of the most important variables. This study proposes three scheduling algorithms capable of answering key issues to solve the scheduling problem. Firstly, it unveils BaRRS, a scheduling solution that exploits parallelism and optimizes runtime and monetary cost. Secondly, it proposes GA-ETI, a scheduler capable of returning the number of resources that a given workflow requires for execution. Finally, it describes PSO-DS, a scheduler based on particle swarm optimization to efficiently schedule large workflows. To test the algorithms, five well-known benchmarks are selected that represent different scientific applications. The experiments found the novel algorithms solutions substantially improve efficiency, reducing makespan by 11% to 78%. The proposed frameworks open a path for building a complete system that encompasses the capabilities of a workflow manager, scheduler, and a cloud resource broker in order to offer scientists a single tool to run computationally intensive applications

Collaborative gold mining algorithm : an optimization algorithm based on the natural gold mining process

In optimization algorithms, there are some challenges, including lack of optimal solution, slow convergence, lack of scalability, partial search space, and high computational demand. Inspired by the process of gold exploration and exploitation, we propose a new meta-heuristic and stochastic optimization algorithm called collaborative gold mining (CGM). The proposed algorithm has several iterations; in each of these, the center of mass of points with the highest amount of gold is calculated for each miner (agent), with this process continuing until the point with the highest amount of gold or when the optimal solution is found. In an n-dimensional geographic space, the CGM algorithm can locate the best position with the highest amount of gold in the entire search space by collaborating with several gold miners. The proposed CGM algorithm was applied to solve several continuous mathematical functions and several practical problems, namely, the optimal placement of resources, the traveling salesman problem, and bag-of-tasks scheduling. In order to evaluate its efficiency, the CGM results were compared with the outputs of some famous optimization algorithms, such as the genetic algorithm, simulated annealing, particle swarm optimization, and invasive weed optimization. In addition to determining the optimal solutions for all the evaluated problems, the experimental results show that the CGM mechanism has an acceptable performance in terms of optimal solution, convergence, scalability, search space, and computational demand for solving continuous and discrete problems

Optimal Task Scheduling in the Cloud Environment using a Mean Grey Wolf Optimization Algorithm

Cloud computing is one of the emerging areas in computing platforms, supporting heterogeneous, parallel and distributed environments. An important challenging issue in cloud computing is task scheduling, which directly influences system performance and its efficiency. The primary objective of task scheduling involves scheduling tasks related to resources and minimizing the time span of the schedule. In this study, we propose a Modified Mean Grey Wolf Optimization (MGWO) algorithm to enhance system performance, and consequently reduce scheduling issues. The main objective of this method is focused upon minimizing the makespan (execution time) and energy consumption.  These two objective functions are elaborated in the algorithm in order to suitably regulate the quality of results based on response, in order to achieve a near optimal solution. The implementation results of the proposed algorithm are evaluated using the CloudSim toolkit for standard workloads (normal and uniform). The advantage of the proposed method is evident from the simulation results, which show a comprehensive reduction in makespan and energy consumption. The outcomes of these results show that the proposed Mean GWO algorithm achieves a 8.85% makespan improvement compared to the PSO algorithm, and 3.09% compared to the standard GWO algorithm for the normal dataset. In addition, the proposed algorithm achieves 9.05% and 9.2% improvement in energy conservation compared to the PSO and standard GWO algorithms for the uniform dataset, respectively

A dynamic task scheduler tolerant to multiple hibernations in cloud environments

International audienceCloud platforms usually offer several types of Virtual Machines (VMs) with different guarantees in terms of availability and volatility, provisioning the same resource through multiple pricing models. For instance, in the Amazon EC2 cloud, the user pays per use for on-demand VMs while spot VMs are instances available at lower prices. However, a spot VM can be terminated or hibernated by EC2 at any moment. In this work, we propose the Hibernation-Aware Dynamic Scheduler (HADS) that schedules Bag-of-Tasks (BoT) applications with deadline constraints in both hibernation prone spots VMs and on-demand VMs. HADS aims at minimizing the monetary costs of executing BoT applications on Clouds ensuring that their deadlines are respected even in the presence of multiple hibernations. Results collected from experiments on Amazon EC2 VMs using synthetic applications and a NAS benchmark application show the effectiveness of HADS in terms of monetary costs when compared to on-demand VM only solutions

Intelligent Business Process Optimization for the Service Industry

The company\u27s sustainable competitive advantage derives from its capacity to create value for customers and to adapt the operational practices to changing situations. Business processes are the heart of each company. Therefore process excellence has become a key issue. This book introduces a novel approach focusing on the autonomous optimization of business processes by applying sophisticated machine learning techniques such as Relational Reinforcement Learning and Particle Swarm Optimization

Data Placement And Task Mapping Optimization For Big Data Workflows In The Cloud

Data-centric workflows naturally process and analyze a huge volume of datasets. In this new era of Big Data there is a growing need to enable data-centric workflows to perform computations at a scale far exceeding a single workstation\u27s capabilities. Therefore, this type of applications can benefit from distributed high performance computing (HPC) infrastructures like cluster, grid or cloud computing. Although data-centric workflows have been applied extensively to structure complex scientific data analysis processes, they fail to address the big data challenges as well as leverage the capability of dynamic resource provisioning in the Cloud. The concept of “big data workflows” is proposed by our research group as the next generation of data-centric workflow technologies to address the limitations of exist-ing workflows technologies in addressing big data challenges. Executing big data workflows in the Cloud is a challenging problem as work-flow tasks and data are required to be partitioned, distributed and assigned to the cloud execution sites (multiple virtual machines). In running such big data work-flows in the cloud distributed across several physical locations, the workflow execution time and the cloud resource utilization efficiency highly depends on the initial placement and distribution of the workflow tasks and datasets across the multiple virtual machines in the Cloud. Several workflow management systems have been developed for scientists to facilitate the use of workflows; however, data and work-flow task placement issue has not been sufficiently addressed yet. In this dissertation, I propose BDAP strategy (Big Data Placement strategy) for data placement and TPS (Task Placement Strategy) for task placement, which improve workflow performance by minimizing data movement across multiple virtual machines in the Cloud during the workflow execution. In addition, I propose CATS (Cultural Algorithm Task Scheduling) for workflow scheduling, which improve workflow performance by minimizing workflow execution cost. In this dissertation, I 1) formalize data and task placement problems in workflows, 2) propose a data placement algorithm that considers both initial input dataset and intermediate datasets obtained during workflow run, 3) propose a task placement algorithm that considers placement of workflow tasks before workflow run, 4) propose a workflow scheduling strategy to minimize the workflow execution cost once the deadline is provided by user and 5)perform extensive experiments in the distributed environment to validate that our proposed strategies provide an effective data and task placement solution to distribute and place big datasets and tasks into the appropriate virtual machines in the Cloud within reasonable time

Advances in Grid Computing

This book approaches the grid computing with a perspective on the latest achievements in the field, providing an insight into the current research trends and advances, and presenting a large range of innovative research papers. The topics covered in this book include resource and data management, grid architectures and development, and grid-enabled applications. New ideas employing heuristic methods from swarm intelligence or genetic algorithm and quantum encryption are considered in order to explain two main aspects of grid computing: resource management and data management. The book addresses also some aspects of grid computing that regard architecture and development, and includes a diverse range of applications for grid computing, including possible human grid computing system, simulation of the fusion reaction, ubiquitous healthcare service provisioning and complex water systems