Cloud-aware data intensive workflow scheduling on volunteer computing systems

Volunteer computing systems offer high computing power to the scientific communities to run large data intensive scientific workflows. However, these computing environments provide the best effort infrastructure to execute high performance jobs. This work aims to schedule scientific and data intensive workflows on hybrid of the volunteer computing system and Cloud resources to enhance the utilization of these environments and increase the percentage of workflow that meets the deadline. The proposed workflow scheduling system partitions a workflow into sub-workflows to minimize data dependencies among the sub-workflows. Then these sub-workflows are scheduled to distribute on volunteer resources according to the proximity of resources and the load balancing policy. The execution time of each sub-workflow on the selected volunteer resources is estimated in this phase. If any of the sub-workflows misses the sub-deadline due to the large waiting time, we consider re-scheduling of this sub-workflow into the public Cloud resources. This re-scheduling improves the system performance by increasing the percentage of workflows that meet the deadline. The proposed Cloud-aware data intensive scheduling algorithm increases the percentage of workflow that meet the deadline with a factor of 75% in average with respect to the execution of workflows on the volunteer resources

Deadline-constrained workflow scheduling in volunteer computing systems

One of the main challenges in volunteer computing systems is scheduling large-scale applications expressed as scientific workflows. This work aims to integrate partitioning scientific workflows and proximity-aware resource provisioning to increase the percentage of workflows that meet the deadline in peer-to-peer based volunteer computing systems. In the partitioning phase, a scientific workflow is partitioned into sub-workflows in order to minimize data dependencies among them. We utilize knowledge-free load balancing policy and proximity of resources to distribute sub-workflows on volunteer resources. Simulation results show that the proposed workflow scheduling system improves the percentage of scientific workflows that meet the deadline with average of 18% under a moderate workload

Decentralised workflow scheduling in volunteer computing systems

Volunteer computing systems exploiting large amounts of geographically dispersed resources on the Internet for solving complex scientific problems. However, scheduling scientific workflows in a fully decentralised way and low overhead is a challenging task in these environments. To counter this challenge, this paper presents a fully decentralised proximity-aware workflow-scheduling policy for these environments. The proposed scheduling consists of three phases. In the first phase, each workflow application is partitioned into sub-workflows in order to minimise data dependencies among them. The second phase of the workflow-scheduling algorithm finds some resources to execute each sub-workflow. These resources are selected based on Quality of Service (QoS) constraints of the workflow, load balancing and proximity of resources. Each workflow can have QoS constraints in terms of minimum CPU speed and minimum RAM or hard disk requirements. In the third phase, sub-workflows will be executed on each resource based on local scheduling algorithm to minimise the partial makespan. The proposed scheduling policy focuses on the reduction of communication overhead to improve the performance of I/O-intensive and data-intensive workflows. Simulation results show that the proposed workflow-scheduling policy improves the average response time of scientific workflows up to 53.6% under a moderate load

A Proximity-aware Load Balancing in Peer-to-Peer based Volunteer Computing Systems

One of the main challenges in peer-to-peer based volunteer computing systems is efficient resource discovery algorithm. Load balancing is a part of resource discovery algorithm and aims to minimize the overall response time of the system. This paper introduces an analytical model based on distributed parallel queues to optimize the average response time of the system in a distributed manner. The proposed resource discovery algorithm consists of two phases. In the first phase, it selects peers in a load-balanced manner based on QoS constraints of request. In the second phase, a proximity-aware feature is applied to select the peer with minimum communication overhead among selected peers in the first phase. Two dispatching strategies are proposed for the load balancing based on stochastic analysis of routing in the distributed parallel queues. These policies adopt probabilistic and deterministic sequences to redirect requests to the capable peers in the system. Simulation results show that the proposed resource discovery algorithm improves the response time of user’s requests by a factor of 1.8 under a moderate load

Application placement in fog computing with AI approach : taxonomy and a state of the art survey

With the increasing use of the Internet of Things (IoT) in various fields and the need to process and store huge volumes of generated data, Fog computing was introduced to complement Cloud computing services. Fog computing offers basic services at the network for supporting IoT applications with low response time requirements. However, Fogs are distributed, heterogeneous, and their resources are limited, therefore efficient distribution of IoT applications tasks in Fog nodes, in order to meet quality of service (QoS) and quality of experience (QoE) constraints is challenging. In this survey, at first, we have an overview of basic concepts of Fog computing, and then review the application placement problem in Fog computing with focus on Artificial intelligence (AI) techniques. We target three main objectives with considering a characteristics of AI-based methods in Fog application placement problem: (i) categorizing evolutionary algorithms, (ii) categorizing machine learning algorithms, and (iii) categorizing combinatorial algorithms into subcategories includes a combination of machine learning and heuristic, a combination of evolutionary and heuristic, and a combinations of evolutionary and machine learning. Then the security considerations of application placement have been reviewed. Finally, we provide a number of open questions and issues as future works

CycloidGrid : a proximity-aware P2P-based resource discovery architecture in volunteer computing systems

Volunteer computing which benefits from idle cycles of volunteer resources over the Internet can integrate the power of hundreds to thousands of resources to achieve high computing power. In such an environment the resources are heterogeneous in terms of CPU speed, RAM, disk capacity, and network bandwidth. So finding a suitable resource to run a particular job becomes difficult. Resource discovery architecture is a key factor for overall performance of peer-to-peer based volunteer computing systems. The main contribution of this paper is to develop a proximity-aware resource discovery architecture for peer-to-peer based volunteer computing systems. The proposed resource discovery algorithm consists of two stages. In the first stage, it selects resources based on the requested quality of service and current load of peers. In the second stage, a resource with higher priority to communication delay is selected among the discovered resources. Communication delay between two peers is computed by a network model based on queuing theory, taking into account the background traffic of the Internet. Simulation results show that the proposed resource discovery algorithm improves the response time of user's requests by a factor of 4.04 under a moderate load