Enhancement of Ant Colony Optimization for Grid Job Scheduling and Load Balancing

Managing resources in grid computing system is complicated due to the distributed and heterogeneous nature of the resources. Stagnation in grid computing system may occur when all jobs are required or are assigned to the same resources which lead to the resources having high workload or the time taken to process a job is high. This research proposes an Enhanced Ant Colony Optimization (EACO) algorithm that caters dynamic scheduling and load balancing in the grid computing system. The proposed algorithm can overcome stagnation problem, minimize processing time, match jobs with suitable resources, and balance entire resources in grid environment. This research follows the experimental research methodology that consists of problem analysis, developing the proposed framework, constructing the simulation environment, conducting a set of experiments and evaluating the results. There are three new mechanisms in this proposed framework that are used to organize the work of an ant colony i.e. initial pheromone value mechanism, resource selection mechanism and pheromone update mechanism. The resource allocation problem is modeled as a graph that can be used by the ant to deliver its pheromone. This graph consists of four types of vertices which are job, requirement, resource and capacity that are used in constructing the grid job scheduling. The proposed EACO algorithm takes into consideration the capacity of resources and the characteristics of jobs in determining the best resource to process a job. EACO selects the resources based on the pheromone value on each resource which is recorded in a matrix form. The initial pheromone value of each resource for each job is calculated based on the estimated transmission time and execution time of a given job. Resources with high pheromone value are selected to process the submitted jobs. Global pheromone update is performed after the completion of processing the jobs in order to reduce the pheromone value of resources. A simulation environment was developed using Java programming to test the performance of the proposed EACO algorithm against existing grid resource management algorithms such as Antz algorithm, Particle Swarm Optimization algorithm, Space Shared algorithm and Time Shared algorithm, in terms of processing time and resource utilization. Experimental results show that EACO produced better grid resource management solution compared to other algorithms

A Multiconstrained Grid Scheduling Algorithm with Load Balancing and Fault Tolerance

Grid environment consists of millions of dynamic and heterogeneous resources. A grid environment which deals with computing resources is computational grid and is meant for applications that involve larger computations. A scheduling algorithm is said to be efficient if and only if it performs better resource allocation even in case of resource failure. Allocation of resources is a tedious issue since it has to consider several requirements such as system load, processing cost and time, user’s deadline, and resource failure. This work attempts to design a resource allocation algorithm which is budget constrained and also targets load balancing, fault tolerance, and user satisfaction by considering the above requirements. The proposed Multiconstrained Load Balancing Fault Tolerant algorithm (MLFT) reduces the schedule makespan, schedule cost, and task failure rate and improves resource utilization. The proposed MLFT algorithm is evaluated using Gridsim toolkit and the results are compared with the recent algorithms which separately concentrate on all these factors. The comparison results ensure that the proposed algorithm works better than its counterparts

Ant colony optimization algorithm for load balancing in grid computing

Managing resources in grid computing system is complicated due to the distributed and heterogeneous nature of the resources. This research proposes an enhancement of the ant colony optimization algorithm that caters for dynamic scheduling and load balancing in the grid computing system. The proposed algorithm is known as the enhance ant colony optimization (EACO). The algorithm consists of three new mechanisms that organize the work of an ant colony i.e. initial pheromone value mechanism, resource selection mechanism and pheromone update mechanism. The resource allocation problem is modelled as a graph that can be used by the ant to deliver its pheromone.This graph consists of four types of vertices which are job, requirement, resource and capacity that are used in constructing the grid resource management element. The proposed EACO algorithm takes into consideration the capacity of resources and the characteristics of jobs in determining the best resource to process a job. EACO selects the resources based on the pheromone value on each resource which is recorded in a matrix form. The initial pheromone value of each resource for each job is calculated based on the estimated transmission time and execution time of a given job.Resources with high pheromone value are selected to process the submitted jobs. Global pheromone update is performed after the completion of processing the jobs in order to reduce the pheromone value of resources.A simulation environment was developed using Java programming to test the performance of the proposed EACO algorithm against other ant based algorithm, in terms of resource utilization. Experimental results show that EACO produced better grid resource management solution

An enhanced ant colony system algorithm for dynamic fault tolerance in grid computing

Fault tolerance in grid computing allows the system to continue operate despite occurrence of failure. Most fault tolerance algorithms focus on fault handling techniques such as task reprocessing, checkpointing, task replication, penalty, and task migration. Ant colony system (ACS), a variant of ant colony optimization (ACO), is one of the promising algorithms for fault tolerance due to its ability to adapt to both static and dynamic combinatorial optimization problems. However, ACS algorithm does not consider the resource fitness during task scheduling which leads to poor load balancing and lower execution success rate. This research proposes dynamic ACS fault tolerance with suspension (DAFTS) in grid computing that focuses on providing effective fault tolerance techniques to improve the execution success rate and load balancing. The proposed algorithm consists of dynamic evaporation rate, resource fitness-based scheduling process, enhanced pheromone update with trust factor and suspension, and checkpoint-based task reprocessing. The research framework consists of four phases which are identifying fault tolerance techniques, enhancing resource assignment and job scheduling, improving fault tolerance algorithm and, evaluating the performance of the proposed algorithm. The proposed algorithm was developed in a simulated grid environment called GridSim and evaluated against other fault tolerance algorithms such as trust-based ACO, fault tolerance ACO, ACO without fault tolerance and ACO with fault tolerance in terms of total execution time, average latency, average makespan, throughput, execution success rate and load balancing. Experimental results showed that the proposed algorithm achieved the best performance in most aspects, and second best in terms of load balancing. The DAFTS achieved the smallest increase on execution time, average makespan and average latency by 7%, 11% and 5% respectively, and smallest decrease on throughput and execution success rate by 6.49% and 9% respectively as the failure rate increases. The DAFTS also achieved the smallest increment on execution time, average makespan and average latency by 5.8, 8.5 and 8.7 times respectively, and highest increase on throughput and highest execution success rate by 72.9% and 93.7% respectively as the number of jobs increases. The proposed algorithm can effectively overcome load balancing problems and increase execution success rates in distributed systems that are prone to faults

Smart Distributed Processing Technologies For Hedge Fund Management

Distributed processing cluster design using commodity hardware and software has proven to be a technological breakthrough in the field of parallel and distributed computing. The research presented herein is the original investigation on distributed processing using hybrid processing clusters to improve the calculation efficiency of the compute-intensive applications. This has opened a new frontier in affordable supercomputing that can be utilised by businesses and industries at various levels. Distributed processing that uses commodity computer clusters has become extremely popular over recent years, particularly among university research groups and research organisations. The research work discussed herein addresses a bespoke-oriented design and implementation of highly specific and different types of distributed processing clusters with applied load balancing techniques that are well suited for particular business requirements. The research was performed in four phases, which are cohesively interconnected, to find a suitable solution using a new type of distributed processing approaches. The first phase is an implementation of a bespoke-type distributed processing cluster using an existing network of workstations as a calculation cluster based on a loosely coupled distributed process system design that has improved calculation efficiency of certain legacy applications. This approach has demonstrated how to design an innovative, cost-effective, and efficient way to utilise a workstation cluster for distributed processing. The second phase is to improve the calculation efficiency of the distributed processing system; a new type of load balancing system is designed to incorporate multiple processing devices. The load balancing system incorporates hardware, software and application related parameters to assigned calculation tasks to each processing devices accordingly. Three types of load balancing methods are tested, static, dynamic and hybrid, which each of them has their own advantages, and all three of them have further improved the calculation efficiency of the distributed processing system.   The third phase is to facilitate the company to improve the batch processing application calculation time, and two separate dedicated calculation clusters are built using small form factor (SFF) computers and PCs as separate peer-to-peer (P2P) network based calculation clusters. Multiple batch processing applications were tested on theses clusters, and the results have shown consistent calculation time improvement across all the applications tested. In addition, dedicated clusters are built using SFF computers with reduced power consumption, small cluster size, and comparatively low cost to suit particular business needs. The fourth phase incorporates all the processing devices available in the company as a hybrid calculation cluster utilises various type of servers, workstations, and SFF computers to form a high-throughput distributed processing system that consolidates multiple calculations clusters. These clusters can be utilised as multiple mutually exclusive multiple clusters or combined as a single cluster depending on the applications used. The test results show considerable calculation time improvements by using consolidated calculation cluster in conjunction with rule-based load balancing techniques. The main design concept of the system is based on the original design that uses first principle methods and utilises existing LAN and separate P2P network infrastructures, hardware, and software. Tests and investigations conducted show promising results where the company’s legacy applications can be modified and implemented with different types of distributed processing clusters to achieve calculation and processing efficiency for various applications within the company. The test results have confirmed the expected calculation time improvements in controlled environments and show that it is feasible to design and develop a bespoke-type dedicated distributed processing cluster using existing hardware, software, and low-cost SFF computers. Furthermore, a combination of bespoke distributed processing system with appropriate load balancing algorithms has shown considerable calculation time improvements for various legacy and bespoke applications. Hence, the bespoke design is better suited to provide a solution for the calculation of time improvements for critical problems currently faced by the sponsoring company