Integrating multiple clusters for compute-intensive applications

Multicluster grids provide one promising solution to satisfying the growing computational demands of compute-intensive applications. However, it is challenging to seamlessly integrate all participating clusters in different domains into a single virtual computational platform. In order to fully utilize the capabilities of multicluster grids, computer scientists need to deal with the issue of joining together participating autonomic systems practically and efficiently to execute grid-enabled applications. Driven by several compute-intensive applications, this theses develops a multicluster grid management toolkit called Pelecanus to bridge the gap between user\u27s needs and the system\u27s heterogeneity. Application scientists will be able to conduct very large-scale execution across multiclusters with transparent QoS assurance. A novel model called DA-TC (Dynamic Assignment with Task Containers) is developed and is integrated into Pelecanus. This model uses the concept of a task container that allows one to decouple resource allocation from resource binding. It employs static load balancing for task container distribution and dynamic load balancing for task assignment. The slowest resources become useful rather than be bottlenecks in this manner. A cluster abstraction is implemented, which not only provides various cluster information for the DA-TC execution model, but also can be used as a standalone toolkit to monitor and evaluate the clusters\u27 functionality and performance. The performance of the proposed DA-TC model is evaluated both theoretically and experimentally. Results demonstrate the importance of reducing queuing time in decreasing the total turnaround time for an application. Experiments were conducted to understand the performance of various aspects of the DA-TC model. Experiments showed that our model could significantly reduce turnaround time and increase resource utilization for our targeted application scenarios. Four applications are implemented as case studies to determine the applicability of the DA-TC model. In each case the turnaround time is greatly reduced, which demonstrates that the DA-TC model is efficient for assisting application scientists in conducting their research. In addition, virtual resources were integrated into the DA-TC model for application execution. Experiments show that the execution model proposed in this thesis can work seamlessly with multiple hybrid grid/cloud resources to achieve reduced turnaround time

Efficient Methods for Scheduling Jobs in a Simulation Model Using a Multicore Multicluster Architecture

Over the past decade, the fast advance of network technologies, hardware and middleware, as well as software resource sophistication has contributed to the emergence of new computational models. Consequently, there was a capacity increasing for efficient and effective use of resources distributed aiming to integrate them, in order to provide a widely distributed environment, which computational capacity could be used to solve complex computer problems. The two most challenging aspects of distributed systems are resource management and task scheduling. This work contributes to minimize such problems by i) aiming to reduce this problem through the use of migration techniques; ii) implementing a multicluster simulation environment with mechanisms for load balancing; iii) plus, the gang scheduling implementation algorithms will be analyzed through the use of metrics, in order to measure the schedulers performance in different situations. Thus, the results showed a better use of resources, implying operating costs reduction

Arquitectura y Algoritmos Paralelos en HPC: Tendencias Actuales

El eje de esta línea de I/D lo constituye el estudio de tendencias actuales en las áreas de arquitecturas y algoritmos paralelos. Incluye como temas centrales: - Arquitecturas Many-core (GPU, procesadores MIC), Arquitecturas híbridas (diferentes combinaciones de multicores y GPUs) y Arquitecturas heterogéneas. - HPC en Cloud Computing, especialmente para aplicaciones de Big Data. - Lenguajes y Estructuras de Datos para nuevas arquitecturas de cómputo paralelo. - Desarrollo y evaluación de algoritmos paralelos sobre nuevas arquitecturas y su evaluación de rendimiento energético y computacional. - Empleo de contadores de hardware, en particular en toma de decisiones en tiempo de ejecución

An ACO Algorithm for Effective Cluster Head Selection

This paper presents an effective algorithm for selecting cluster heads in mobile ad hoc networks using ant colony optimization. A cluster in an ad hoc network consists of a cluster head and cluster members which are at one hop away from the cluster head. The cluster head allocates the resources to its cluster members. Clustering in MANET is done to reduce the communication overhead and thereby increase the network performance. A MANET can have many clusters in it. This paper presents an algorithm which is a combination of the four main clustering schemes- the ID based clustering, connectivity based, probability based and the weighted approach. An Ant colony optimization based approach is used to minimize the number of clusters in MANET. This can also be considered as a minimum dominating set problem in graph theory. The algorithm considers various parameters like the number of nodes, the transmission range etc. Experimental results show that the proposed algorithm is an effective methodology for finding out the minimum number of cluster heads.Comment: 7 pages, 5 figures, International Journal of Advances in Information Technology (JAIT); ISSN: 1798-2340; Academy Publishers, Finlan

A novel approach to allocating QoS-constrained workflow-based jobs in a multi-cluster grid

Clusters are increasingly interconnected to form multi-cluster systems, which are becoming popular for scientific computation. Grid users often submit their applications in the form of workflows with certain Quality of Service (QoS) requirements imposed on the workflows. These workflows detail the composition of Grid services and the level of service required from the Grid. This paper addresses workload allocation techniques for Grid workflows. We model a resource within a cluster as a G/G/1 queue and minimise failures (QoS requirement violation) of jobs by solving a mixed-integer non-linear program (MINLP). The novel approach is evaluated through an experimental simulation and the results confirm that the proposed workload allocation strategy not only provides QoS guarantee but also performs considerably better in terms of satisfying QoS requirements of Grid workflows than reservation-based scheduling algorithms. © 2006 ACM

An efficient scheduling method for grid systems based on a hierarchical stochastic petri net

This paper addresses the problem of resource scheduling in a grid computing environment. One of the main goals of grid computing is to share system resources among geographically dispersed users, and schedule resource requests in an efficient manner. Grid computing resources are distributed, heterogeneous, dynamic, and autonomous, which makes resource scheduling a complex problem. This paper proposes a new approach to resource scheduling in grid computing environments, the hierarchical stochastic Petri net (HSPN). The HSPN optimizes grid resource sharing, by categorizing resource requests in three layers, where each layer has special functions for receiving subtasks from, and delivering data to, the layer above or below. We compare the HSPN performance with the Min-min and Max-min resource scheduling algorithms. Our results show that the HSPN performs better than Max-min, but slightly underperforms Min-min

Tendencias en Arquitecturas y Algoritmos Paralelos para HPC

El eje de esta línea de I/D lo constituye el estudio de tendencias actuales en las áreas de arquitecturas y algoritmos paralelos. Incluye como temas centrales:\n Arquitecturas Many-core (GPU, procesadores MIC), Arquitecturas híbridas (diferentes combinaciones de multicores y GPUs) y Arquitecturas heterogéneas.\n Lenguajes y Estructuras de Datos para nuevas arquitecturas de cómputo paralelo.\n Desarrollo y evaluación de algoritmos paralelos sobre nuevas arquitecturas y su evaluación de rendimiento.\n Estudio de las arquitecturas tipo Cloud y el desarrollo de software de base y aplicaciones eficientes en Cloud Computing, en particular en el área de cómputo paralelo de altas prestaciones (HPC).\n Aspectos del consumo energético, en particular en relación con clases de instrucciones y algoritmos paralelos.\n Empleo de contadores de hardware, en particular en toma de decisiones en tiempo de ejecución.\nLas temáticas mencionadas se observan como aristas promisorias en el futuro del cómputo paralelo de altas prestaciones.Eje: Procesamiento Distribuido y Paralel

Tendencias en Arquitecturas y Algoritmos Paralelos para HPC

El eje de esta línea de I/D lo constituye el estudio de tendencias actuales en las áreas de arquitecturas y algoritmos paralelos. Incluye como temas centrales:\n Arquitecturas Many-core (GPU, procesadores MIC), Arquitecturas híbridas (diferentes combinaciones de multicores y GPUs) y Arquitecturas heterogéneas.\n Lenguajes y Estructuras de Datos para nuevas arquitecturas de cómputo paralelo.\n Desarrollo y evaluación de algoritmos paralelos sobre nuevas arquitecturas y su evaluación de rendimiento.\n Estudio de las arquitecturas tipo Cloud y el desarrollo de software de base y aplicaciones eficientes en Cloud Computing, en particular en el área de cómputo paralelo de altas prestaciones (HPC).\n Aspectos del consumo energético, en particular en relación con clases de instrucciones y algoritmos paralelos.\n Empleo de contadores de hardware, en particular en toma de decisiones en tiempo de ejecución.\nLas temáticas mencionadas se observan como aristas promisorias en el futuro del cómputo paralelo de altas prestaciones.Eje: Procesamiento Distribuido y Paralel