A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

A Taxonomy of Workflow Management Systems for Grid Computing

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.Comment: 29 pages, 15 figure

Capacity planning and stochastic scheduling in grid computing environments

Imperial Users onl

Limiting Global Warming by Improving Data-Centre Software

Carbon emissions, greenhouse gases and pollution in general are usually related to traditional factories, so the most modern computing factories have gone unnoticed for the general-public opinion. We empirically show through extensive and realistic simulation that: 1) energy consumption, and consequently CO2 emissions, could be reduced from ~15% to ~60% if the correct energy-efficiency policies are applied; and 2) such energy-consumption reduction can be achieved without negatively impacting the correct operation of these infrastructures. To this end, this work is focused on the proposal and analysis of a set of energy-efficiency policies which are applied to traditional and hyper-scale data centres, as well as numerous operation environments, including: 1) the top resource managers used in industry; 2) eight energy-efficiency policies, including aggressive, fine-tuned and adaptive models; and 3) three types of workload-arrival patterns. Finally, we present a realistic analysis of the environmental impact of the application of such energy-efficiency policies on USA data centres. The presented results estimate that 11.5 million of tons of CO2 could be saved, which is equivalent to the removal of 4.79 million of combustion cars, that is, the total car fleet of countries such as Portugal, Austria and Sweden.Ministerio de Ciencia e Innovación RTI2018-098062-A-I0

A comparison of resource allocation process in grid and cloud technologies

Grid Computing and Cloud Computing are two different technologies that have emerged to validate the long-held dream of computing as utilities which led to an important revolution in IT industry. These technologies came with several challenges in terms of middleware, programming model, resources management and business models. These challenges are seriously considered by Distributed System research. Resources allocation is a key challenge in both technologies as it causes the possible resource wastage and service degradation. This paper is addressing a comprehensive study of the resources allocation processes in both technologies. It provides the researchers with an in-depth understanding of all resources allocation related aspects and associative challenges, including: load balancing, performance, energy consumption, scheduling algorithms, resources consolidation and migration. The comparison also contributes an informal definition of the Cloud resource allocation process. Resources in the Cloud are being shared by all users in a time and space sharing manner, in contrast to dedicated resources that governed by a queuing system in Grid resource management. Cloud Resource allocation suffers from extra challenges abbreviated by achieving good load balancing and making right consolidation decision

Improving shared access to Cloud of Things resources.

Cloud of Things (CoT) is an emerging paradigm that integrates Cloud Computing and Internet of Things (IoT) to support a wide range of real-world applications. Resource allocation plays a vital role in CoT, especially when allocating IoT physical resources to Cloud-based applications to ensure seamless application execution. Due to the heterogeneity and the constrained capacities of IoT resources, resource allocation is a challenge. This complexity leads to missing/limiting shared access to the IoT physical resources and consequently lessen the reusability of the resources across multiple applications. This issue results in, 1) replicating IoT deployments making them expensive and not feasible for many prospective users, 2) existing IoT infrastructures are over-provisioned to meet the unpredictable application requirements in which resources may be significantly underutilised, and 3) the adoption of CoT is slowed. Improving shared access to CoT resources can provide efficient resource allocation, improve resource utilisation and likely to reduce the cost of IoT deployments. Existing solutions include small-scale, hardware and platform-dependent mechanisms to enable or improve shared access to IoT resources. The research presented in this thesis considers trading CoT resources in a marketplace as an approach to improve shared access to CoT resources. It proposes a solution to Cot resource allocation that re-imagines CoT resources as commodities that can be provided and consumed by the marketplace participants. The novel contributions of the research presented in this thesis are summarised as follows: 1) a model to describe and quantify the value of CoT resources, 2) a resource sharing and allocation strategy called Exclusive Shared Access (ESA) to CoT resources, 3) a QoS-aware optimisation model for trading CoT resources as a single and multipleobjective optimisation problem, and 4) a marketplace architecture and experimental evaluation to verify its performance and scalability

The Gridbus Toolkit for Service Oriented Grid and Utility Computing: An Overview and Status Report

Grids aim at exploiting synergies that result from cooperation of autonomous distributed entities. The synergies that result from grid cooperation include the sharing, exchange, selection, and aggregation of geographically distributed resources such as computers, data bases, software, and scientific instruments for solving large-scale problems in science, engineering, and commerce. For this cooperation to be sustainable, participants need to have economic incentive. Therefore, "incentive" mechanisms should be considered as one of key design parameters of Grid architectures. In this article, we present an overview and status of an open source Grid toolkit, called Gridbus, whose architecture is fundamentally driven by the requirements of Grid economy. Gridbus technologies provide services for both computational and data grids that power the emerging eScience and eBusiness applications.Comment: 11 pages, 3 figures, 3 table