620 research outputs found
Grid resource discovery based on web services
The size of grid systems has increased substantially in the last decades. Resource discovery in grid systems is a fundamental task which provides searching and locating necessary resources for a given process. Various different approaches are proposed in literature for this problem. Grid resource discovery using web services is an important approach which has resulted in many tools to become de facto standards of today's grid resource management. In this paper, we propose a survey of recent grid resource discovery studies based on web services. We provide synthesis, analysis and evaluation of these studies by classification. We also give a comparative study of different classes proposed
Recommended from our members
HARD: Hybrid Adaptive Resource Discovery for Jungle Computing
In recent years, Jungle Computing has emerged as a distributed computing paradigm based on simultaneous combination of various hierarchical and distributed computing environments which are composed by large number of heterogeneous resources. In such a computing environment, the resources and the underlying computation and communication infrastructures are highly-hierarchical and heterogeneous. This creates a lot of difficulty and complexity for finding the proper resources in a precise way in order to run a particular job on the system efficiently. This paper proposes Hybrid Adaptive Resource Discovery (HARD), a novel efficient and highly scalable resource-discovery approach which is built upon a virtual hierarchical overlay based on self-organization and self-adaptation of processing resources in the system, where the computing resources are organized into distributed hierarchies according to a proposed hierarchical multi-layered resource description model. The proposed approach supports distributed query processing within and across hierarchical layers by deploying various distributed resource discovery services and functionalities in the system which are implemented using different adapted algorithms and mechanisms in each level of hierarchy. The proposed approach addresses the requirements for resource discovery in Jungle Computing environments such as high-hierarchy, high-heterogeneity, high-scalability and dynamicity. Simulation results show significant scalability and efficiency of the proposed approach over highly heterogeneous, hierarchical and dynamic computing environments
DECENTRALIZED RESOURCE ORCHESTRATION FOR HETEROGENEOUS GRIDS
Modern desktop machines now use multi-core CPUs to enable improved performance. However, achieving high performance on multi-core machines without optimized software support is still difficult even in a single machine, because contention for shared resources can make it hard to exploit multiple computing resources efficiently. Moreover, more diverse and heterogeneous hardware platforms (e.g. general-purpose GPU and Cell processors) have emerged and begun to impact grid computing. Given that heterogeneity and diversity are now a major trend going forward, grid computing must support these environmental changes.
In this dissertation, I design and evaluate a decentralized resource management scheme to exploit heterogeneous multiple computing resources effectively. I suggest resource management algorithms that can efficiently utilize a diverse computational environment, including multiple symmetric computing entities and heterogeneous multi-computing entities, and achieve good load-balancing and high total system throughput. Moreover, I propose expressive resource description techniques to accommodate more heterogeneous environments, allowing incoming jobs with complex requirements to be matched to available resources.
First, I develop decentralized resource management frameworks and job scheduling schemes to exploit multi-core nodes in peer-to-peer grids. I present two new load-balancing schemes that explicitly account for resource sharing and contention across multiple cores within a single machine, and propose a simple performance prediction model that can represent a continuum of resource sharing among cores of a CPU. Second, I provide scalable resource discovery and load balancing techniques to accommodate nodes with many types of computing elements, such as multi-core CPUs and GPUs, in a peer-to-peer grid architecture. My scheme takes into account diverse aspects of heterogeneous nodes to maximize overall system throughput as well as minimize messaging costs without sacrificing the failure resilience provided by an underlying peer-to-peer overlay network. Finally, I propose an expressive resource discovery method to support multi-attribute, range-based job constraints. The common approach of using simple attribute indexes does not suffice, as range-based constraints may be satisfied by more than a single value. I design a compact ID-based representation for resource characteristics, and integrate this representation into the decentralized resource discovery framework.
By extensive experimental results via simulation, I show that my schemes can match heterogeneous jobs to heterogeneous resources both effectively (good matches are found, load is balanced), and efficiently (the new functionality imposes little overhead)
Resource Management in a Peer to Peer Cloud Network for IoT
Software-Defined Internet of Things (SDIoT) is defined as merging heterogeneous objects in a form of interaction among physical and virtual entities. Large scale of data centers, heterogeneity issues and their interconnections have made the resource management a hard problem specially when there are different actors in cloud system with different needs. Resource management is a vital requirement to achieve robust networks specially with facing continuously increasing amount of heterogeneous resources and devices to the network. The goal of this paper is reviews to address IoT resource management issues in cloud computing services. We discuss the bottlenecks of cloud networks for IoT services such as mobility. We review Fog computing in IoT services to solve some of these issues. It provides a comprehensive literature review of around one hundred studies on resource management in Peer to Peer Cloud Networks and IoT. It is very important to find a robust design to efficiently manage and provision requests and available resources. We also reviewed different search methodologies to help clients find proper resources to answer their needs
A Practical Study of Self-Stabilization for Prefix-Tree Based Overlay Networks
Service discovery is crucial in the development of fully decentralized computational grids. Among the significant amount of work produced by the convergence of peer-to-peer (P2P) systems and grids, a new kind of overlay networks, based on prefix trees, has emerged. In particular, the Distributed Lexicographic Placement Table (DLPT) approach is a decentralized and dynamic service discovery service. Fault-tolerance within the DLPT approach is achieved through best-effort policies relying on formal self-stabilization results. Self-stabilization means that the tree can become transiently inconsistent, but is guaranteed to autonomously converge to a correct topology after arbitrary crashes, in a finite time. However, during convergence, the tree may not be able to process queries correctly. In this paper, we present some simulation results having several objectives. First, we investigate the interest of self-stabilization for such architectures. Second, we explore, still based on simulation, a simple Time-To-Live policy to avoid useless processing during convergence time
Efficiency of Tree-Structured Peer-to-Peer Service Discovery Systems
The efficiency of service discovery is a crucial point in the development of fully decentralized middlewares intended to manage large scale computational grids. The work conducted on this issue led to the design of many peer-to-peer fashioned approaches. More specifically, the need for flexibility and complexity in the service discovery has seen the emergence of a new kind of overlays, based on tries, also known as lexicographic trees. Although these overlays are efficient and well designed, they require a costly maintenance and do not accurately take into account the heterogeneity of nodes and the changing popularity of the services requested by users. In this paper, we focus on reducing the cost of the maintenance of a particular architecture, based on a dynamic prefix tree, while enhancing it with some load balancing techniques that dynamically adapt the load of the nodes in order to maximize the throughput of the system. The algorithms developed couple a self-organizing prefix tree overlay with load balancing techniques inspired by similar previous works undertaken for distributed hash tables. After some simulation results showing how our load balancing heuristics perform in such an overlay and compare to other heuristics, we provide a fair comparison of this architecture and similar overlays recently proposed.L’efficacité de la découverte de services est un point crucial du développement d’intergiciels de grille totalement décentralisés. Les travaux ayant pour but la résolution de ce problème ont généré un certain nombre d’approches pair-à-pair. le besoin de flexibilité et d’expressivité a donné lieu au développement d’architecture s’appuyant sur des arbres de préfixes(ou arbres lexicographiques). Ces overlays souffrent d’une maintenance couteuse et ne prennent pas en compte la nature hétérogène de la plate-forme physique sous-jacente et la popularité différente et changeante de chaque ressource enregistrée.Dans ce rapport, nous nous focalisons sur la réduction du cout de maintenance d’une telle architecture, basée sur un arbre de préfixes dynamique,tout en lui donnant la possibilité de s’adapter à l’hétérogénéité précitée par l’enrichissant de mécanismes de répartition de la charge qui adaptent dynamiquement la charge des nœuds dans le but de maximiser le débit sur service. Notre approche couple des travaux de répartition de la charge dans les DHTs avec un overlay en arbre de préfixes auto-organisant. Après des résultats de simulation mettant en évidence l’efficacité de notre heuristique, nous comparons notre approche avec les travaux s’appuyant sur des structures distribuées similaires
Semantic-Based, Scalable, Decentralized and Dynamic Resource Discovery for Internet-Based Distributed System
Resource Discovery (RD) is a key issue in Internet-based distributed sytems such as
grid. RD is about locating an appropriate resource/service type that matches the user's
application requirements. This is very important, as resource reservation and task
scheduling are based on it. Unfortunately, RD in grid is very challenging as resources
and users are distributed, resources are heterogeneous in their platforms, status of the
resources is dynamic (resources can join or leave the system without any prior notice)
and most recently the introduction of a new type of grid called intergrid (grid of grids)
with the use of multi middlewares. Such situation requires an RD system that has rich
interoperability, scalability, decentralization and dynamism features. However,
existing grid RD systems have difficulties to attain these features. Not only that, they
lack the review and evaluation studies, which may highlight the gap in achieving the
required features. Therefore, this work discusses the problem associated with intergrid
RD from two perspectives. First, reviewing and classifying the current grid RD
systems in such a way that may be useful for discussing and comparing them. Second,
propose a novel RD framework that has the aforementioned required RD features. In
the former, we mainly focus on the studies that aim to achieve interoperability in the
first place, which are known as RD systems that use semantic information (semantic
technology). In particular, we classify such systems based on their qualitative use of
the semantic information. We evaluate the classified studies based on their degree of
accomplishment of interoperability and the other RD requirements, and draw the
future research direction of this field. Meanwhile in the latter, we name the new
framework as semantic-based scalable decentralized dynamic RD. The framework
further contains two main components which are service description, and service
registration and discovery models. The earlier consists of a set of ontologies and
services. Ontologies are used as a data model for service description, whereas the
services are to accomplish the description process. The service registration is also based on ontology, where nodes of the service (service providers) are classified to
some classes according to the ontology concepts, which means each class represents a
concept in the ontology. Each class has a head, which is elected among its own class
I
nodes/members. Head plays the role of a registry in its class and communicates with
I
the other heads of the classes in a peer to peer manner during the discovery process.
We further introduce two intelligent agents to automate the discovery process which
are Request Agent (RA) and Description Agent (DA). Eaclj. node is supposed to have
both agents. DA describes the service capabilities based on the ontology, and RA
I
carries the service requests based on the ontology as well. We design a service search
I
algorithm for the RA that starts the service look up from the class of request origin
first, then to the other classes.
We finally evaluate the performance of our framework ~ith extensive simulation
experiments, the result of which confirms the effectiveness of the proposed system in
satisfying the required RD features (interoperability, scalability, decentralization and
dynamism). In short, our main contributions are outlined new key taxonomy for the
semantic-based grid RD studies; an interoperable semantic description RD component
model for intergrid services metadata representation; a semantic distributed registry
architecture for indexing service metadata; and an agent-qased service search and
selection algorithm.
Vll
Recommended from our members
Grid-based semantic integration of heterogeneous data resources: Implementation on a HealthGrid
This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University.The semantic integration of geographically distributed and heterogeneous data
resources still remains a key challenge in Grid infrastructures. Today's
mainstream Grid technologies hold the promise to meet this challenge in a
systematic manner, making data applications more scalable and manageable. The
thesis conducts a thorough investigation of the problem, the state of the art, and
the related technologies, and proposes an Architecture for Semantic Integration of
Data Sources (ASIDS) addressing the semantic heterogeneity issue. It defines a
simple mechanism for the interoperability of heterogeneous data sources in order
to extract or discover information regardless of their different semantics. The
constituent technologies of this architecture include Globus Toolkit (GT4) and
OGSA-DAI (Open Grid Service Architecture Data Integration and Access)
alongside other web services technologies such as XML (Extensive Markup
Language). To show this, the ASIDS architecture was implemented and tested in a
realistic setting by building an exemplar application prototype on a HealthGrid
(pilot implementation).
The study followed an empirical research methodology and was informed by
extensive literature surveys and a critical analysis of the relevant technologies and
their synergies. The two literature reviews, together with the analysis of the
technology background, have provided a good overview of the current Grid and
HealthGrid landscape, produced some valuable taxonomies, explored new paths
by integrating technologies, and more importantly illuminated the problem and
guided the research process towards a promising solution. Yet the primary
contribution of this research is an approach that uses contemporary Grid
technologies for integrating heterogeneous data resources that have semantically
different. data fields (attributes). It has been practically demonstrated (using a
prototype HealthGrid) that discovery in semantically integrated distributed data
sources can be feasible by using mainstream Grid technologies, which have been
shown to have some Significant advantages over non-Grid based approaches
Resource discovery for distributed computing systems: A comprehensive survey
Large-scale distributed computing environments provide a vast amount of heterogeneous computing resources from different sources for resource sharing and distributed computing. Discovering appropriate resources in such environments is a challenge which involves several different subjects. In this paper, we provide an investigation on the current state of resource discovery protocols, mechanisms, and platforms for large-scale distributed environments, focusing on the design aspects. We classify all related aspects, general steps, and requirements to construct a novel resource discovery solution in three categories consisting of structures, methods, and issues. Accordingly, we review the literature, analyzing various aspects for each category
- …