9,200 research outputs found
A grid-based infrastructure for distributed retrieval
In large-scale distributed retrieval, challenges of latency, heterogeneity, and dynamicity emphasise the importance of infrastructural support in reducing the development costs of state-of-the-art solutions. We present a service-based infrastructure for distributed retrieval which blends middleware facilities and a design framework to ‘lift’ the resource sharing approach and the computational services of a European Grid platform into the domain of e-Science applications. In this paper, we give an overview of the DILIGENT Search Framework and illustrate its exploitation in the field of Earth Science
A Fault Tolerant, Dynamic and Low Latency BDII Architecture for Grids
The current BDII model relies on information gathering from agents that run
on each core node of a Grid. This information is then published into a Grid
wide information resource known as Top BDII. The Top level BDIIs are updated
typically in cycles of a few minutes each. A new BDDI architecture is proposed
and described in this paper based on the hypothesis that only a few attribute
values change in each BDDI information cycle and consequently it may not be
necessary to update each parameter in a cycle. It has been demonstrated that
significant performance gains can be achieved by exchanging only the
information about records that changed during a cycle. Our investigations have
led us to implement a low latency and fault tolerant BDII system that involves
only minimal data transfer and facilitates secure transactions in a Grid
environment.Comment: 18 pages; 10 figures; 4 table
A Backend Framework for the Efficient Management of Power System Measurements
Increased adoption and deployment of phasor measurement units (PMU) has
provided valuable fine-grained data over the grid. Analysis over these data can
provide insight into the health of the grid, thereby improving control over
operations. Realizing this data-driven control, however, requires validating,
processing and storing massive amounts of PMU data. This paper describes a PMU
data management system that supports input from multiple PMU data streams,
features an event-detection algorithm, and provides an efficient method for
retrieving archival data. The event-detection algorithm rapidly correlates
multiple PMU data streams, providing details on events occurring within the
power system. The event-detection algorithm feeds into a visualization
component, allowing operators to recognize events as they occur. The indexing
and data retrieval mechanism facilitates fast access to archived PMU data.
Using this method, we achieved over 30x speedup for queries with high
selectivity. With the development of these two components, we have developed a
system that allows efficient analysis of multiple time-aligned PMU data
streams.Comment: Published in Electric Power Systems Research (2016), not available
ye
Impliance: A Next Generation Information Management Appliance
ably successful in building a large market and adapting to the changes of the
last three decades, its impact on the broader market of information management
is surprisingly limited. If we were to design an information management system
from scratch, based upon today's requirements and hardware capabilities, would
it look anything like today's database systems?" In this paper, we introduce
Impliance, a next-generation information management system consisting of
hardware and software components integrated to form an easy-to-administer
appliance that can store, retrieve, and analyze all types of structured,
semi-structured, and unstructured information. We first summarize the trends
that will shape information management for the foreseeable future. Those trends
imply three major requirements for Impliance: (1) to be able to store, manage,
and uniformly query all data, not just structured records; (2) to be able to
scale out as the volume of this data grows; and (3) to be simple and robust in
operation. We then describe four key ideas that are uniquely combined in
Impliance to address these requirements, namely the ideas of: (a) integrating
software and off-the-shelf hardware into a generic information appliance; (b)
automatically discovering, organizing, and managing all data - unstructured as
well as structured - in a uniform way; (c) achieving scale-out by exploiting
simple, massive parallel processing, and (d) virtualizing compute and storage
resources to unify, simplify, and streamline the management of Impliance.
Impliance is an ambitious, long-term effort to define simpler, more robust, and
more scalable information systems for tomorrow's enterprises.Comment: This article is published under a Creative Commons License Agreement
(http://creativecommons.org/licenses/by/2.5/.) You may copy, distribute,
display, and perform the work, make derivative works and make commercial use
of the work, but, you must attribute the work to the author and CIDR 2007.
3rd Biennial Conference on Innovative Data Systems Research (CIDR) January
710, 2007, Asilomar, California, US
AstroGrid-D: Grid Technology for Astronomical Science
We present status and results of AstroGrid-D, a joint effort of
astrophysicists and computer scientists to employ grid technology for
scientific applications. AstroGrid-D provides access to a network of
distributed machines with a set of commands as well as software interfaces. It
allows simple use of computer and storage facilities and to schedule or monitor
compute tasks and data management. It is based on the Globus Toolkit middleware
(GT4). Chapter 1 describes the context which led to the demand for advanced
software solutions in Astrophysics, and we state the goals of the project. We
then present characteristic astrophysical applications that have been
implemented on AstroGrid-D in chapter 2. We describe simulations of different
complexity, compute-intensive calculations running on multiple sites, and
advanced applications for specific scientific purposes, such as a connection to
robotic telescopes. We can show from these examples how grid execution improves
e.g. the scientific workflow. Chapter 3 explains the software tools and
services that we adapted or newly developed. Section 3.1 is focused on the
administrative aspects of the infrastructure, to manage users and monitor
activity. Section 3.2 characterises the central components of our architecture:
The AstroGrid-D information service to collect and store metadata, a file
management system, the data management system, and a job manager for automatic
submission of compute tasks. We summarise the successfully established
infrastructure in chapter 4, concluding with our future plans to establish
AstroGrid-D as a platform of modern e-Astronomy.Comment: 14 pages, 12 figures Subjects: data analysis, image processing,
robotic telescopes, simulations, grid. Accepted for publication in New
Astronom
Cyberaide JavaScript: A Web Application Development Framework for Cyberinfrastructure
This thesis work introduces a service oriented architecture based Grid abstraction framework that allows users to access Grid infrastructure through JavaScript. Such a framework integrates well with other Web 2.0 technologies since it provides JavaScript toolkit to build web applications. The framework consists of two essential parts. A client Application Programming lnterface (API) to access the Grid via JavaScript and a full service stack in server side through which the Grid access is channeled. The framework uses commodity Web service standards and provides extended functionality such as asynchronous task management, file transfer, etc. The availability of this framework simplifies not only the development of new services, but also the development of advanced client side Grid applications that can be accessed through Web browsers. The effectiveness of the framework is demonstrated by providing an Grid portal example that integrates a variety of useful services to be accessed through a JavaScript enabled client desktop via a Web browser, as well as the opensocial gadgets for
Grid task management and file transfer. Overall, Grid developers will have another tool at their disposal that projects a simpler way to distribute and maintain cyberinfrastructure related software, while simultaneously delivering advanced interfaces and integrating social services for the scientific community
AMP: A Science-driven Web-based Application for the TeraGrid
The Asteroseismic Modeling Portal (AMP) provides a web-based interface for
astronomers to run and view simulations that derive the properties of Sun-like
stars from observations of their pulsation frequencies. In this paper, we
describe the architecture and implementation of AMP, highlighting the
lightweight design principles and tools used to produce a functional
fully-custom web-based science application in less than a year. Targeted as a
TeraGrid science gateway, AMP's architecture and implementation are intended to
simplify its orchestration of TeraGrid computational resources. AMP's web-based
interface was developed as a traditional standalone database-backed web
application using the Python-based Django web development framework, allowing
us to leverage the Django framework's capabilities while cleanly separating the
user interface development from the grid interface development. We have found
this combination of tools flexible and effective for rapid gateway development
and deployment.Comment: 7 pages, 2 figures, in Proceedings of the 5th Grid Computing
Environments Worksho
- …