Distributed Management of Massive Data: an Efficient Fine-Grain Data Access Scheme

This paper addresses the problem of efficiently storing and accessing massive data blocks in a large-scale distributed environment, while providing efficient fine-grain access to data subsets. This issue is crucial in the context of applications in the field of databases, data mining and multimedia. We propose a data sharing service based on distributed, RAM-based storage of data, while leveraging a DHT-based, natively parallel metadata management scheme. As opposed to the most commonly used grid storage infrastructures that provide mechanisms for explicit data localization and transfer, we provide a transparent access model, where data are accessed through global identifiers. Our proposal has been validated through a prototype implementation whose preliminary evaluation provides promising results

A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

ATLAS Distributed Data management Operations

ATLAS Distributed Data Management (DDM) service is developed for data transfer between ATLAS sites and for data cataloguing. The Data Management Software (SW) is based on DQ2 and end-users tools (aka dq2_get package). In this paper we address the issue of DDM day-by-day operation, DDM operations team organization, roles and responsibilities of Tier-1s and Tier-2s DDM coordinators

Distributed Data Management in 2020?

Work on distributed data management commenced shortly after the introduction of the relational model in the mid-1970's. 1970's and 1980's were very active periods for the development of distributed relational database technology, and claims were made that in the following ten years centralized databases will be an “antique curiosity” and most organizations will move toward distributed database managers [1]. That prediction has certainly become true, and all commercial DBMSs today are distributed

Waste Management Using a Multilevel Distributed System and Data Mining

Administration is conducted through the control of events and management of problems in the territory. Economical growth and nowadays technologies lead to difficult problems related to environmental protection against pollution and to people safety against various direct threats from air soil, food. In this respect, an increasing importance get the collection of information and its processing and interpretation just to understand and discover threats and potential disturbance of the environment and health. The paper proposes a multilevel system for the administrative bodies involved in environment matters at local regional and national levels, which may collect and scrutiny data on waste generation, spread and reuse/elimination, and provide sound instruments to assist decision makers of the corresponding levels, using Data Mining and Business Intelligence.Management, environment, information system, business intelligence, data mining.