Enabling object storage via shims for grid middleware

The Object Store model has quickly become the basis of most commercially successful mass storage infrastructure, backing so-called "Cloud" storage such as Amazon S3, but also underlying the implementation of most parallel distributed storage systems. Many of the assumptions in Object Store design are similar, but not identical, to concepts in the design of Grid Storage Elements, although the requirement for "POSIX-like" filesystem structures on top of SEs makes the disjunction seem larger. As modern Object Stores provide many features that most Grid SEs do not (block level striping, parallel access, automatic file repair, etc.), it is of interest to see how easily we can provide interfaces to typical Object Stores via plugins and shims for Grid tools, and how well experiments can adapt their data models to them. We present evaluation of, and first-deployment experiences with, (for example) Xrootd-Ceph interfaces for direct object-store access, as part of an initiative within GridPP[1] hosted at RAL. Additionally, we discuss the tradeoffs and experience of developing plugins for the currently-popular Ceph parallel distributed filesystem for the GFAL2 access layer, at Glasgow

Storing RDF as a Graph

RDF is the first W3C standard for enriching information resources of the Web with detailed meta data. The semantics of RDF data is defined using a RDF schema. The most expressive language for querying RDF is RQL, which enables querying of semantics. In order to support RQL, a RDF storage system has to map the RDF graph model onto its storage structure. Several storage systems for RDF data have been developed, which store the RDF data as triples in a relational database. To evaluate an RQL query on those triple structures, the graph model has to be rebuilt from the triples. In this paper, we presented a new approach to store RDF data as a graph in a object-oriented database. Our approach avoids the costly rebuilding of the graph and efficiently queries the storage structure directly. The advantages of our approach have been shown by performance test on our prototype implementation OO-Store

Improvements for Store-Collect and Atomic Snapshot Objects under Continuous Churn

The field of distributed computing has given rise to many algorithms to share data among nodes in a network. This work focuses on the store-collect and the atomic snapshot objects in an asynchronous, crash-prone message-passing dynamic system with nodes continuously entering and leaving the system. We assume that the maximum number of nodes that enter, leave or crash during some time interval is proportional to the size of the system. A store-collect object is a distributed object that allows nodes to store data in the system in a variable that can be read by all nodes, but only modified by the node that stored it. This is achieved through two basic operations: the store operation, which stores information into the network, and collect, which collects a copy of all the information stored by every node in the network at the beginning of the time interval in which the operation is active. The atomic snapshot object is quite similar. It provides two operations, scan and update, that behave in a very similar fashion to the collect and store operations given by the store-collect object; however the atomic snapshot object must satisfy the linearizability condition, which means that it is always possible to arrange all the operations performed into an ordered sequence even if there are operations that occur simultaneously. This work improves upon the store-collect and atomic snapshot implementations given in Attiya et al [SSS, 2020]. We developed a method for quantifying the churn of a network subject to certain assumptions. This new method allows us to prove the correctness of the store-collect algorithm under less restrictive conditions than those found in the original proof of Attiya et al. Additionally, we developed an improved implementation of the atomic snapshot object based on a store-collect object that requires fewer messages to complete a scan or an update operation

Path planning by querying persistent stores of trajectory segments

We introduce an algorithm for path planning (long duration) paths of dynamical systems, given a persistent object store containing suitable collections of short duration trajectory segments. We also describe experimental results from a proof-of-concept implementation of the algorithm. The basic idea is to interpret a path planning algorithm as a suitable query on a persistent object store consisting of short duration trajectory segments. The query returns a concatenation of short duration trajectory segments which is close to the desired path. The needed short duration segments are computed by using a divide and conquer algorithm to break up the original path into shorter paths; each shorter path is then matched to a nearby trajectory segment which is part of the persistent object store by using a suitable index function

Event Indexing Systems for Efficient Selection and Analysis of HERA Data

The design and implementation of two software systems introduced to improve the efficiency of offline analysis of event data taken with the ZEUS Detector at the HERA electron-proton collider at DESY are presented. Two different approaches were made, one using a set of event directories and the other using a tag database based on a commercial object-oriented database management system. These are described and compared. Both systems provide quick direct access to individual collision events in a sequential data store of several terabytes, and they both considerably improve the event analysis efficiency. In particular the tag database provides a very flexible selection mechanism and can dramatically reduce the computing time needed to extract small subsamples from the total event sample. Gains as large as a factor 20 have been obtained.Comment: Accepted for publication in Computer Physics Communication

Associative access in persistent object stores : a thesis presented in partial fulfilment of the requirements for the degree of Master of Information Sciences in Information Systems at Massey University

Page 276 missing from original copy.The overall aim of the thesis is to study associative access in a Persistent Object Store (POS) providing necessary object storage and retrieval capabilities to an Object Oriented Database System (OODBS) (Delis, Kanitkar & Kollios, 1998 cited in Kirchberg & Tretiakov, 2002). Associative access in an OODBS often includes navigational access to referenced or referencing objects of the object being accessed (Kim. Kim. & Dale. 1989). The thesis reviews several existing approaches proposed to support associative and navigational access in an OODBS. It was found that the existing approaches proposed for associative access could not perform well when queries involve multiple paths or inheritance hierarchies. The thesis studies how associative access can be supported in a POS regardless of paths or inheritance hierarchies involved with a query. The thesis proposes extensions to a model of a POS such that approaches that are proposed for navigational access can be used to support associative access in the extended POS. The extensions include (1) approaches to cluster storage objects in a POS on their storage classes or values of attributes, and (2) approaches to distinguish references between storage objects in a POS based on criteria such as reference types - inheritance and association, storage classes of referenced storage objects or referencing storage objects, and reference names. The thesis implements Matrix-Index Coding (MIC) approach with the extended POS by several coding techniques. The implementation demonstrates that (1) a model of a POS extended by proposed extensions is capable of supporting associative access in an OODBS and (2) the MIC implemented with the extended POS can support a query that requires associative access in an OODBS and involves multiple paths or inheritance hierarchies. The implementation also provides proof of the concepts suggested by Kirchberg & Tretiakov (2002) that (1) the MIC can be made independent from a coding technique, and (2) data compression techniques should be considered as appropriate alternatives to implement the MIC because they could reduce the storage size required

Designing and Implementing a Learning Object Repository: Issues of Complexity, Granularity, and User Sense-Making

4th International Conference on Open RepositoriesThis presentation was part of the session : DSpace User Group PresentationsDate: 2009-05-20 03:30 PM – 05:00 PMThe Texas Center for Digital Knowledge at the University of North Texas is designing and implementing a DSpace/Manakin learning object repository (LOR) for the Texas Higher Education Coordinating Board to store and provide access to redesigned undergraduate courses being created through the Board's Texas Course Redesign Project (TCRP). The content for the THECB LOR differs in significant ways from content stored in other well-known and evolving LORs, since the content is in the form of complete or partial courses. While this content can be represented as a single learning object (i.e., a complete course as one learning object), the THECB LOR is making the complete courses available as learning objects and it is providing access to components of the courses' content as discrete learning objects for reuse and repurposing. A number of challenges and issues have emerged in the design, development, and implementation the LOR, and this paper focuses on three key aspects and the solutions we are pursuing: 1) complexity of the course content and granularity; 2) submission of complex objects and metadata; and 3) user interface design to assist users in making sense of this repository and its contents.Texas Higher Education Coordinating Boar