Persistent object stores

The design and development of a type secure persistent object store is presented as part of an architecture to support experiments in concurrency, transactions and distribution. The persistence abstraction hides the physical properties of data from the programs that manipulate it. Consequently, a persistent object store is required to be of unbounded size, infinitely fast and totally reliable. A range of architectural mechanisms that can be used to simulate these three features is presented. Based on a suitable selection of these mechanisms, two persistent object stores are presented. The first store is designed for use with the programming language PS-algol. Its design is evolved to yield a more flexible layered architecture. The layered architecture is designed to provide each distinct architectural mechanism as a separate architectural layer conforming to a specified interface. The motivation for this design is two-fold. Firstly, the particular choice of layers greatly simplifies the resulting implementation and secondly, the layered design can support experimental architecture implementations. Since each layer conforms to a specified interface, it is possible to experiment with the implementation of an individual layer without affecting the implementation of the remaining architectural layers. Thus, the layered architecture is a convenient vehicle for experimenting with the implementation of persistent object stores. An implementation of the layered architecture is presented together with an example of how it may be used to support a distributed system. Finally, the architecture's ability to support a variety of storage configurations is presented

Incremental garbage collection in massive object stores

© 2001 IEEEThere are only a few garbage collection algorithms that have been designed to operate over massive object stores. These algorithms operate at two levels, locally via incremental collection of small partitions and globally via detection of cross partition garbage, including cyclic garbage. At each level there is a choice of collection mechanism. For example, the PMOS collector employs tracing at the local level and reference counting at the global level. Another approach implemented in the Thor object database uses tracing at both levels. In this paper we present two new algorithms that both employ reference counting at the local level. One algorithm uses reference counting at the higher level and the other uses tracing at the higher level. An evaluation strategy is presented to support comparisons between these four algorithms and preliminary experiments are outlined

An extensible view system for supporting the integration and interoperation of heterogeneous, autonomous, and distributed database management systems

In this thesis the problem of integrating heterogeneous, autonomous and distributed database management systems (DBMSs) is addressed. To provide a solution, we have developed an approach, a design method, and a view system. Our approach is based on the invention of the abstract view constructs that have uniform and stable representations for supporting semantic relativism and distributed abstraction modeling. Our design method applies object-oriented techniques and software engineering concepts to manage the system complexity. Our view system has been constructed upon established experience with the development of large-scale distributed systems in a distributed object infrastructure provided by the Common Object Request Broker Architecture (CORBA). The scope of our research identifies the goals of Project Zeus in which we have created the Zeus View Mechanism ( ZVM) as the theoretical foundation of our approach. The notion of frameworks has been introduced as part of our design methodology to promote code/design reuse and enhance the portability/extensibility of the architectural design. A multidatabase system, the Zeus Multidatabase System ( ZMS), has provided a test bed for our concept. Project Zeus has exciting prospects. The foundation established in this research has created new directions in multidatabase research and will have a significant impact on future integration and interoperation technologies

Modules as values in a persistent object store

Journal ArticleWe report on an object manager (OM) providing persistent implementations for C ++ classes. Our OM generalizes this problem to that of managing persistent modules, where the module concept is an abstract data type (ADT). This approach permits a powerful suite of module manipulation operations to be applied uniformly to modules of many provenances, including non-class based entities such as conventional object files, application libraries, and shared system libraries. OMOS, a generalized linker and loader, plays a central role in our OM. Class implementations are represented by OMOS modules, which in turn are constructed from OMOS meta-objects encapsulating linkage blueprints. We cleanly solve the problems of (i) logically (but not physically) including executable object files in our OM, (ii) reconciling class inheritance history and linkage history, and (iii) supporting alternative implementations of a class, for client interoperability or version control

PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development

This paper describes PlinyCompute, a system for development of high-performance, data-intensive, distributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the "PC object model") and associated memory management system that has been designed from the ground-up for high performance, distributed, data-intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/function dispatch to the JVM. This hybrid approach---declarative in the large, trusting the programmer's ability to utilize PC object model efficiently in the small---results in a system that is ideal for the development of reusable, data-intensive tools and libraries. Through extensive benchmarking, we show that implementing complex objects manipulation and non-trivial, library-style computations on top of PlinyCompute can result in a speedup of 2x to more than 50x or more compared to equivalent implementations on Spark.Comment: 48 pages, including references and Appendi

Geometry Helps to Compare Persistence Diagrams

Exploiting geometric structure to improve the asymptotic complexity of discrete assignment problems is a well-studied subject. In contrast, the practical advantages of using geometry for such problems have not been explored. We implement geometric variants of the Hopcroft--Karp algorithm for bottleneck matching (based on previous work by Efrat el al.) and of the auction algorithm by Bertsekas for Wasserstein distance computation. Both implementations use k-d trees to replace a linear scan with a geometric proximity query. Our interest in this problem stems from the desire to compute distances between persistence diagrams, a problem that comes up frequently in topological data analysis. We show that our geometric matching algorithms lead to a substantial performance gain, both in running time and in memory consumption, over their purely combinatorial counterparts. Moreover, our implementation significantly outperforms the only other implementation available for comparing persistence diagrams.Comment: 20 pages, 10 figures; extended version of paper published in ALENEX 201

A Methodology for Engineering Collaborative and ad-hoc Mobile Applications using SyD Middleware

Today’s web applications are more collaborative and utilize standard and ubiquitous Internet protocols. We have earlier developed System on Mobile Devices (SyD) middleware to rapidly develop and deploy collaborative applications over heterogeneous and possibly mobile devices hosting web objects. In this paper, we present the software engineering methodology for developing SyD-enabled web applications and illustrate it through a case study on two representative applications: (i) a calendar of meeting application, which is a collaborative application and (ii) a travel application which is an ad-hoc collaborative application. SyD-enabled web objects allow us to create a collaborative application rapidly with limited coding effort. In this case study, the modular software architecture allowed us to hide the inherent heterogeneity among devices, data stores, and networks by presenting a uniform and persistent object view of mobile objects interacting through XML/SOAP requests and responses. The performance results we obtained show that the application scales well as we increase the group size and adapts well within the constraints of mobile devices