Database evolution on an orthogonal persistent programming system: A semi-transparent approach

In this paper the problem of the evolution of an object-oriented database in the context of orthogonal persistent programming systems is addressed. We have observed two characteristics in that type of systems that offer particular conditions to implement the evolution in a semi-transparent fashion. That transparency can further be enhanced with the obliviousness provided by the Aspect-Oriented Programming techniques. Was conceived a meta-model and developed a prototype to test the feasibility of our approach. The system allows programs, written to a schema, access semi-transparently to data in other versions of the schema

Safe Class and Data Evolution in Large and Long-Lived Java Applications

There is a growing class of applications implemented in object-oriented languages that are large and complex, that exploit object persistence, and need to run uninterrupted for long periods of time. Development and maintenance of such applications can present challenges in the following interrelated areas: consistent and scalable evolution of persistent data and code, optimal build management, and runtime changes to applications. The research presented in this thesis addresses the above issues. Since Java is becoming increasingly popular platform for implementing large and long-lived applications, it was chosen for experiments. The first part of the research was undertaken in the context of the PJama system, an orthogonally persistent platform for Java. A technology that supports persistent class and object evolution for this platform was designed, built and evaluated. This technology integrates build management, persistent class evolution, and support for several forms of eager conversion of persistent objects. Research in build management for Java has resulted in the creation of a generally applicable, compiler-independent smart recompilation technology, which can be re-used in a Java IDE, or as a standalone Java-specific utility similar to make. The technology for eager object conversion that we developed allows the developers to perform arbitrarily complex changes to persistent objects and their collections. A high level of developer's control over the conversion process was achieved in part due to introduction of a mechanism for dynamic renaming of old class versions. This mechanism was implemented using minor non-standard extensions to the Java language. However, we also demonstrate how to achieve nearly the same results without modifying the language specification. In this form, we believe, our technology can be largely re-used with practically any persistent object solution for Java. The second part of this research was undertaken using as an implementation platform the HotSpot Java Virtual Machine (JVM), which is currently Sun's main production JVM. A technology was developed that allows the engineers to redefine classes on-the-fly in the running VM. Our main focus was on the runtime evolution of server-type applications, though we also address modification of applications running in the debugger. Unlike the only other similar system for Java known to us, our technology supports redefinition of classes that have methods currently active. Several policies for handling such methods have been proposed, one of them is currently operational, another one is in the experimental stage. We also propose to re-use the runtime evolution technology for dynamic fine-grain profiling of applications

Limited Copies and Leased References for Distributed Persistent Objects

As businesses become global organisations and as e-commerce opens up markets to customers across the Internet, demand grows for increasingly ambitious distributed software applications and platforms. Where these applications run over potentially huge collections of data, sophisticated management of data storage and communication is required. There is a need for well-integrated persistence and distribution support that considers the implications for long-term maintenance of valuable persistent data. Orthogonal persistence is intended to ease the programmer's job by providing support for data management that is integrated with a programming language. The simplicity of the orthogonal persistence model argues for its use in distributed systems, in order to make life simpler for the application programmer. PJRMI is an implementation of Java RMI for the orthogonally-persistent PJama platform. This dissertation addresses two problem areas raised by combining orthogonal persistence with support for distributed applications. These problem areas are illustrated by PJRMI. The first problem is raised as a consequence of attempting to provide the illusion of a persistent connection between stores. Distribution-related errors easily break this illusion. In an open system, it can be difficult to determine when an object should become persistent by remote reachability. In the long term, persistent references to remote objects threaten the maintainability of the persistent stores involved. A solution has been implemented to address the problems raised by maintaining persistent references between distributed stores. Greater autonomy of individual stores is achieved by limiting remote access to objects to a duration of time associated with a specific distributed application's lifetime. Within the application's lifetime, the benefits are retained of persistence of inter-store references for resilience. The second problem is encountered when copying object graphs between stores. Large object graphs tend to build up in persistent stores over time. Copying such large object graphs can be prohibitively expensive in terms of resources and performance. A programmer may assume that the size of graph they are copying is acceptable, based on their knowledge of a system in its infancy. However, the problem is that, in a long-lived system, their assumptions may be challenged, since the size of an object graph and the context in which it is used are more likely to change during a persistent object graph's lifetime. The combination of a typically statically-defined policy for passing objects to remote sites and programmer assumptions that fail to take into account the lifetime of an object can also result in other problems. These problems include failure to support different requirements on remote use of the same object graph by different applications during that object graph's lifetime. A solution has been implemented to address the problems raised by remote copying of large object graphs. Flexibility of control over such copying is achieved. Separation of policy from object definition ensures flexibility. Choice of object-copying policy for a specific distributed application's lifetime provides control, while ensuring it is adaptable to changes in size of persistent object graphs over their lifetime and to changes in the context in which these graphs are used

Management of Long-Running High-Performance Persistent Object Stores

The popularity of object-oriented programming languages, such as Java and C++, for large application development has stirred an interest in improved technologies for high-performance, reliable, and scalable object storage. Such storage systems are typically referred to as Persistent Object Stores. This thesis describes the design and implementation of Sphere, a new persistent object store developed at the University of Glasgow, Scotland. The requirements for Sphere included high performance, support for transactional multi-threaded loads, scalability, extensibility, portability, reliability, referential integrity via the use of disk garbage collection, provision for flexible schema evolution, and minimised interaction with the mutator. The Sphere architecture is split into two parts: the core and the application-specific customisations. The core was designed to be modular, in order to encourage research and experimentation, and to be as light-weight as possible, in an attempt to achieve high performance through simplicity. The customisation part includes the code that deals with and is optimised for the specific load of the application that Sphere has to support: object formats, free-space management, etc. Even though specialising this part of the store is not trivial, it has the benefit that the interaction between the mutator and Sphere is direct and more efficient, as translation layers are not necessary. Major design decisions for Sphere included (i) splitting the store into partitions, to facilitate incremental disk garbage collection and schema evolution, (ii) using a flexible two-level free-space management, (Hi) introducing a three-dimensional method-dispatch matrix to invoke store operations, which contributes to Sphere's ease-of-extensibility, (iv) adopting a logical addressing scheme, to allow straightforward object and partition relocation, (v) requiring that Sphere can identify reference fields inside objects, so that it does not have to interact with the mutator in order to do so, and (vi) adopting the well-known ARIES recovery algorithm to ensure fault-tolerance. The thesis contains a detailed overview of Sphere and the context in which it was developed. Then, it concentrates on two areas that were explored using Sphere as the implementation platform. First, bulk object-loading issues are discussed and the Ghosted Allocation promotion algorithm is described. This algorithm was designed to allocate large numbers of objects to a store efficiently and with minimal log traffic and was evaluated using large-scale experiments. Second, the disk garbage collection framework of Sphere is overviewed and the implemented compacting, relocating garbage collector is described, along with the model of synchronisation with the mutator

Data Persistence in Eiffel

This dissertation describes an extension to the Eiffel programming language that provides automatic object persistence (the ability of programs to store objects and later recreate those objects in a subsequent execution of a program). The mechanism is orthogonal to other aspects of the Eiffel language. The mechanism serves four main purposes: 1) it gives Eiffel programmers a needed service, filling a gap between serialization, which provides limited persistence functions and database-mapping, which is cumbersome to use; 2) it greatly reduces the coding burden incurred by the programmer when objects must persist, allowing the programmer to focus instead on the business model; 3) it provides a platform for testing the benefits of orthogonal persistence in Eiffel, and 4) it furnishes a model for orthogonal persistence in other object-oriented languages. During my research, I created a prototype implementation of the persistence mechanism using it effectively in several programs. Performance measurements showed acceptable performance with some increase in program memory usage. The prototype gives the programmer the ability to add automatic persistence to existing code with the addition of only a few lines of code. The size of this additional code remains constant regardless of the total number of lines of code in the project. Eiffel syntax remains unchanged and nonpersistent Eiffel code runs as is while incur- ring only a very small speed penalty

Transactions and schema evolution in a persistent object-oriented programming system

Applications are subject of a continuous evolution process with a profound impact on their underlining data model, hence requiring frequent updates in the applications' class structure and database structure as well. This twofold problem, schema evolution and instance adaptation, usually known as database evolution, is addressed in this thesis. Additionally, we address concurrency and error recovery problems with a novel meta-model and its aspect-oriented implementation. Modern object-oriented databases provide features that help programmers deal with object persistence, as well as all related problems such as database evolution, concurrency and error handling. In most systems there are transparent mechanisms to address these problems, nonetheless the database evolution problem still requires some human intervention, which consumes much of programmers' and database administrators' work effort. Earlier research works have demonstrated that aspect-oriented programming (AOP) techniques enable the development of flexible and pluggable systems. In these earlier works, the schema evolution and the instance adaptation problems were addressed as database management concerns. However, none of this research was focused on orthogonal persistent systems. We argue that AOP techniques are well suited to address these problems in orthogonal persistent systems. Regarding the concurrency and error recovery, earlier research showed that only syntactic obliviousness between the base program and aspects is possible. Our meta-model and framework follow an aspect-oriented approach focused on the object-oriented orthogonal persistent context. The proposed meta-model is characterized by its simplicity in order to achieve efficient and transparent database evolution mechanisms. Our meta-model supports multiple versions of a class structure by applying a class versioning strategy. Thus, enabling bidirectional application compatibility among versions of each class structure. That is to say, the database structure can be updated because earlier applications continue to work, as well as later applications that have only known the updated class structure. The specific characteristics of orthogonal persistent systems, as well as a metadata enrichment strategy within the application's source code, complete the inception of the meta-model and have motivated our research work. To test the feasibility of the approach, a prototype was developed. Our prototype is a framework that mediates the interaction between applications and the database, providing them with orthogonal persistence mechanisms. These mechanisms are introduced into applications as an {\it aspect} in the aspect-oriented sense. Objects do not require the extension of any super class, the implementation of an interface nor contain a particular annotation. Parametric type classes are also correctly handled by our framework. However, classes that belong to the programming environment must not be handled as versionable due to restrictions imposed by the Java Virtual Machine. Regarding concurrency support, the framework provides the applications with a multithreaded environment which supports database transactions and error recovery. The framework keeps applications oblivious to the database evolution problem, as well as persistence. Programmers can update the applications' class structure because the framework will produce a new version for it at the database metadata layer. Using our XML based pointcut/advice constructs, the framework's instance adaptation mechanism is extended, hence keeping the framework also oblivious to this problem. The potential developing gains provided by the prototype were benchmarked. In our case study, the results confirm that mechanisms' transparency has positive repercussions on the programmer's productivity, simplifying the entire evolution process at application and database levels. The meta-model itself also was benchmarked in terms of complexity and agility. Compared with other meta-models, it requires less meta-object modifications in each schema evolution step. Other types of tests were carried out in order to validate prototype and meta-model robustness. In order to perform these tests, we used an OO7 small size database due to its data model complexity. Since the developed prototype offers some features that were not observed in other known systems, performance benchmarks were not possible. However, the developed benchmark is now available to perform future performance comparisons with equivalent systems. In order to test our approach in a real world scenario, we developed a proof-of-concept application. This application was developed without any persistence mechanisms. Using our framework and minor changes applied to the application's source code, we added these mechanisms. Furthermore, we tested the application in a schema evolution scenario. This real world experience using our framework showed that applications remains oblivious to persistence and database evolution. In this case study, our framework proved to be a useful tool for programmers and database administrators. Performance issues and the single Java Virtual Machine concurrent model are the major limitations found in the framework

Open Multithreaded Transactions: A Transaction Model for Concurrent Object-Oriented Programming

To read the abstract, please go to my PhD home page

Towards Scalable and Recoverable Object Evolution for the PJama Persistent Platform

Abstract. Two developing technologies, namely Java Platform based on the Solaris Research VM and Sphere persistent store, have been recently combined to provide a new version of orthogonally persistent Java | PJama1. Having these powerful technologies, the developers were given an excellent window of opportunity to provide evolution support as a standard, scalable and reliable mechanism. This paper describes the issues arising during the design of the store level foundation for the evolution technology, and our solutions. We have achieved our goals of making evolution scalable and recoverable, yet retaining the unchanged view of the \old object world " during object conversion, at a price of minor constraints to the semantics of custom evolution (conversion) code. A way to remove this constraint is proposed.