Safe Class and Data Evolution in Large and Long-Lived Java Applications

There is a growing class of applications implemented in object-oriented languages that are large and complex, that exploit object persistence, and need to run uninterrupted for long periods of time. Development and maintenance of such applications can present challenges in the following interrelated areas: consistent and scalable evolution of persistent data and code, optimal build management, and runtime changes to applications. The research presented in this thesis addresses the above issues. Since Java is becoming increasingly popular platform for implementing large and long-lived applications, it was chosen for experiments. The first part of the research was undertaken in the context of the PJama system, an orthogonally persistent platform for Java. A technology that supports persistent class and object evolution for this platform was designed, built and evaluated. This technology integrates build management, persistent class evolution, and support for several forms of eager conversion of persistent objects. Research in build management for Java has resulted in the creation of a generally applicable, compiler-independent smart recompilation technology, which can be re-used in a Java IDE, or as a standalone Java-specific utility similar to make. The technology for eager object conversion that we developed allows the developers to perform arbitrarily complex changes to persistent objects and their collections. A high level of developer's control over the conversion process was achieved in part due to introduction of a mechanism for dynamic renaming of old class versions. This mechanism was implemented using minor non-standard extensions to the Java language. However, we also demonstrate how to achieve nearly the same results without modifying the language specification. In this form, we believe, our technology can be largely re-used with practically any persistent object solution for Java. The second part of this research was undertaken using as an implementation platform the HotSpot Java Virtual Machine (JVM), which is currently Sun's main production JVM. A technology was developed that allows the engineers to redefine classes on-the-fly in the running VM. Our main focus was on the runtime evolution of server-type applications, though we also address modification of applications running in the debugger. Unlike the only other similar system for Java known to us, our technology supports redefinition of classes that have methods currently active. Several policies for handling such methods have been proposed, one of them is currently operational, another one is in the experimental stage. We also propose to re-use the runtime evolution technology for dynamic fine-grain profiling of applications

Views and concerns and interrelationships : Lessons learned from developing the multi-View software engineering environment PIROL

Software-Entwicklungsumgebungen sind komplexe Systeme mit besonderen Anforderungen an Modularität und Anpaßbarkeit. Diese Arbeit beschreibt die Entwicklung der Umgebung PIROL. Die Beschreibung ist dabei in eine Abfolge der folgenden 12 Themen gegliedert. (1) Metamodellierung ist das Grundkonzept, nach dem PIROL seine Daten gemäß einem objektorientierten Datenmodell zerlegt, so daß beliebige Werkzeuge auch auf die Daten anderer Werkzeuge auf sinnvolle Art und Weise zuzugreifen können. (2) Das Metamodell wird zur persistenten Speicherung der Daten auf Konzepte des Repositories H-PCTE abgebildet. (3) Die Granularität eines Metamodells ist für die Effektivität und Effizienz des Gesamtsystems entscheidend. PIROL unterstützt hybride Modellierung als Kompromiß beider Extreme. (4) Durch Methoden des Metamodells wird Verhaltensmodellierung für verschiedenste Aufgaben unterstützt. (5) Ausnahmebehandlung wird systematisch unterstützt. (6) Verschiedene Mechanismen zur Wahrung der Datenintegrität sind enthalten. (7) Das System wurde nach einer Client-Server Architektur entwickelt, deren zentrale Komponente eine "Workbench" ist, die die Repository-Sprache Lua/P ausführt. (8) Steuerungsintegration erlaubt durch verteilte Steuerflüsse das enge Zusammenspiel lose gekoppelter Komponenten. (9) Die koordinierte Zusammenarbeit mehrerer Benutzer wird unterstützt. (10) Die logische Unabhängigkeit von Werkzeugen wird durch das neue Konzept der Dynamic View Connectors erreicht. (11) Allgemeine Dienste sind in der Umgebung einheitlich verfügbar. (12) Das System unterstützt die Weiterentwicklung. All diese Themengebiete sind sehr eng miteinander verzahnt und die Darstellung ist zu großem Teil der gegenseitigen Beeinflussung gewidmet. Es wird gezeigt, wie eine Großzahl der Entwurfsentscheidungen genau aus diesen Beeinflussungen motiviert sind. Die Beschreibung folgt damit dem Konzept der "Concern Interaction Matrix", das hier zur Bewältigung von Komplexität vorgeschlagen wird. Dabei werden Charakteristika einzelner Anliegen und einzelner Zusammenhänge herausgearbeitet. Die Beschreibung PIROLs wird durch die Liste der integrierten Werkzeuge, Ansätze von Laufzeit-Messungen und einige Betrachtungen zur Beurteilung abgerundet. Abschließend werden verschiedene Konzepte rund um den Begriff "Sichten" erörtert. Sichten sind ein zentrale Anliegen von PIROL. Außerdem generalisiert die Diskussion über die mehrdimensionale Darstellung des Hauptteiles. Es werden Begrifflichkeit, Konzepte und Techniken für Sichten in der Softwaretechnik vorgestellt und diskutiert. Dabei wird die Brücke geschlagen von Sichten in objektorientierten Datenbanken, über aspekt-orientierte Softwareentwicklung bis hin zum allgemeinen "Concern Modeling", zu dem die o.g. Methode einen Beitrag leisten soll. Sichten werden dabei als ein zentrales Konzept der Softwaretechnik neben Abstraktion und Zerlegung beurteilt. Dynamic View Connectors sind ein wesentlicher Beitrag von PIROL, durch den Datenbanksichten und aspektorientierte Programmierung zusammengeführt werden. Zwar ist der Sichten-Begriff längst nicht so scharf definiert, wie die Begriffe Abstraktion und Zerlegung, aber gerade die Überlappungen und Diskrepanzen, die durch Sichten abgebildet werden können, machen dies Konzept zu einem starken Strukturierungsprinzip, das zwar einigen Aufwand zur Behandlung von Inkonsistenzen erfordert, aber andererseits hilft, komplexe Systeme handhabbar und wartbar zu gestalten.Software engineering environments are complex systems with special requirements regarding modularity and adaptability. This thesis describes the development of the environment PIROL. The description is structured as a sequence of the following 12 concerns: (1) Meta modeling is the basic concept by which PIROL decomposes its data in accordance to an object-oriented data model. This allows arbitrary tools to access data of other tools in a meaningful way. (2) For persistent storage the model is mapped to the concepts of the repository H-PCTE. (3) The granularity of a meta model determines effectiveness and efficiency of the system. PIROL supports hybrid modeling as a compromise between extremes. (4) By methods of the meta model behavior modeling is supported for a wide range of tasks. (5) Exception handling is supported systematically. (6) Several mechanisms for preserving data integrity are integrated. (7) The system follows a client-server architecture. As its, central component the "workbench" executes the repository language LuaP. (8) Control integration allows for close cooperation of loosely coupled components by means of distributed control flows. (9) The coordinated cooperation of multiple users is supported. (10) Logical independence of tools is achieved by the novel concept of Dynamic View Connectors. (11) Common services are available throughout the environment in a uniform way. (12) The system is prepared for evolution. All these concerns are tightly interlocked. A considerable share of the presentation is dedicated to such mutual interactions. Is is shown, how a large number of design decisions is motivated exactly by these interactions. The description follows the concept of a "Concern Interaction Matrix" which is proposed for managing complexity. Characteristics of concerns and their interactions are elaborated. The description of PIROL is completed by a list of integrated tools, initial performance measurements and evaluation. Finally, several concepts relating to the notion of "views" are discussed. Views are a central concern of PIROL. Furthermore, the discussion generalizes over the multi-dimensional presentation in the body of this thesis. Notions, concepts and techniques for views in software engineering are presented and discussed. This discussion connects views in object-oriented databases, aspect-oriented software development and general "concern modeling", to which the method of "Concern Interaction Matrices" contributes. Views are regarded as a central concept of software engineering at the same level as abstraction and decomposition. Dynamic View Connectors are a significant contribution of PIROL that combines database views and aspect-oriented programming. The notion of "views" is defined with far less precision than abstraction and decomposition, but indeed by the overlap and mismatches, which can be captured by views, this concept is a strong principle for structuring software and information. Effort is needed for handling inconsistencies as they may arise, but after all, views are a suitable means for managing the complexity of systems and for designing these systems for evolution

Developing Error Handling Software for Object-Oriented Geographical Information

The inclusion of error handling capabilities within geographical information systems (GIS) is seen by many as crucial to the future commercial and legal stability of the technology. This thesis describes the analysis, design, implementation and use of a GIS able to handle both geographical information (GI) and the error associated with that GI. The first stage of this process is the development of an error-sensitive GIS, able to provide core error handling functionality in a form flexible enough to be widely applicable to error-prone GI. Object-oriented (OO) analysis, design and programming techniques, supported by recent developments in formal OO theory, are used to implement an error-sensitive GIS within Laser-Scan Gothic OOGIS software. The combination of formal theory and GIS software implementation suggests that error-sensitive GIS are a practical possibility using OO technology. While the error-sensitive GIS is an important step toward full error handling systems, it is expected that most GIS users would require additional high level functionality before use of error- sensitive GIS could become commonplace. There is a clear need to provide error handling systems that actively assist non-expert users in assessing, using and understanding error in GI. To address this need, an error-aware GIS offering intelligent domain specific error handling software tools was developed, based on the core error-sensitive functionality. In order to provide a stable software bridge between the flexible error-sensitive GIS and specialised error-aware software tools, the error-aware GIS makes use of a distributed systems component architecture. The component architecture allows error-aware software tools that extend core error-sensitive functionality to be developed with minimal time and cost overheads. Based on a telecommunications application in Kingston-upon-Hull, UK, three error-aware tools were developed to address particular needs identified within the application. First, an intelligent hypertext system in combination with a conventional expert system was used to assist GIS users with error-sensitive database design. Second, an inductive learning algorithm was used to automatically populate the error-sensitive database with information about error, based on a small pilot error assessment. Finally, a visualisation and data integration tool was developed to allow access to the error-sensitive database and error propagation routines to users across the Internet. While a number of important avenues of further work are implied by this research, the results of this research provide a blueprint for the development of practical error handling capabilities within GIS. The architecture used is both robust and flexible, and arguably represents a framework both for future research and for the development of commercial error handling GIS

Model-driven generative programming for BIS mobile applications

The burst on the availability of smart phones based on the Android platform calls for cost-effective techniques to generate mobile apps for general purpose, distributed business information systems (BIS). To mitigate this problem our research aims at applying model-driven techniques to automatically generate usable prototypes with a sound, maintainable, architecture. Following three base principles: model-based generation, separation of concerns, paradigm seamlessness, we try to answer the main guiding question – how to reduce development time and cost by transforming a given domain model into an Android application? To answer this question we propose to develop an application that follows a generative approach for mobile BIS apps that will mitigate the identified problems. Its input is a platform independent model (PIM), with business rules specified in OCL (Object Constraint Language). We adopted the Design Science Research methodology, that helps gaining problem understanding, identifying systemically appropriate solutions, and in effectively evaluating new and innovative solutions. To better evaluate our solution, besides resorting to third party tools to test specific components integration, we demonstrated its usage and evaluated how well it mitigates a subset of the identified problems in an observational study (we presented our generated apps to an outside audience in a controlled environment to study our model-based centered and, general apps understandability) and communicated its effectiveness to researchers and practitioners.O grande surto de disponibilidade de dispositivos móveis para a plataforma Android requer, técnicas generativas de desenvolvimento de aplicações para sistemas comuns e/ou distribuídos de informação empresariais/negócio, que otimizem a relação custo-benefício. Para mitigar este problema, esta investigação visa aplicar técnicas orientadas a modelos para, automaticamente, gerar protótipos funcionais de aplicações com uma arquitetura robusta e fácil de manter. Seguindo para tal três princípios base: geração baseada no modelo, separação de aspetos, desenvolvimento sem soturas (sem mudança de paradigma), tentamos dar resposta à pergunta orientadora – como reduzir o tempo e custo de desenvolvimento de uma aplicação Android por transformação de um dado modelo de domínio? De modo a responder a esta questão nós propomos desenvolver uma aplicação que segue uma abordagem generativa para aplicações de informação empresariais/negócio móveis de modo a mitigar os problemas identificados. Esta recebe modelos independentes de plataforma (PIM), com regras de negócio especificadas em OCL (Object Constraint Language). Seguimos a metodologia Design Science Research que ajuda a identificar e perceber o problema, a identificar sistematicamente soluções apropriadas aos problemas e a avaliar mais eficientemente soluções novas e inovadoras. Para melhor avaliar a nossa solução, apesar de recorrermos a ferramentas de terceiros para testar a integração de componentes específicos, também demonstramos a sua utilização, através de estudos experimentais (em um ambiente controlado, apresentamos as nossas aplicações geradas a uma audiência externa que nos permitiu estudar a compreensibilidade baseada e centrada em modelos e, de um modo geral, das aplicações) avaliamos o quanto esta mitiga um subconjunto de problemas identificados e comunicamos a sua eficácia para investigadores e profissionais

CazDataProvider: a solution to the object-relational mismatch

Dissertação de mestrado em Engenharia de InformáticaToday, most software applications require mechanisms to store information persistently. For decades, Relational Database Management Systems (RDBMSs) have been the most common technology to provide efficient and reliable persistence. Due to the object-relational paradigm mismatch, object oriented applications that store data in relational databases have to deal with Object Relational Mapping (ORM) problems. Since the emerging of new ORM frameworks, there has been an attempt to lure developers for a radical paradigm shift. However, they still often have troubles finding the best persistence mechanism for their applications, especially when they have to bear with legacy database systems. The aim of this dissertation is to discuss the persistence problem on object oriented applications and find the best solutions. The main focus lies on the ORM limitations, patterns, technologies and alternatives. The project supporting this dissertation was implemented at Cachapuz under the Project Global Weighting Solutions (GWS). Essentially, the objectives of GWS were centred on finding the optimal persistence layer for CazFramework, mostly providing database interoperability with close-to-Structured Query Language (SQL) querying. Therefore, this work provides analyses on ORM patterns, frameworks, alternatives to ORM like Object-Oriented Database Management Systems (OODBMSs). It also describes the implementation of CazDataProvider, a .NET library tool providing database interoperability and dynamic query features. In the end, there is a performance comparison of all the technologies debated in this dissertation. The result of this dissertation provides guidance for adopting the best persistence technology or implement the most suitable ORM architectures.Hoje, a maioria dos aplicações requerem mecanismos para armazenar informação persistentemente. Durante décadas, as RDBMSs têm sido a tecnologia mais comum para fornecer persistência eficiente e confiável. Devido à incompatibilidade dos paradigmas objetos-relacional, as aplicações orientadas a objetos que armazenam dados em bases de dados relacionais têm de lidar com os problemas do ORM. Desde o surgimento de novas frameworks ORM, houve uma tentativa de atrair programadores para uma mudança radical de paradigmas. No entanto, eles ainda têm muitas vezes dificuldade em encontrar o melhor mecanismo de persistência para as suas aplicações, especialmente quando eles têm de lidar com bases de dados legadss. O objetivo deste trabalho é discutir o problema de persistência em aplicações orientadas a objetos e encontrar as melhores soluções. O foco principal está nas limitações, padrões e tecnologias do ORM bem como suas alternativas. O projeto de apoio a esta dissertação foi implementado na Cachapuz no âmbito do Projeto GWS. Essencialmente, os objetivos do GWS foram centrados em encontrar a camada de persistência ideal para a CazFramework, principalmente fornecendo interoperabilidade de base de dados e consultas em SQL. Portanto, este trabalho fornece análises sobre padrões, frameworks e alternativas ao ORM como OODBMS. Além disso descreve a implementação do CazDataProvider, uma biblioteca .NET que fornece interoperabilidade de bases de dados e consultas dinâmicas. No final, há uma comparação de desempenho de todas as tecnologias discutidas nesta dissertação. O resultado deste trabalho fornece orientação para adotar a melhor tecnologia de persistência ou implementar as arquiteturas ORM mais adequadas

BUILDING DSS USING KNOWLEDGE DISCOVERY IN DATABASE APPLIED TO ADMISSION & REGISTRATION FUNCTIONS

This research investigates the practical issues surrounding the development and implementation of Decision Support Systems (DSS). The research describes the traditional development approaches analyzing their drawbacks and introduces a new DSS development methodology. The proposed DSS methodology is based upon four modules; needs' analysis, data warehouse (DW), knowledge discovery in database (KDD), and a DSS module. The proposed DSS methodology is applied to and evaluated using the admission and registration functions in Egyptian Universities. The research investigates the organizational requirements that are required to underpin these functions in Egyptian Universities. These requirements have been identified following an in-depth survey of the recruitment process in the Egyptian Universities. This survey employed a multi-part admission and registration DSS questionnaire (ARDSSQ) to identify the required data sources together with the likely users and their information needs. The questionnaire was sent to senior managers within the Egyptian Universities (both private and government) with responsibility for student recruitment, in particular admission and registration. Further, access to a large database has allowed the evaluation of the practical suitability of using a data warehouse structure and knowledge management tools within the decision making framework. 1600 students' records have been analyzed to explore the KDD process, and another 2000 records have been used to build and test the data mining techniques within the KDD process. Moreover, the research has analyzed the key characteristics of data warehouses and explored the advantages and disadvantages of such data structures. This evaluation has been used to build a data warehouse for the Egyptian Universities that handle their admission and registration related archival data. The decision makers' potential benefits of the data warehouse within the student recruitment process will be explored. The design of the proposed admission and registration DSS (ARDSS) will be developed and tested using Cool: Gen (5.0) CASE tools by Computer Associates (CA), connected to a MSSQL Server (6.5), in a Windows NT (4.0) environment. Crystal Reports (4.6) by Seagate will be used as a report generation tool. CLUST AN Graphics (5.0) by CLUST AN software will also be used as a clustering package. Finally, the contribution of this research is found in the following areas: A new DSS development methodology; The development and validation of a new research questionnaire (i.e. ARDSSQ); The development of the admission and registration data warehouse; The evaluation and use of cluster analysis proximities and techniques in the KDD process to find knowledge in the students' records; And the development of the ARDSS software that encompasses the advantages of the KDD and DW and submitting these advantages to the senior admission and registration managers in the Egyptian Universities. The ARDSS software could be adjusted for usage in different countries for the same purpose, it is also scalable to handle new decision situations and can be integrated with other systems

The advantages and cost effectiveness of database improvement methods

Relational databases have proved inadequate for supporting new classes of applications, and as a consequence, a number of new approaches have been taken (Blaha 1998), (Harrington 2000). The most salient alternatives are denormalisation and conversion to an object-oriented database (Douglas 1997). Denormalisation can provide better performance but has deficiencies with respect to data modelling. Object-oriented databases can provide increased performance efficiency but without the deficiencies in data modelling (Blaha 2000). Although there have been various benchmark tests reported, none of these tests have compared normalised, object oriented and de-normalised databases. This research shows that a non-normalised database for data containing type code complexity would be normalised in the process of conversion to an objectoriented database. This helps to correct badly organised data and so gives the performance benefits of de-normalisation while improving data modelling. The costs of conversion from relational databases to object oriented databases were also examined. Costs were based on published benchmark tests, a benchmark carried out during this study and case studies. The benchmark tests were based on an engineering database benchmark. Engineering problems such as computer-aided design and manufacturing have much to gain from conversion to object-oriented databases. Costs were calculated for coding and development, and also for operation. It was found that conversion to an object-oriented database was not usually cost effective as many of the performance benefits could be achieved by the far cheaper process of de-normalisation, or by using the performance improving facilities provided by many relational database systems such as indexing or partitioning or by simply upgrading the system hardware. It is concluded therefore that while object oriented databases are a better alternative for databases built from scratch, the conversion of a legacy relational database to an object oriented database is not necessarily cost effective

Managing Schema Change in an Heterogeneous Environment

Change is inevitable even for persistent information. Effectively managing change of persistent information, which includes the specification, execution and the maintenance of any derived information, is critical and must be addressed by all database systems. Today, for every data model there exists a well-defined set of change primitives that can alter both the structure (the schema) and the data. Several proposals also exist for incrementally propagating a primitive change to any derived information (or view). However, existing support is lacking in two ways. First, change primitives as presented in literature are very limiting in terms of their capabilities allowing users to simply add or remove schema elements. More complex types of changes such the merging or splitting of schema elements are not supported in a principled manner. Second, algorithms for maintaining derived information often do not account for the potential heterogeneity between the source and the target. The goal of this dissertation is to provide solutions that address these two key issues. The first part of this dissertation addresses the challenge of expressing a rich complex set of changes. We propose the SERF (Schema Evolution through an Extensible, Re-usable and Flexible) framework that allows users to perform a wide range of complex user-defined schema transformations. Our approach combines existing schema evolution primitives using OQL (object query language) as the glue logic. Within the context of this work, we look at the different domains in which SERF can be applied, including web site management. To further enrich our framework, we also investigate the optimization and verification of SERF transformations. The second part of this dissertation addresses the problem of maintaining views in the face of source changes when the source and the view are not in the same data model. With today\u27s increasing heterogeneity in information structure, it is critical that maintenance of views addresses the data model boundaries. However, view definitions that go across data models are limited to hard-coded algorithms, thereby making it difficult to develop general maintenance algorithms. We provide a two-step solution for this problem. We have developed a cross algebra, that defines views such that there is no restriction that forces the view and the source data models to be the same. We then define update propagation algorithms that can propagate changes from source to target irrespective of the exact translation and the data models. We validate our ideas by applying them to translation and change propagation between the XML and relational data models