Multi-Schema-Version Data Management

Modern agile software development methods allow to continuously evolve software systems by easily adding new features, fixing bugs, and adapting the software to changing requirements and conditions while it is continuously used by the users. A major obstacle in the agile evolution is the underlying database that persists the software system’s data from day one on. Hence, evolving the database schema requires to evolve the existing data accordingly—at this point, the currently established solutions are very expensive and error-prone and far from agile. In this thesis, we present InVerDa, a multi-schema-version database system to facilitate agile database development. Multi-schema-version database systems provide multiple schema versions within the same database, where each schema version itself behaves like a regular single-schema database. Creating new schema versions is very simple to provide the desired agility for database development. All created schema versions can co-exist and write operations are immediately propagated between schema versions with a best-effort strategy. Developers do not have to implement the propagation logic of data accesses between schema versions by hand, but InVerDa automatically generates it. To facilitate multi-schema-version database systems, we equip developers with a relational complete and bidirectional database evolution language (BiDEL) that allows to easily evolve existing schema versions to new ones. BiDEL allows to express the evolution of both the schema and the data both forwards and backwards in intuitive and consistent operations; the BiDEL evolution scripts are orders of magnitude shorter than implementing the same behavior with standard SQL and are even less likely to be erroneous, since they describe a developer’s intention of the evolution exclusively on the level of tables without further technical details. Having the developers’ intentions explicitly given in the BiDEL scripts further allows to create a new schema version by merging already existing ones. Having multiple co-existing schema versions in one database raises the need for a sophisticated physical materialization. Multi-schema-version database systems provide full data independence, hence the database administrator can choose a feasible materialization, whereby the multi-schema-version database system internally ensures that no data is lost. The search space of possible materializations can grow exponentially with the number of schema versions. Therefore, we present an adviser that releases the database administrator from diving into the complex performance characteristics of multi-schema-version database systems and merely proposes an optimized materialization for a given workload within seconds. Optimized materializations have shown to improve the performance for a given workload by orders of magnitude. We formally guarantee data independence for multi-schema-version database systems. To this end, we show that every single schema version behaves like a regular single-schema database independent of the chosen physical materialization. This important guarantee allows to easily evolve and access the database in agile software development—all the important features of relational databases, such as transaction guarantees, are preserved. To the best of our knowledge, we are the first to realize such a multi-schema-version database system that allows agile evolution of production databases with full support of co-existing schema versions and formally guaranteed data independence

The exploitation of parallelism on shared memory multiprocessors

PhD ThesisWith the arrival of many general purpose shared memory multiple processor (multiprocessor) computers into the commercial arena during the mid-1980's, a rift has opened between the raw processing power offered by the emerging hardware and the relative inability of its operating software to effectively deliver this power to potential users. This rift stems from the fact that, currently, no computational model with the capability to elegantly express parallel activity is mature enough to be universally accepted, and used as the basis for programming languages to exploit the parallelism that multiprocessors offer. To add to this, there is a lack of software tools to assist programmers in the processes of designing and debugging parallel programs. Although much research has been done in the field of programming languages, no undisputed candidate for the most appropriate language for programming shared memory multiprocessors has yet been found. This thesis examines why this state of affairs has arisen and proposes programming language constructs, together with a programming methodology and environment, to close the ever widening hardware to software gap. The novel programming constructs described in this thesis are intended for use in imperative languages even though they make use of the synchronisation inherent in the dataflow model by using the semantics of single assignment when operating on shared data, so giving rise to the term shared values. As there are several distinct parallel programming paradigms, matching flavours of shared value are developed to permit the concise expression of these paradigms.The Science and Engineering Research Council

Graph Processing in Main-Memory Column Stores

Evermore, novel and traditional business applications leverage the advantages of a graph data model, such as the offered schema flexibility and an explicit representation of relationships between entities. As a consequence, companies are confronted with the challenge of storing, manipulating, and querying terabytes of graph data for enterprise-critical applications. Although these business applications operate on graph-structured data, they still require direct access to the relational data and typically rely on an RDBMS to keep a single source of truth and access. Existing solutions performing graph operations on business-critical data either use a combination of SQL and application logic or employ a graph data management system. For the first approach, relying solely on SQL results in poor execution performance caused by the functional mismatch between typical graph operations and the relational algebra. To the worse, graph algorithms expose a tremendous variety in structure and functionality caused by their often domain-specific implementations and therefore can be hardly integrated into a database management system other than with custom coding. Since the majority of these enterprise-critical applications exclusively run on relational DBMSs, employing a specialized system for storing and processing graph data is typically not sensible. Besides the maintenance overhead for keeping the systems in sync, combining graph and relational operations is hard to realize as it requires data transfer across system boundaries. A basic ingredient of graph queries and algorithms are traversal operations and are a fundamental component of any database management system that aims at storing, manipulating, and querying graph data. Well-established graph traversal algorithms are standalone implementations relying on optimized data structures. The integration of graph traversals as an operator into a database management system requires a tight integration into the existing database environment and a development of new components, such as a graph topology-aware optimizer and accompanying graph statistics, graph-specific secondary index structures to speedup traversals, and an accompanying graph query language. In this thesis, we introduce and describe GRAPHITE, a hybrid graph-relational data management system. GRAPHITE is a performance-oriented graph data management system as part of an RDBMS allowing to seamlessly combine processing of graph data with relational data in the same system. We propose a columnar storage representation for graph data to leverage the already existing and mature data management and query processing infrastructure of relational database management systems. At the core of GRAPHITE we propose an execution engine solely based on set operations and graph traversals. Our design is driven by the observation that different graph topologies expose different algorithmic requirements to the design of a graph traversal operator. We derive two graph traversal implementations targeting the most common graph topologies and demonstrate how graph-specific statistics can be leveraged to select the optimal physical traversal operator. To accelerate graph traversals, we devise a set of graph-specific, updateable secondary index structures to improve the performance of vertex neighborhood expansion. Finally, we introduce a domain-specific language with an intuitive programming model to extend graph traversals with custom application logic at runtime. We use the LLVM compiler framework to generate efficient code that tightly integrates the user-specified application logic with our highly optimized built-in graph traversal operators. Our experimental evaluation shows that GRAPHITE can outperform native graph management systems by several orders of magnitude while providing all the features of an RDBMS, such as transaction support, backup and recovery, security and user management, effectively providing a promising alternative to specialized graph management systems that lack many of these features and require expensive data replication and maintenance processes

Toward New Ecologies of Cyberphysical Representational Forms, Scales, and Modalities

Research on tangible user interfaces commonly focuses on tangible interfaces acting alone or in comparison with screen-based multi-touch or graphical interfaces. In contrast, hybrid approaches can be seen as the norm for established mainstream interaction paradigms. This dissertation describes interfaces that support complementary information mediations, representational forms, and scales toward an ecology of systems embodying hybrid interaction modalities. I investigate systems combining tangible and multi-touch, as well as systems combining tangible and virtual reality interaction. For each of them, I describe work focusing on design and fabrication aspects, as well as work focusing on reproducibility, engagement, legibility, and perception aspects

Development of Bridge Information Model (BrIM) for digital twinning and management using TLS technology

In the current modern era of information and technology, the concept of Building Information Model (BIM), has made revolutionary changes in different aspects of engineering design, construction, and management of infrastructure assets, especially bridges. In the field of bridge engineering, Bridge Information Model (BrIM), as a specific form of BIM, includes digital twining of the physical asset associated with geometrical inspections and non-geometrical data, which has eliminated the use of traditional paper-based documentation and hand-written reports, enabling professionals and managers to operate more efficiently and effectively. However, concerns remain about the quality of the acquired inspection data and utilizing BrIM information for remedial decisions in a reliable Bridge Management System (BMS) which are still reliant on the knowledge and experience of the involved inspectors, or asset manager, and are susceptible to a certain degree of subjectivity. Therefore, this research study aims not only to introduce the valuable benefits of Terrestrial Laser Scanning (TLS) as a precise, rapid, and qualitative inspection method, but also to serve a novel sliced-based approach for bridge geometric Computer-Aided Design (CAD) model extraction using TLS-based point cloud, and to contribute to BrIM development. Moreover, this study presents a comprehensive methodology for incorporating generated BrIM in a redeveloped element-based condition assessment model while integrating a Decision Support System (DSS) to propose an innovative BMS. This methodology was further implemented in a designed software plugin and validated by a real case study on the Werrington Bridge, a cable-stayed bridge in New South Wales, Australia. The finding of this research confirms the reliability of the TLS-derived 3D model in terms of quality of acquired data and accuracy of the proposed novel slice-based method, as well as BrIM implementation, and integration of the proposed BMS into the developed BrIM. Furthermore, the results of this study showed that the proposed integrated model addresses the subjective nature of decision-making by conducting a risk assessment and utilising structured decision-making tools for priority ranking of remedial actions. The findings demonstrated acceptable agreement in utilizing the proposed BMS for priority ranking of structural elements that require more attention, as well as efficient optimisation of remedial actions to preserve bridge health and safety

Decision support tools for environmentally conscious chemical process design

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Chemical Engineering, 1999.Electronic version available online."September 2000."Includes bibliographical references.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.The environment has emerged as an important determinant of the performance of the modern chemical industry. Process engineering in the 21st century needs to evolve to include environmental issues as part of the design objectives, rather than as constraints on operations. A frequently cited objection to the use of quantitative indicators of environmental performance in product and process design is that the underlying data are too uncertain for the numbers to have any real meaning. This thesis demonstrates that explicit incorporation of uncertainties allows bounds to be established on the confidence of decisions made on the basis of uncertain indicators. The examples provided show that large uncertainties in indicators used to assess environmental performance do not necessarily imply uncertainty in decision-making. A series of computer-aided decision making tools have been developed to decrease the barriers to the use of environmental valuation functions in routine design activities. These tools include: uncertainty propagation of relative performance measures, a spreadsheet-based fate, transport and exposure model for chemicals, an information content chart for assessing the quality of uncertain indicators, a screening procedure to identify the most important structural and parametric uncertainties in multimedia exposure models,a process by product input-output life cycle assessment method to generate correlated distributions of unit environmental indicators, an extension of the deterministic equivalent modeling method for the generation of spreadsheet based polynomial chaos expansion metamodels of process flowsheet models, and a database for managing uncertain parameters used in environmental valuation models. Case studies are presented to help the reader in learning the use of the tools. The tools are also applied to an analysis of the U.S. toxics release inventory, in which confidence bounds are developed for the trends in impacts and the contributions of industrial sectors and specific chemical compounds to overall potential impact. Although the tools were developed bearing in mind the need for methods to evaluate the environmental performance of chemical process design alternatives, the ideas can be applied to any decision context in which there are significant uncertainties in the parameters of the objective function.by José Alejandro Cano Ruiz.Ph.D

The Telecommunications and Data Acquisition Report

This quarterly publication provides archival reports on developments in programs managed by JPL's Telecommunications and Mission Operations Directorate (TMOD), which now includes the former Telecommunications and Data Acquisition (TDA) Office. In space communications, radio navigation, radio science, and ground-based radio and radar astronomy, it reports on activities of the Deep Space Network (DS) in planning, supporting research and technology, implementation, and operations. Also included are standards activity at JPL for space data and information systems and reimbursable DSN work performed for other space agencies through NASA. The preceding work is all performed for NASA's Office of Space Communications (OSC)