The advantages and cost effectiveness of database improvement methods

Relational databases have proved inadequate for supporting new classes of applications, and as a consequence, a number of new approaches have been taken (Blaha 1998), (Harrington 2000). The most salient alternatives are denormalisation and conversion to an object-oriented database (Douglas 1997). Denormalisation can provide better performance but has deficiencies with respect to data modelling. Object-oriented databases can provide increased performance efficiency but without the deficiencies in data modelling (Blaha 2000). Although there have been various benchmark tests reported, none of these tests have compared normalised, object oriented and de-normalised databases. This research shows that a non-normalised database for data containing type code complexity would be normalised in the process of conversion to an objectoriented database. This helps to correct badly organised data and so gives the performance benefits of de-normalisation while improving data modelling. The costs of conversion from relational databases to object oriented databases were also examined. Costs were based on published benchmark tests, a benchmark carried out during this study and case studies. The benchmark tests were based on an engineering database benchmark. Engineering problems such as computer-aided design and manufacturing have much to gain from conversion to object-oriented databases. Costs were calculated for coding and development, and also for operation. It was found that conversion to an object-oriented database was not usually cost effective as many of the performance benefits could be achieved by the far cheaper process of de-normalisation, or by using the performance improving facilities provided by many relational database systems such as indexing or partitioning or by simply upgrading the system hardware. It is concluded therefore that while object oriented databases are a better alternative for databases built from scratch, the conversion of a legacy relational database to an object oriented database is not necessarily cost effective

Multi Agent Systems for the Active Management of Electrical Distribution Networks

This Thesis presents an investigation on the technical impacts caused by the steady state operation of Small-Scale Embedded Generators (SSEGs) and also introduces the Small Scale Energy Zone (SSEZ) concept which aims to remove the technical barriers associated with SSEGs through intelligent coordination of large numbers of customerowned SSEGs, energy storage units and controllable loads. This approach represents a move away from the conventional passive, “fit-and-forget” philosophy under which the majority of Low Voltage (LV) distribution networks are currently operated and towards a higher degree of network operational management. The employment of a distributed management and control approach for an SSEZ, realised through the Multi Agent Systems (MAS) technology, is proposed due to the advantages that can potentially be realised in the areas of: (i) scalability and openness, (ii) reliability and resilience and (iii) communications efficiency. A FIPA-compliant MAS-based control approach is designed, developed and evaluated based on the specific SSEZ control requirements. The MAS is composed of three types of agents: direct control agents, indirect control agents and utility agents, exchanging information through the employment of a common ontology. In addition, a relational database management system is also designed and developed in order to be coupled with the developed MAS for data management purposes

Model Transformation Languages with Modular Information Hiding

Model transformations, together with models, form the principal artifacts in model-driven software development. Industrial practitioners report that transformations on larger models quickly get sufficiently large and complex themselves. To alleviate entailed maintenance efforts, this thesis presents a modularity concept with explicit interfaces, complemented by software visualization and clustering techniques. All three approaches are tailored to the specific needs of the transformation domain

DACA: arquitetura para implementação de mecanismos dinâmicos de controlo de acesso em camadas de negócio

Doutoramento em Ciências da ComputaçãoAccess control is a software engineering challenge in database applications. Currently, there is no satisfactory solution to dynamically implement evolving fine-grained access control mechanisms (FGACM) on business tiers of relational database applications. To tackle this access control gap, we propose an architecture, herein referred to as Dynamic Access Control Architecture (DACA). DACA allows FGACM to be dynamically built and updated at runtime in accordance with the established fine-grained access control policies (FGACP). DACA explores and makes use of Call Level Interfaces (CLI) features to implement FGACM on business tiers. Among the features, we emphasize their performance and their multiple access modes to data residing on relational databases. The different access modes of CLI are wrapped by typed objects driven by FGACM, which are built and updated at runtime. Programmers prescind of traditional access modes of CLI and start using the ones dynamically implemented and updated. DACA comprises three main components: Policy Server (repository of metadata for FGACM), Dynamic Access Control Component (DACC) (business tier component responsible for implementing FGACM) and Policy Manager (broker between DACC and Policy Server). Unlike current approaches, DACA is not dependent on any particular access control model or on any access control policy, this way promoting its applicability to a wide range of different situations. In order to validate DACA, a solution based on Java, Java Database Connectivity (JDBC) and SQL Server was devised and implemented. Two evaluations were carried out. The first one evaluates DACA capability to implement and update FGACM dynamically, at runtime, and, the second one assesses DACA performance against a standard use of JDBC without any FGACM. The collected results show that DACA is an effective approach for implementing evolving FGACM on business tiers based on Call Level Interfaces, in this case JDBC.Controlo de acesso é um desafio para a engenharia de software nas aplicações de bases de dados. Atualmente, não há uma solução satisfatória para a implementação dinâmica de mecanismos finos e evolutivos de controlo de acesso (FGACM) ao nível das camadas de negócio de aplicações de bases de dados relacionais. Para solucionar esta lacuna, propomos uma arquitetura, aqui referida como Arquitetura Dinâmica de Controlo de Acesso (DACA). DACA permite que FGACM sejam dinamicamente construídos e atualizados em tempo de execução de acordo com as políticas finas de controlo de acesso (FGACP) estabelecidas. DACA explora e utiliza as características das Call Level Interfaces (CLI) para implementar FGACM ao nível das camadas de negócio. De entre as características das CLI, destacamos o seu desempenho e os diversos modos para acesso a dados armazenados em bases de dados relacionais. Na DACA, os diversos modos de acesso das CLI são envolvidos por objetos tipados derivados de FGACM, que são construídos e atualizados em tempo de execução. Os programadores prescindem dos modos tradicionais de acesso das CLI e passam a utilizar os dinamicamente construídos e atualizados. DACA compreende três componentes principais: Policy Server (repositório de meta-data dos FGACM), Dynamic Access Control Component (componente da camada de negócio que é responsável pela implementação dos FGACM) e Policy Manager (broker entre DACC e Policy Server). Ao contrário das soluções atuais, DACA não é dependente de qualquer modelo de controlo de acesso ou de qualquer política de controlo de acesso, promovendo assim a sua aplicabilidade a muitas e diversificadas situações. Com o intuito de validar DACA, foi concebida e desenvolvida uma solução baseada em Java, Java Database Connectivity (JDBC) e SQL Server. Foram efetuadas duas avaliações. A primeira avalia DACA quanto à sua capacidade para dinamicamente, em tempo de execução, implementar e atualizar FGACM e, a segunda, avalia o desempenho de DACA contra uma solução sem FGACM que utiliza o JDBC normalizado. Os resultados recolhidos mostram que DACA é uma solução válida para implementar FGACM evolutivos em camadas de negócio baseadas em CLI

Model Transformation Languages with Modular Information Hiding

Model transformations, together with models, form the principal artifacts in model-driven software development. Industrial practitioners report that transformations on larger models quickly get sufficiently large and complex themselves. To alleviate entailed maintenance efforts, this thesis presents a modularity concept with explicit interfaces, complemented by software visualization and clustering techniques. All three approaches are tailored to the specific needs of the transformation domain

From XML to relational database.

by Yan, Men-Hin.Thesis (M.Phil.)--Chinese University of Hong Kong, 2001.Includes bibliographical references (leaves 114-119).Abstracts in English and Chinese.Abstract --- p.iiAcknowledgments --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Storing XML in Database Systems --- p.2Chapter 1.2 --- Outline of the Thesis --- p.4Chapter 2 --- Related Work --- p.5Chapter 2.1 --- Overview of XML --- p.5Chapter 2.1.1 --- Extensible Markup Language (XML) --- p.5Chapter 2.1.2 --- Data Type Definition (DTD) --- p.6Chapter 2.1.3 --- "ID, IDREF and IDREFS" --- p.9Chapter 2.2 --- Using Special-Purpose Database to Store XML Data --- p.10Chapter 2.3 --- Using Relational Databases to Store XML Data --- p.11Chapter 2.3.1 --- Extracting Schemas with STORED --- p.11Chapter 2.3.2 --- Using Simple Schemes Based on Labeled Graph --- p.12Chapter 2.3.3 --- Generating Schemas from DTDs --- p.12Chapter 2.3.4 --- Commercial Approaches --- p.13Chapter 2.4 --- Discovering Functional Dependencies --- p.14Chapter 2.4.1 --- Functional Dependency --- p.14Chapter 2.4.2 --- Finding Functional Dependencies --- p.14Chapter 2.4.3 --- TANE and Partition Refinement --- p.15Chapter 2.5 --- Multivalued Dependencies --- p.17Chapter 2.5.1 --- Example of Multivalued Dependency --- p.18Chapter 3 --- Using RDBMS to Store XML Data --- p.20Chapter 3.1 --- Global Schema Extraction Algorithm --- p.22Chapter 3.1.1 --- Step 1: Simplify DTD --- p.22Chapter 3.1.2 --- Step 2: Construct Schema Prototype Trees --- p.24Chapter 3.1.3 --- Step 3: Generate Relational Schema Prototype --- p.29Chapter 3.1.4 --- Step 4: Discover Functional Dependencies and Candidate Keys --- p.31Chapter 3.1.5 --- Step 5: Normalize the Relational Schema Prototypes --- p.32Chapter 3.1.6 --- Discussion --- p.32Chapter 3.2 --- DTD-splitting Schema Extraction Algorithm --- p.34Chapter 3.2.1 --- Step 1: Simplify DTD --- p.35Chapter 3.2.2 --- Step 2: Construct Schema Prototype Trees --- p.36Chapter 3.2.3 --- Step 3: Generate Relational Schema Prototype --- p.45Chapter 3.2.4 --- Step 4: Discover Functional Dependencies and Candidate Keys --- p.46Chapter 3.2.5 --- Step 5: Normalize the Relational Schema Prototypes --- p.47Chapter 3.2.6 --- Discussion --- p.49Chapter 3.3 --- Experimental Results --- p.50Chapter 3.3.1 --- Real Life XML Data: SIGMOD Record XML --- p.50Chapter 3.3.2 --- Synthetic XML Data --- p.58Chapter 3.3.3 --- Discussion --- p.68Chapter 4 --- Finding Multivalued Dependencies --- p.75Chapter 4.1 --- Validation of Multivalued Dependencies --- p.77Chapter 4.2 --- Search Strategy and Pruning --- p.80Chapter 4.2.1 --- Search Strategy for Left-hand Sides Candidates --- p.81Chapter 4.2.2 --- Search Strategy for Right-hand Sides Candidates --- p.82Chapter 4.2.3 --- Other Pruning --- p.85Chapter 4.3 --- Computing with Partitions --- p.87Chapter 4.3.1 --- Computing Partitions --- p.88Chapter 4.4 --- Algorithm --- p.89Chapter 4.4.1 --- Generating Next Level Candidates --- p.92Chapter 4.4.2 --- Computing Partitions --- p.93Chapter 4.5 --- Experimental Results --- p.94Chapter 4.5.1 --- Results of the Algorithm --- p.95Chapter 4.5.2 --- Evaluation on the Results --- p.96Chapter 4.5.3 --- Scalability of the Algorithm --- p.98Chapter 4.5.4 --- Using Multivalued Dependencies in Schema Extraction Al- gorithms --- p.101Chapter 5 --- Conclusion --- p.108Chapter 5.1 --- Discussion --- p.108Chapter 5.2 --- Future Work --- p.110Chapter 5.2.1 --- Translate Semistructured Queries to SQL --- p.110Chapter 5.2.2 --- Improve the Multivalued Dependency Discovery Algorithm --- p.112Chapter 5.2.3 --- Incremental Update of Resulting Schema --- p.113Bibliography --- p.113Appendix --- p.120Chapter A --- Simple Proof for Minimality in Multivalued Dependencies --- p.120Chapter B --- Third and Fourth Normal Form Decompositions --- p.122Chapter B.1 --- 3NF Decomposition Algorithm --- p.123Chapter B.2 --- 4NF Decomposition Algorithm --- p.12

Tackling the veracity and variety of big data

This thesis tackles the veracity and variety challenges of big data, especially focusing on graphs and relational data. We start with proposing a class of graph association rules (GARs) to specify regularities between entities in graphs, which capture both missing links and inconsistencies. A GAR is a combination of a graph pattern and a dependency; it may take as predicates machine learning classifiers for link prediction. We formalize association deduction with GARs in terms of the chase, and prove its Church-Rosser property. We show that the satisfiability, implication and association deduction problems for GARs are coNP-complete, NP-complete and NP-complete, respectively. The incremental deduction problem is DP-complete for GARs. In addition, we provide parallel algorithms for association deduction and incremental deduction. We next develop a parallel algorithm to discover GARs, which applies an applicationdriven strategy to cut back rules and data that are irrelevant to users’ interest, by training a machine learning model to identify data pertaining to a given application. Moreover, we introduce a sampling method to reduce a big graph G to a set H of small sample graphs. Given expected support and recall bounds, this method is able to deduce samples in H and mine rules from H to satisfy the bounds in the entire G. Then we propose a class of temporal association rules (TACOs) for event prediction in temporal graphs. TACOs are defined on temporal graphs in terms of change patterns and (temporal) conditions, and may carry machine learning predicates for temporal event prediction. We settle the complexity of reasoning about TACOs, including their satisfiability, implication and prediction problems. We develop a system that discovers TACOs by iteratively training a rule creator based on generative models in a creatorcritic framework, and predicts events by applying the discovered TACOs in parallel. Finally, we propose an approach to querying relations D and graphs G taken together in SQL. The key idea is that if a tuple t in D and a vertex v in G are determined to refer to the same real-world entity, then we join t and v, correlate their information and complement tuple t with additional attributes of v from graphs. We show how to do this in SQL extended with only syntactic sugar, for both static joins when t is a tuple in D and dynamic joins when t comes from intermediate results of sub-queries on D. To support the semantic joins, we propose an attribute extraction scheme based on Kmeans clustering, to identify and fetch graph properties that are linked to v via paths. Moreover, we develop a scheme to extract a relation schema for entities in graphs, and a heuristic join method based on the schema to strike a balance between the complexity and accuracy of dynamic joins

Reinventing the Social Scientist and Humanist in the Era of Big Data

This book explores the big data evolution by interrogating the notion that big data is a disruptive innovation that appears to be challenging existing epistemologies in the humanities and social sciences. Exploring various (controversial) facets of big data such as ethics, data power, and data justice, the book attempts to clarify the trajectory of the epistemology of (big) data-driven science in the humanities and social sciences

Heterogeneous verification of model transformations

Esta tesis trata sobre la verificación formal en el contexto de la Ingeniería Dirigida por Modelos (MDE por sus siglas en inglés). El paradigma propone un ciclo de vida de la ingeniería de software basado en una abstracción de su complejidad a través de la definición de modelos y en un proceso de construcción (semi)automático guiado por transformaciones de estos modelos. Nuestro propósito es abordar la verificación de transformaciones de modelos la cual incluye, por extensión, la verificación de sus modelos. Comenzamos analizando la literatura relacionada con la verificación de transformaciones de modelos para concluir que la heterogeneidad de las propiedades que interesa verificar y de los enfoques para hacerlo, sugiere la necesidad de utilizar diversos dominios lógicos, lo cual es la base de nuestra propuesta. En algunos casos puede ser necesario realizar una verificación heterogénea, es decir, utilizar diferentes formalismos para la verificación de cada una de las partes del problema completo. Además, es beneficioso permitir a los expertos formales elegir el dominio en el que se encuentran más capacitados para llevar a cabo una prueba formal. El principal problema reside en que el mantenimiento de múltiples representaciones formales de los elementos de MDE en diferentes dominios lógicos, puede ser costoso si no existe soporte automático o una relación formal clara entre estas representaciones. Motivados por esto, definimos un entorno unificado que permite la verificación formal transformaciones de modelos mediante el uso de métodos de verificación heterogéneos, de forma tal que es posible automatizar la traducción formal de los elementos de MDE entre dominios logicos. Nos basamos formalmente en la Teoría de Instituciones, la cual proporciona una base sólida para la representación de los elementos de MDE (a través de instituciones) sin depender de ningúningún dominio lógico específico. También proporciona una forma de especificar traducciones (a través de comorfismos) que preservan la semántica entre estos elementos y otros dominios lógicos. Nos basamos en estándares para la especificación de los elementos de MDE. De hecho, definimos una institución para la buena formación de los modelos especificada con una versión simplificada del MetaObject Facility y otra institución para transformaciones utilizando Query/View/Transformation Relations. No obstante, la idea puede ser generalizada a otros enfoques de transformación y lenguajes.Por último, demostramos la viabilidad del entorno mediante el desarrollo de un prototipo funcional soportado por el Heterogeneous Tool Set (HETS). HETS permite realizar una especificación heterogénea y provee facilidades para el monitoreo de su corrección global. Los elementos de MDE se conectan con otras lógicas ya soportadas en HETS (por ejemplo: lógica de primer orden, lógica modal, entre otras) a través del Common Algebraic Specification Language (CASL). Esta conexión se expresa teóricamente mediante comorfismos desde las instituciones de MDE a la institución subyacente en CASL. Finalmente, discutimos las principales contribuciones de la tesis. Esto deriva en futuras líneas de investigación que contribuyen a la adopción de métodos formales para la verificación en el contexto de MDE.This thesis is about formal verification in the context of the Model-Driven Engineering (MDE) paradigm. The paradigm proposes a software engineering life-cycle based on an abstraction from its complexity by defining models, and on a (semi)automatic construction process driven by model transformations. Our purpose is to address the verification of model transformations which includes, by extension, the verification of their models. We first review the literature on the verification of model transformations to conclude that the heterogeneity we find in the properties of interest to verify, and in the verification approaches, suggests the need of using different logical domains, which is the base of our proposal. In some cases it can be necessary to perform a heterogeneous verification, i.e. using different formalisms for the verification of each part of the whole problem. Moreover, it is useful to allow formal experts to choose the domain in which they are more skilled to address a formal proof. The main problem is that the maintenance of multiple formal representations of the MDE elements in different logical domains, can be expensive if there is no automated assistance or a clear formal relation between these representations. Motivated by this, we define a unified environment that allows formal verification of model transformations using heterogeneous verification approaches, in such a way that the formal translations of the MDE elements between logical domains can be automated. We formally base the environment on the Theory of Institutions, which provides a sound basis for representing MDE elements (as so called institutions) without depending on any specific logical domain. It also provides a way for specifying semantic-preserving translations (as so called comorphisms) from these elements to other logical domains. We use standards for the specification of the MDE elements. In fact, we define an institution for the well-formedness of models specified with a simplified version of the MetaObject Facility, and another institution for Query/View/Transformation Relations transformations. However, the idea can be generalized to other transformation approaches and languages. Finally, we evidence the feasibility of the environment by the development of a functional prototype supported by the Heterogeneous Tool Set (HETS). HETS supports heterogeneous specifications and provides capabilities for monitoring their overall correctness. The MDE elements are connected to the other logics already supported in HETS (e.g. first-order logic, modal logic, among others) through the Common Algebraic Specification Language (CASL). This connection is defined by means of comorphisms from the MDE institutions to the underlying institution of CASL. We carry out a final discussion of the main contributions of this thesis. This results in future research directions which contribute with the adoption of formal tools for the verification in the context of MDE