2,241 research outputs found
UK utility data integration: overcoming schematic heterogeneity
In this paper we discuss syntactic, semantic and schematic issues which inhibit the integration of utility data in the UK. We then focus on the techniques employed within the VISTA project to overcome schematic heterogeneity. A Global
Schema based architecture is employed. Although automated approaches to Global Schema definition were attempted
the heterogeneities of the sector were too great. A manual approach to Global Schema definition was employed. The
techniques used to define and subsequently map source utility data models to this schema are discussed in detail. In order to ensure a coherent integrated model, sub and cross domain validation issues are then highlighted. Finally the proposed framework and data flow for schematic integration is introduced
bdbms -- A Database Management System for Biological Data
Biologists are increasingly using databases for storing and managing their
data. Biological databases typically consist of a mixture of raw data,
metadata, sequences, annotations, and related data obtained from various
sources. Current database technology lacks several functionalities that are
needed by biological databases. In this paper, we introduce bdbms, an
extensible prototype database management system for supporting biological data.
bdbms extends the functionalities of current DBMSs to include: (1) Annotation
and provenance management including storage, indexing, manipulation, and
querying of annotation and provenance as first class objects in bdbms, (2)
Local dependency tracking to track the dependencies and derivations among data
items, (3) Update authorization to support data curation via content-based
authorization, in contrast to identity-based authorization, and (4) New access
methods and their supporting operators that support pattern matching on various
types of compressed biological data types. This paper presents the design of
bdbms along with the techniques proposed to support these functionalities
including an extension to SQL. We also outline some open issues in building
bdbms.Comment: This article is published under a Creative Commons License Agreement
(http://creativecommons.org/licenses/by/2.5/.) You may copy, distribute,
display, and perform the work, make derivative works and make commercial use
of the work, but, you must attribute the work to the author and CIDR 2007.
3rd Biennial Conference on Innovative Data Systems Research (CIDR) January
710, 2007, Asilomar, California, US
A Generic business rules validation system for ORACLE Applications
Picture this : You have just spent the remainder of your IT budget on a new software package for Human Resources. Despite its excellent functionality, it does not perform all of the complex validation that your old in-house-developed system did. How can you improve the standard software package given the following constraints : You cannot afford to pay the vendor for modifications to the package Modifying the package yourself is out of the question. We describe a tool designed to implement the validation of complex business rules for any ORACLE database application - without incurring any modification to the user interface. It enhances your productís standard capabilities and improves data quality as soon as data has been entered or modified. . Our initial implementations was for the ORACLE Human Resources package. Our tool consists of four independent components: A description of the data and its location, A set of rules (written in a simple pseudo-code), A generic on-line change detection system, A core engine that checks data consistency by applying the rules to relevant changes. The Validation system makes extensive use of state of the art techniques with database triggers and dynamic PL/SQL
Accurate Algorithms for Spatial Operations on the Spheroid in a Spatial Database Management System
[EN] Some of the most powerful spatial analysis software solutions (Oracle, Google Earth Engine, PostgreSQL + PostGIS, etc.) are currently performing geometric calculations directly on the ellipsoid (a quadratic surface that models the earth shape), with a double purpose: to attain a high degree of accuracy and to allow the full management of large areas of territory (countries or even continents). It is well known that both objectives are impossible to achieve by means of the traditional approach using local mathematical projections and Cartesian coordinates. This paper demonstrates in a quantitative methodological way that most of the spatial analysis software products make important deviations in calculations regarding to geodesics, being the users unaware of the magnitude of these inaccuracies, which can easily reach meters depending on the distance. This is due to the use of ellipsoid calculations in an approximate way (e.g., using a sphere instead of an ellipsoid). This paper presents the implementation of two algorithms that solve with high accuracy (less than 100 nm) and efficiently (few iterations) two basic geometric calculations on the ellipsoid that are essential to build more complex spatial operators: the intersection of two geodesics and the minimum distance from a point to a geodesic.Martínez Llario, JC.; Baselga Moreno, S.; Coll-Aliaga, E. (2021). Accurate Algorithms for Spatial Operations on the Spheroid in
a Spatial Database Management System. Applied Sciences. 11(11):1-21. https://doi.org/10.3390/app11115129121111
Development of a centralized log management system
Os registos de um sistema são uma peça crucial de qualquer sistema e fornecem
uma visão útil daquilo que este está fazendo e do que acontenceu em caso de falha.
Qualquer processo executado num sistema gera registos em algum formato.
Normalmente, estes registos ficam armazenados em memória local. À medida que os
sistemas evoluiram, o número de registos a analisar também aumentou, e, como
consequência desta evolução, surgiu a necessidade de produzir um formato de registos
uniforme, minimizando assim dependências e facilitando o processo de análise.
A ams é uma empresa que desenvolve e cria soluções no mercado dos sensores.
Com vinte e dois centros de design e três locais de fabrico, a empresa fornece os seus
serviços a mais de oito mil clientes em todo o mundo. Um centro de design está
localizado no Funchal, no qual está incluida uma equipa de engenheiros de aplicação
que planeiam e desenvolvem applicações de software para clientes internos. O processo
de desenvolvimento destes engenheiros envolve várias aplicações e programas, cada
um com o seu próprio sistema de registos.
Os registos gerados por cada aplicação são mantido em sistemas de
armazenamento distintos. Se um desenvolvedor ou administrador quiser solucionar um
problema que abrange várias aplicações, será necessário percorrer as várias localizações
onde os registos estão armazenados, colecionando-os e correlacionando-os de forma a
melhor entender o problema. Este processo é cansativo e, se o ambiente for
dimensionado automaticamente, a solução de problemas semelhantes torna-se
inconcebível.
Este projeto teve como principal objetivo resolver estes problemas, criando
assim um Sistema de Gestão de Registos Centralizado capaz de lidar com registos de
várias fontes, como também fornecer serviços que irão ajudar os desenvolvedores e
administradores a melhor entender os diferentes ambientes afetados.
A solução final foi desenvolvida utilizando um conjunto de diferentes tecnologias
de código aberto, tais como a Elastic Stack (Elasticsearch, Logstash e Kibana), Node.js,
GraphQL e Cassandra.
O presente documento descreve o processo e as decisões tomadas para chegar
à solução apresentada.Logs are a crucial piece of any system and give a helpful insight into what it is
doing as well as what happened in case of failure. Every process running on a system
generates logs in some format. Generally, these logs are written to local storage
resources. As systems evolved, the number of logs to analyze increased, and, as a
consequence of this progress, there was the need of having a standardized log format,
minimizing dependencies and making the analysis process easier.
ams is a company that develops and creates sensor solutions. With twenty-two
design centers and three manufacturing locations, the company serves to over eight
thousand clients worldwide. One design center is located in Funchal, which includes a
team of application engineers that design and develop software applications to clients
inside the company. The application engineer’s development process is comprised of
several applications and programs, each having its own logging system.
Log entries generated by different applications are kept in separate storage
systems. If a developer or administrator wants to troubleshoot an issue that includes
several applications, he/she would have to go to different database systems or locations
to collect the logs and correlate them across the several requests. This is a tiresome
process and if the environment is auto-scaled, then troubleshooting an issue is
inconceivable.
This project aimed to solve these problems by creating a Centralized Log
Management System that was capable of handling logs from a variety of sources, as well
as to provide services that will help developers and administrators better understand
the different affected environments.
The deployed solution was developed using a set of different open-source
technologies, such as the Elastic Stack (Elasticsearch, Logstash and Kibana), Node.js,
GraphQL and Cassandra.
The present document describes the process and decisions taken to achieve the
solution
Selection of heterogeneous test environments for the execution of automated tests
As software complexity grows so does the size of automated test suites that enable us to validate
the expected behavior of the system under test. When that occurs, problems emerge for
developers in the form of increased effort to manage the test process and longer execution time
of test suites.
Manual managing automated tests is especially problematic, as the recurring costa of guaranteeing
that the automated tests (e.g.: thousands) are correctly configured to execute on
the available test environments (e.g.: dozens or hundreds), on a regular basis and during the
products lifetime may become huge, with unbearable human effort involved. This problem increases
substantially when the system under test is one highly configurable product, requiring
to be validated in heterogeneous environments, especially when these target test environments
also evolve frequently (e.g.: new operating systems, new browsers, new mobile devices, ...).
Being an integral part of software development, testing needs to evolve and break free from
the conventional methods. This dissertation presents a technique that extends one existent
algorithm to reduce the number of test executions, and extend it, enabling to perform the test
case distribution over multiples heterogeneous test environments.
The development, implementation and validation of the technique presented in this dissertation
were conducted in the industrial context of an international software house. Real development
scenarios were used to conduct experiments and validations, and the results demonstrated that
the proposed technique is effective in terms of eliminating the human effort involved in test
distribution.À medida que a complexidade do software aumenta o mesmo acontece com a dimensão das
suites de testes automizados que permitem validar o comportamento esperado do sistema que
está a ser testado. Quando isso ocorre, aparecem problemas para os programadores sob a forma
de aumento de esforço necessário para gerir o processo de teste e maior tempo de execução
das suites de teste.
Gerir manualmente milhares de testes automatizados é especialmente problemático uma vez
que os custos recorrentes de garantir que os testes automatizados (ex: milhares) estão corretamente
configurados para executar nos ambientes de testes disponíveis (ex: dezenas ou
centennas), durante o tempo de vida dos produtos pode tornar-se gigantesco. Este problema
aumenta substancialmente quando o sistema que está a ser testado é um produto altamente
configurável, precisando de ser validado em ambientes heterogéneos, especialmente quando
também estes ambientes destino de testes também evoluem frequentemente (ex: novos sistemas
operativos, novos browsers, novos devices móveis, ...). O tempo de execução destas
suites de testes torna-se também um problema enorme, dado que não é viável executar todos
as suites de testes em todas as configurações possiveis.
Sendo uma parte integral do desenvolvimento de software, a forma de testar precisa de evoluir
e libertar-se dos métodos convencionais. Esta dissertação apresenta uma técnica que estende
um algoritmo existente que permite reduzir o número de execuções de testes, e estende-o, permitindo
fazer a distribuição de casos de teste sobre múltiplos ambientes de teste heterogéneos.
O desenvolvimento, implementação e validação da técnica proposta na presente dissertação
foram conduzidos no contexto industrial de uma empresa internacional de desenvolvimento
de software. Foram utilizados cenários de desenvolvimento de software reais para conduzir
experiências e validações, e os resultados demonstraram que a técnica proposta é eficaz em
termos de eliminar o esforo humano envolvido na distribuição de testes
- …