Managing polyglot systems metadata with hypergraphs

A single type of data store can hardly fulfill every end-user requirements in the NoSQL world. Therefore, polyglot systems use different types of NoSQL datastores in combination. However, the heterogeneity of the data storage models makes managing the metadata a complex task in such systems, with only a handful of research carried out to address this. In this paper, we propose a hypergraph-based approach for representing the catalog of metadata in a polyglot system. Taking an existing common programming interface to NoSQL systems, we extend and formalize it as hypergraphs for managing metadata. Then, we define design constraints and query transformation rules for three representative data store types. Furthermore, we propose a simple query rewriting algorithm using the catalog itself for these data store types and provide a prototype implementation. Finally, we show the feasibility of our approach on a use case of an existing polyglot system.Peer ReviewedPostprint (author's final draft

New Perspectives for NoSQL Database Design: A Systematic Review

The use of NoSQL databases has increasingly become a trend in software development, mainly due to the expansion of Web 2.0 systems. However, there is not yet a standard to be used for the design of this type of database even with the growing number of studies related to this subject. This paper presents a systematic review looking for new trends regarding strategies used in this context. The result of this process demonstrates that there are still few methodologies for the NoSQL database design and there are no design methodologies capable of working with polyglot persistence

NOSQL design for analytical workloads: Variability matters

Big Data has recently gained popularity and has strongly questioned relational databases as universal storage systems, especially in the presence of analytical workloads. As result, co-relational alternatives, commonly known as NOSQL (Not Only SQL) databases, are extensively used for Big Data. As the primary focus of NOSQL is on performance, NOSQL databases are directly designed at the physical level, and consequently the resulting schema is tailored to the dataset and access patterns of the problem in hand. However, we believe that NOSQL design can also benefit from traditional design approaches. In this paper we present a method to design databases for analytical workloads. Starting from the conceptual model and adopting the classical 3-phase design used for relational databases, we propose a novel design method considering the new features brought by NOSQL and encompassing relational and co-relational design altogether.Peer ReviewedPostprint (author's final draft

Rancang bangun aplikasi pemesanan produk olahan makanan umkm kabupaten pati berbasis android menggunakan layanan mobile backend as a service

Perkembangan teknologi dari tahun ke tahun dirasa sangat pesat, khususnya teknologi mobile. International Data Corporation (IDC) menyebutkan bahwa teknologi mobile merupakan salah satu teknologi yang berkembang sangat pesat. Penggunaan teknologi mobile digunakan hampir seluruh kegiatan manusia sehari-hari untuk mempermudah aktivitas. Teknologi mobile dinilai lebih ringkas ketimbang teknologi yang lainnya, kita dapat mengaksesnya di mana pun dan kapan pun kita perlukan. Penelitian ini mengangkat tema aplikasi pemesanan produk olahan makanan Usaha, Mikro, Kecil dan Menengah (UMKM) Kabupaten Pati guna mempermudah konsumen untuk melakukan pemesanan produk dan mempermudah pelaku UMKM untuk mempromosikan produknya. Penelitian ini menggunakan metode pengembangan sistem waterfall yang mudah dilakukan karena memiliki tahapan-tahapan dalam pengembangan sistem. Pada pengembangan aplikasi, penelitian ini menggunakan bahasa pemrograman Java untuk membangun aplikasi mobile, bahasa pemrograman Javascript untuk pengembangan aplikasi web, dan menggunakan salah satu layer teknologi cloud, yaitu layanan Backend Mobile as a Service (MBaaS). Penggunaan layanan MBaaS dapat mempermudah pengembangan aplikasi dari sisi backend

Test model for database architectures: an assessment for job search engine systems

Information systems are increasingly complex structures due to the diversity of processes involved and the big data generated, hence data management is essential. NoSQL databases adopt new approaches to data management differing from relational structures. In this study, three databases were designed, a relational database using PostgreSQL and two NoSQL databases made in MongoDB applied to operation of a job offer system, with the aim of comparing its operation and efficiency. A method was proposed for the metric-guided evaluation of database models using functionality and efficiency criteria according to Systems and Software Standard Quality Requirements and Evaluation (SQuaRE). Testing cases were created considering the International Software Testing Qualifications Board (ISTQB) best practices. Relational data model was selected as a pattern, for this reason, to populate NoSQL databases a reference framework was applied for data migration from one environment to another, thus the tests were performed under the same hardware, software and data conditions. This study determined that the SQL schema provides greater functionality, ensuring transaction support and data integrity. On the other hand, the NoSQL schemas are more efficient in response to big data processing, although they have a certain level of data duplication, transaction support fails and some join operations are not support

PDDM: A Database Design Method for Polyglot Persistence

Databases by Web 2.0 has revealed the limitations of the relational model related to scalability. This led to the emergence of NoSQL databases, with data storage models other than relational ones. These databases propose solutions to such limitations through horizontal scalability and partially compromise data consistency. The combination of multiple data models, called polyglot persistence, extends these solutions by providing resources for the implementation of complex systems that have components with distinct requirements that would not be possible by the use of only one data model in a satisfactory way. However, there are no consolidated methods for the NoSQL database design and neither methods for design systems that apply the polyglot persistence. This work proposes a database design method applied to systems that use polyglot persistence, combining different data models. This method can be applied to the relational model and aggregate-oriented NoSQL data models. The method defines a set of sub-steps based on the existing concepts of database design. The goal is to define a formal process to assist in defining the data models to be used and to transform the conceptual design into a logical design. The method application is demonstrated in some test cases, in order to show its results and applicability for later execution of the physical design of these databases

An ontology-based secure design framework for graph-based databases

Graph-based databases are concerned with performance and flexibility. Most of the existing approaches used to design secure NoSQL databases are limited to the final implementation stage, and do not involve the design of security and access control issues at higher abstraction levels. Ensuring security and access control for Graph-based databases is difficult, as each approach differs significantly depending on the technology employed. In this paper, we propose the first technology-ascetic framework with which to design secure Graph-based databases. Our proposal raises the abstraction level by using ontologies to simultaneously model database and security requirements together. This is supported by the TITAN framework, which facilitates the way in which both aspects are dealt with. The great advantages of our approach are, therefore, that it: allows database designers to focus on the simultaneous protection of security and data while ignoring the implementation details; facilitates the secure design and rapid migration of security rules by deriving specific security measures for each underlying technology, and enables database designers to employ ontology reasoning in order to verify whether the security rules are consistent. We show the applicability of our proposal by applying it to a case study based on a hospital data access control.This work has been developed within the AETHER-UA (PID2020-112540RB-C43), AETHER-UMA (PID2020-112540RB-C41) and AETHER-UCLM (PID2020-112540RB-C42), ALBA (TED2021-130355B-C31, TED2021-130355B-C33), PRESECREL (PID2021-124502OB-C42) projects funded by the “Ministerio de Ciencia e Innovación”, Andalusian PAIDI program with grant (P18-RT-2799) and the BALLADER Project (PROMETEO/2021/088) funded by the “Consellería de Innovación, Universidades, Ciencia Sociedad Digital”, Generalitat Valenciana

How I Learned to Stop Worrying and Love NoSQL Databases

National audienceThe absence of a schema in NoSQL databases can disorient tradi- tional database specialists and can make the design activity in this context a leap of faith. However, we show in this paper that an effective design methodology for NoSQL systems supporting the scalability, performance, and consistency of next-generation Web applications can be indeed devised. The approach is based on NoAM (NoSQL Abstract Model), a novel abstract data model for NoSQL databases, which is used to specify a system-independent representation of the application data. This intermediate representation can be then implemented in target NoSQL databases, taking into account their specific features

A digital vault solution for banking institutions

Trabalho de projecto de mestrado, Segurança Informática, Universidade de Lisboa, Faculdade de Ciências, 2019Este projeto surgiu no âmbito da necessidade que a empresa Securibox tem em fornecer um produto de armazenamento seguro compatível com o funcionamento na nuvem, para as instituições bancárias que operam no mercado francês. Com o aparecimento da banca on-line e o intuito de atrair mais clientes, as instituições bancárias começaram a oferecer serviços que vão para além dos serviços convencionais deste setor. Muitas vezes esses serviços tratam ou armazenam dados sensíveis dos seus clientes e podem até incluir informação e documentos pessoais dos utilizadores que estão hospedados noutras entidades, tais como faturas eletrónicas, transações bancárias de outras instituições financeiras e recibos de vencimento. No entanto, sempre que for necessário armazenar informação dos clientes, este processo tem de respeitar um conjunto de boas práticas e normas do país onde a instituição opera, utilizando para o efeito um cofre digital. No caso do mercado francês, existem poucas soluções que satisfazem, parcialmente ou totalmente, as normas e a legislação respeitante aos cofres digitais e que sejam tecnicamente eficientes e competitivas. O objetivo deste trabalho visou desenvolver uma versão inicial de uma solução que colmata a necessidade atual do mercado bancário francês relativo à área de armazenamento e manuseamento inteligente de dados. Para satisfazer as normas da União Europeia e da França em particular, é necessário armazenar os ficheiros de forma cifrada, registar o seu formato, como, quando e por quem estes formas acedidos e os seus meta-dados de modo a garantir a sua preservação mesmo após a eliminação dos mesmos. Este desafio foi resolvido, e para se destacar das soluções atualmente existentes, foi construída a base para no futuro integrar esta solução com o serviço Securibox ParseXtract, que tem a capacidade de analisar e extrair informação importante do conteúdo dos documentos, de uma forma estruturada e precisa, recorrendo a aprendizagem automática. Para o armazenamento dos documentos a solução adotada foi o OpenStack Swift – um software de código aberto, compatível com nuvens pública e privadas. Uma vez que os documentos podem ser eliminados do sistema pelo utilizador, é necessário a existência de uma plataforma, separada do OpenStack, para armazenar os dados relativos aos meta-dados dos documentos e acessos ao sistema. A solução encontrada para o armazenamento destes dados, consiste no seu registo, através de logs, numa base de dados não relacional – o MongoDB, que é compatível com tecnologias em nuvem e é eficiente com grandes volumes de dados. Para realizar a comunicação entre os vários componentes do cofre digital, foi criado um serviço que oferece uma REST API, o núcleo da solução. Nesta camada, os documentos são cifrados garantindo também a integridade, confidencialidade e o não-repúdio dos dados. Por último, um servidor Web que comunica com a REST API foi criado para demonstrar todas as funcionalidades do cofre digital. As principais vantagens desta solução consistem na utilização de tecnologias código aberto, na compatibilidade com o funcionamento na nuvem, na escalabilidade de todas as suas camadas, tais como o armazenamento de dados, logs e serviço web API, e numa melhor integração com outros produtos da Securibox, que deste modo reduzem o custo da solução para o cliente final. Do ponto de vista conceptual, esta solução pode ser utilizada não apenas pelo sector bancário, mas também por qualquer outra área empresarial onde é necessário armazenar grandes volumes de dados em nuvem privada e pública em simultâneo, tendo como base uma solução facilmente escalável e onde todas as ações dos seus utilizadores são rastreáveis em conformidade com a legislação.This project is a result of the Securibox need to provide a digital vault storage solution for some of their bank clients, operating in the French market. Since electronic banking has emerged, banking institutions began to provide online services that go beyond conventional bank services to attract more users. Sometimes those services involve operations with personal data of their customers which can include data and documents from other services, entities and companies. All this information must be stored on the banking institution side, using a secure digital vault storage, while respecting the legislation of the country where the institution is located. The goal of this work was to develop an initial solution, that would address the current needs of the French banking market, regarding intelligent data handling and storage. To be compliant with the European Union and the French legislation it is necessary to ensure the security and the privacy of the costumers documents and data. To address those requirements a REST API solution was developed using .Net technology. This solution is divided in 3 layers. The document storage layer, the metadata and log storage layer and the core layer. The documents are encrypted and stored at the OpenStack Swift environment, while metadata is stored at the MongoDB database as journal log entries. The information processing and the communication between OpenStack and MongoDB occurs at the core layer. This solution relies on open-source technologies, is easily scalable and compatible with other Securibox products. Conceptually it can be used, not only by banking institutions, but also by any organization or company that have to store and deal with large amounts of information

Data migration between different data models of NOSQL databases

Orientador : Marcos Didonet Del FabroDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 17/02/2017Inclui referências : f. 76-79Resumo: Desde sua origem, as bases de dados Nosql têm alcançado um uso generalizado. Devido à falta de padrões de desenvolvimento nesta nova tecnologia emergem grandes desafios. Existem modelos de dados , linguagens de acesso e frameworks heterogêneos, o que torna a migração de dados ainda mais complexa. A maior parte das soluções disponíveis hoje se concentra em fornecer uma representação abstrata e genérica para todos os modelos de dados. Essas soluções se concentram em adaptadores para acessar homogeneamente os dados, mas não para implementar especificamente transformações entre eles. Essas abordagens muitas vezes precisam de um framework para acessar os dados, o que pode impedir de usá-los em alguns cenários. Entre estes desafios, a migração de dados entre as várias soluções revelou-se particularmente difícil. Esta dissertação propõe a criação de um metamodelo e uma série de regras capazes de auxiliar na tarefa de migração de dados. Os dados podem ser convertidos para vários formatos desejados através de um estado intermediário. Para validar a solução foram realizados vários testes com diversos sistemas e utilizando dados reais disponíveis. Palavras Chave: NoSql Databases. Metamodelo. Migração de Dados.Abstract: Since its origin the NoSql Database have achieved widespread use. Due to the lack of standards for development in this new technology great challenges emerges. Among these challenges, the data migration between the various solutions has proved particularly difficult. There are heterogeneous datamodels, access languages and frameworks available, which makes data migration even more complex. Most part of the solutions available today focus on providing an abstract and generic representation for all data models. These solutions focus in design adapters to homogeneously access the data, but not to specifically implement transformations between them. These approaches often need a framework to access the data, which may prevent from using them in some scenarios. This dissertation proposes the creation of a metamodel and a series of rules capable of assisting in the data migration task. The data can be converted to various desired formats through an intermediate state. To validate the solution several tests were performed with different systems and using real data available. Key-words: NoSql Databases. Metamodel. Data Migration