Automatic generation of data merging program codes.

Data merging is an essential part of ETL (Extract-Transform-Load) processes to build a data warehouse system. To avoid rewheeling merging techniques, we propose a Data Merging Meta-model (DMM) and its transformation into executable program codes in the manner of model driven engineering. DMM allows defining relationships of different model entities and their merging types in conceptual level. Our formalized transformation described using ATL (ATLAS Transformation Language) enables automatic generation of PL/SQL packages to execute data merging in commercial ETL tools. With this approach data warehouse engineers can be relieved from the burden of repetitive complex script coding and the pain of maintaining consistency of design and implementation

Automatic generation of data merging program codes

Data merging is an essential part of ETL (Extract-Transform-Load) processes to build a data warehouse system. To avoid rewheeling merging techniques, we propose a Data Merging Meta-model (DMM) and its transformation into executable program codes in the manner of model driven engineering. DMM allows defining relationships of different model entities and their merging types in conceptual level. Our formalized transformation described using ATL (ATLAS Transformation Language) enables automatic generation of PL/SQL packages to execute data merging in commercial ETL tools. With this approach data warehouse engineers can be relieved from the burden of repetitive complex script coding and the pain of maintaining consistency of design and implementation

A case study on model driven data integration for data centric software development.

Model Driven Data Integration is a data integration approach that proactively incorporates and utilizes metadata across the data integration process. By decoupling data and metadata, MDDI drastically reduces complexity of data integration; whilst also providing an integrated standard development method, which is associated with Model Driven Architecture. This paper introduces a case study to adopt MDA technology as an MDDI framework for data centric software development; including data merging and data customization for data mining. A data merging model is also proposed to define relationships between different models at a conceptual level which is then transformed into a physical model. In this case study we collect and integrate historical data from various universities into the Data Warehouse system in order to develop student intervention services through data mining

Managing Metadata in Data Warehouses: Pitfalls and Possibilities

This paper motivates a comprehensive academic study of metadata and the roles that metadata plays in organizational information systems. While the benefits of metadata and challenges in implementing metadata solutions are widely addressed in practitioner publications, explicit discussion of metadata in academic literature is rare. Metadata, when discussed, is perceived primarily as a technology solution. Integrated management of metadata and its business value are not well addressed. This paper discusses both the benefits offered by and the challenges associated with integrating metadata. It also describes solutions for addressing some of these challenges. The inherent complexity of an integrated metadata repository is demonstrated by reviewing the metadata functionality required in a data warehouse: a decision support environment where its importance is acknowledged. Comparing this required functionality with metadata management functionalities offered by data warehousing software products identifies crucial gaps. Based on these analyses, topics for further research on metadata are proposed

Showing the Benefits of Applying a Model Driven Architecture for Developing Secure OLAP Applications

Data Warehouses (DW) manage enterprise information that is queried for decision making purposes by using On-Line Analytical Processing (OLAP) tools. The establishment of security constraints in all development stages and operations of the DW is highly important since otherwise, unauthorized users may discover vital business information. The final users of OLAP tools access and analyze the information from the corporate DW by using specific views or cubes based on the multidimensional modelling containing the facts and dimensions (with the corresponding classification hierarchies) that a decision maker or group of decision makers are interested in. Thus, it is important that security constraints will be also established over this metadata layer that connects the DW's repository with the decision makers, that is, directly over the multidimensional structures that final users manage. In doing so, we will not have to define specific security constraints for every particular user, thereby reducing the developing time and costs for secure OLAP applications. In order to achieve this goal, a model driven architecture to automatically develop secure OLAP applications from models has been defined. This paper shows the benefits of this architecture by applying it to a case study in which an OLAP application for an airport DW is automatically developed from models. The architecture is composed of: (1) the secure conceptual modelling by using a UML profile; (2) the secure logical modelling for OLAP applications by using an extension of CWM; (3) the secure implementation into a specific OLAP tool, SQL Server Analysis Services (SSAS); and (4) the transformations needed to automatically generate logical models from conceptual models and the final secure implementation.This research is part of the following projects: SERENIDAD (PEII11- 037-7035) financed by the ”Viceconsejería de Ciencia y Tecnología de la Junta de Comunidades de Castilla-La Mancha” (Spain) and FEDER, and SIGMA-CC (TIN2012-36904) and GEODAS (TIN2012-37493-C03-01) financed by the ”Ministerio de Economía y Competitividad” (Spain)

An MDA approach for developing Secure OLAP applications: metamodels and transformations

Decision makers query enterprise information stored in Data Warehouses (DW) by using tools (such as On-Line Analytical Processing (OLAP) tools) which employ specific views or cubes from the corporate DW or Data Marts, based on multidimensional modelling. Since the information managed is critical, security constraints have to be correctly established in order to avoid unauthorized access. In previous work we defined a Model-Driven based approach for developing a secure DW repository by following a relational approach. Nevertheless, it is also important to define security constraints in the metadata layer that connects the DW repository with the OLAP tools; that is, over the same multidimensional structures that end users manage. This paper incorporates a proposal for developing secure OLAP applications within our previous approach: it improves a UML profile for conceptual modelling; it defines a logical metamodel for OLAP applications; and it defines and implements transformations from conceptual to logical models, as well as from logical models to secure implementation in a specific OLAP tool (SQL Server Analysis Services).This research is part of the following projects: SIGMA-CC (TIN2012-36904), GEODAS-BC (TIN2012-37493-C01) and GEODAS-BI (TIN2012-37493-C03) funded by the Ministerio de Economía y Competitividad and Fondo Europeo de Desarrollo Regional FEDER. SERENIDAD (PEII11-037-7035) and MOTERO (PEII11- 0399-9449) funded by the Consejería de Educación, Ciencia y Cultura de la Junta de Comunidades de Castilla La Mancha, and Fondo Europeo de Desarrollo Regional FEDER

Applying the UML and the Unified Process to the Design of Data Warehouses

The design, development and deployment of a data warehouse (DW) is a complex, time consuming and prone to fail task. This is mainly due to the different aspects taking part in a DW architecture such as data sources, processes responsible for Extracting, Transforming and Loading (ETL) data into the DW, the modeling of the DW itself, specifying data marts from the data warehouse or designing end user tools. In the last years, different models, methods and techniques have been proposed to provide partial solutions to cover the different aspects of a data warehouse. Nevertheless, none of these proposals addresses the whole development process of a data warehouse in an integrated and coherent manner providing the same notation for the modeling of the different parts of a DW. In this paper, we propose a data warehouse development method, based on the Unified Modeling Language (UML) and the Unified Process (UP), which addresses the design and development of both the data warehouse back-stage and front-end. We use the extension mechanisms (stereotypes, tagged values and constraints) provided by the UML and we properly extend it in order to accurately model the different parts of a data warehouse (such as the modeling of the data sources, ETL processes or the modeling of the DW itself) by using the same notation. To the best of our knowledge, our proposal provides a seamless method for developing data warehouses. Finally, we apply our approach to a case study to show its benefit.This work has been partially supported by the METASIGN project (TIN2004-OO779) from the Spanish Ministry of Education and Science, by the DADASMECA project (GV05/220) from the Valencia Government, and by the DADS (PBC-05-QI 2-2) project from the Regional Science arid Technology Ministry of CastiIla-La Mancha (Spain)

An MDA approach for developing secure OLAP applications: Metamodels and transformations

Decision makers query enterprise information stored in DataWarehouses (DW) by using tools (such as On-Line Analytical Processing (OLAP) tools) which employ specific views or cubes from the corporate DW or Data Marts, based on multidimensional modelling. Since the information managed is critical, security constraints have to be correctly established in order to avoid unauthorized access. In previous work we defined a Model-Driven based approach for developing a secure DW repository by following a relational approach. Nevertheless, it is also important to define security constraints in the metadata layer that connects the DW repository with the OLAP tools; that is, over the same multidimensional structures that end users manage. This paper incorporates a proposal for developing secure OLAP applications within our previous approach: it improves a UML profile for conceptual modelling; it defines a logical metamodel for OLAP applications; and it defines and implements transformations from conceptual to logical models, as well as from logical models to secure implementation in a specific OLAP tool (SQL Server Analysis Services). © 2015 ComSIS Consortium. All rights reserved.This research is part of the following projects: SIGMA-CC (TIN2012-36904), GEODAS-BC (TIN2012-37493-C01) and GEODAS-BI (TIN2012-37493-C03) funded by the Ministerio de Economía y Competitividad and Fondo Europeo de Desarrollo Regional FEDER