Rancang Bangun Real-Time Business Intelligence Untuk Subjek Kegiatan Akademik pada Universitas Menggunakan Change Data Capture

Abstract. The running of academic activities in university continuously adds more data to the existing operational system. The data are not ready for the university strategic decision making, preparing reports for accreditation purposes and academic units. Real-time business intelligence application using data warehouse can become a solution for data analysis. The process of creating a data warehouse includes designing data warehouse, retrieving academic data from multiple data sources, extracting, transforming, loading (ETL) process, creating cube; and generating report. ETL processes are conducted by using a Pull Change Data Capture approach so that data changes during a certain period can be transferred in real-time. The higher the frequency of data change requests brings us closer to real-time and requires less time than loading all the data.Keywords: real-time, business intelligence, data warehouse, academic, change data capture Abstrak.  Kegiatan akademik di universitas berjalan terus menerus dan semakin menambah banyak data pada sistem operasional yang sudah ada. Data tersebut masih belum dapat dimanfaatkan oleh pihak universitas dalam pengambilan keputusan strategis, pembuatan laporan untuk keperluan akreditasi dan unit-unit akademik. Aplikasi real-time business intelligence menggunakan data warehouse menjadi solusi untuk analisa data. Proses pembuatan data warehouse meliputi perancangan data warehouse; pengambilan data akademik dari sumber data; proses extraction, transformation, loading (ETL); pembuatan cube; dan pembuatan laporan. Proses ETL dilakukan menggunakan pendekatan Change Data Capture Pull agar perubahan data selama periode tertentu dapat dipindahkan secara real-time. Semakin tinggi frekuensi permintaan perubahan data akan semakin mendekati real-time dan semakin membutuhkan waktu yang singkat dibandingkan dengan me-load semua data.Kata Kunci: real-time, business intelligence, data warehouse, akademik, change data captur

A Data Centric Privacy Preserved Mining Model for Business Intelligence Applications

In present day competitive scenario, the techniques such as data warehouse and on-line analytical process (OLAP) have become a very significant approach for decision support in data centric applications and industries. In fact the decision support mechanism puts certain moderately varied needs on database technology as compared to OLAP based applications. Data centric decision support schemes (DSS) like data warehouse might play a significant role in extracting details from various areas and for standardizing data throughout the organization to achieve a singular way of detail presentation. Such efficient data presentation facilitates information for decision making in business intelligence (BI) applications in various industrial services. In order to enhance the effectiveness and robust computation in BI applications, the optimization in data mining and its processing is must. On the other hand, being in a multiuser scenario, the security of data on warehouse is also a critical issue, which is not explored till date. In this paper a data centric and service oriented privacy preserved model for BI applications has been proposed. The optimization in data mining has been accomplished by means of C5.0 classification algorithm and comparative study has been done with C4.5 algorithm. The implementation of enhanced C5.0 algorithm with BI model would provide much better performance with privacy preservation facility for Business Intelligence applications

Practices that organizations employ to enhance business intelligence agility

In today's rapidly changing business environment, organizations strive to be agile in order to accommodate changes and seize opportunities. Since organizations use information system as a tool to serve their needs, it is important for these systems also to be agile. One prominent type of such systems is business intelligence, which provides organizations with information to gain and retain competitive advantage. This thesis focuses on business intelligence agility, which is widely discussed in practice however not extensively covered in information systems literature. Therefore, this thesis seeks to identify the practices employed by organizations to enhance business intelligence agility. To find the answer to the research question this thesis first compiles a theoretical framework on business intelligence, information systems agility in general and business intelligence agility in specific using academic literature and market white papers. This compiled framework is comprised of four enabling factors 1) sensing business changes, 2) development approach, 3) IT governance, and 4) technical factors. This thesis conducts a qualitative research based on semi-structured interviews with business intelligence experts. Based on analysis of the empirical data this thesis identified a set of practices organized in terms of the enabling factors. The practices in sensing business changes are enabling business staff to sense changes and incorporating business staff feedback into data requirements. Regarding development approach, this thesis identifies the practices as applying an iterative development approach, building collaborative team of skilled members, enabling a centric role of business staff, reducing use of approval documents and learning from each project. In IT governance, applying a centralized or decentralized development were the two practices. Regarding practices in technical factors, this thesis identifies integrating data through either building an enterprise-wide data warehouse or applying an appropriate modeling approach while managing multiple data warehouses, using multiple front-end applications, and adopting cloud business intelligence. The findings of this thesis provide organizations with a pool of practices that can be used to enhance business intelligence agility

An evaluation of the challenges of Multilingualism in Data Warehouse development

In this paper we discuss Business Intelligence and define what is meant by support for Multilingualism in a Business Intelligence reporting context. We identify support for Multilingualism as a challenging issue which has implications for data warehouse design and reporting performance. Data warehouses are a core component of most Business Intelligence systems and the star schema is the approach most widely used to develop data warehouses and dimensional Data Marts. We discuss the way in which Multilingualism can be supported in the Star Schema and identify that current approaches have serious limitations which include data redundancy and data manipulation, performance and maintenance issues. We propose a new approach to enable the optimal application of multilingualism in Business Intelligence. The proposed approach was found to produce satisfactory results when used in a proof-of-concept environment. Future work will include testing the approach in an enterprise environmen

Instant-on scientific data warehouses: Lazy ETL for data-intensive research

In the dawning era of data intensive research, scientific discovery deploys data analysis techniques similar to those that drive business intelligence. Similar to classical Extract, Transform and Load (ETL) processes, data is loaded entirely from external data sources (repositories) into a scientific data warehouse before it can be analyzed. This process is both, time and resource intensive and may not be entirely necessary if only a subset of the data is of interest to a particular user. To overcome this problem, we propose a novel technique to lower the costs for data loading: Lazy ETL. Data is extracted and loaded transparently on-the-fly only for the required data items. Extensive experiments demonstrate the significant reduction of the time from source data availability to query answer compared to state-of-the-art solutions. In addition to reducing the costs for bootstrapping a scientific data warehouse, our approach also reduces the costs for loading new incoming data

A systems thinking approach to business intelligence solutions based on cloud computing

Thesis (S.M. in System Design and Management)--Massachusetts Institute of Technology, Engineering Systems Division, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 73-74).Business intelligence is the set of tools, processes, practices and people that are used to take advantage of information to support decision making in the organizations. Cloud computing is a new paradigm for offering computing resources that work on demand, are scalable and are charged by the time they are used. Organizations can save large amounts of money and effort using this approach. This document identifies the main challenges companies encounter while working on business intelligence applications in the cloud, such as security, availability, performance, integration, regulatory issues, and constraints on network bandwidth. All these challenges are addressed with a systems thinking approach, and several solutions are offered that can be applied according to the organization's needs. An evaluations of the main vendors of cloud computing technology is presented, so that business intelligence developers identify the available tools and companies they can depend on to migrate or build applications in the cloud. It is demonstrated how business intelligence applications can increase their availability with a cloud computing approach, by decreasing the mean time to recovery (handled by the cloud service provider) and increasing the mean time to failure (achieved by the introduction of more redundancy on the hardware). Innovative mechanisms are discussed in order to improve cloud applications, such as private, public and hybrid clouds, column-oriented databases, in-memory databases and the Data Warehouse 2.0 architecture. Finally, it is shown how the project management for a business intelligence application can be facilitated with a cloud computing approach. Design structure matrices are dramatically simplified by avoiding unnecessary iterations while sizing, validating, and testing hardware and software resources.by Eumir P. Reyes.S.M.in System Design and Managemen

Business Intelligence Approaches

Business Intelligence (BI) is unanimous considered the art of gaining business advantage from data; therefore BI systems and infrastructures must integrate disparate data sources into a single coherent framework for real-time reporting and detailed analysis within the extended enterprise. Also the solution to a business problem is a process that includes business intelligence, BI, by itself, is rarely the complete solution to the problem. Therefore, BI tools must understand the process and how to be part of it. Subordinated to performance management, Business Intelligence approaches help firms to optimize business performance. Looking inside the business and at the environment in which they operate, managers are able to fundament the most productive and profitable decisions. The new trend of social BI in business analysis comes with an innovative approach in consolidating performance management. A data warehouse schema for social BI will be a good start for future debates

Desenho e implementação de um data warehouse para a empresa AdClick

Esta dissertação incide sobre a problemática da construção de um data warehouse para a empresa AdClick que opera na área de marketing digital. O marketing digital é um tipo de marketing que utiliza os meios de comunicação digital, com a mesma finalidade do método tradicional que se traduz na divulgação de bens, negócios e serviços e a angariação de novos clientes. Existem diversas estratégias de marketing digital tendo em vista atingir tais objetivos, destacando-se o tráfego orgânico e tráfego pago. Onde o tráfego orgânico é caracterizado pelo desenvolvimento de ações de marketing que não envolvem quaisquer custos inerentes à divulgação e/ou angariação de potenciais clientes. Por sua vez o tráfego pago manifesta-se pela necessidade de investimento em campanhas capazes de impulsionar e atrair novos clientes. Inicialmente é feita uma abordagem do estado da arte sobre business intelligence e data warehousing, e apresentadas as suas principais vantagens as empresas. Os sistemas business intelligence são necessários, porque atualmente as empresas detêm elevados volumes de dados ricos em informação, que só serão devidamente explorados fazendo uso das potencialidades destes sistemas. Nesse sentido, o primeiro passo no desenvolvimento de um sistema business intelligence é concentrar todos os dados num sistema único integrado e capaz de dar apoio na tomada de decisões. É então aqui que encontramos a construção do data warehouse como o sistema único e ideal para este tipo de requisitos. Nesta dissertação foi elaborado o levantamento das fontes de dados que irão abastecer o data warehouse e iniciada a contextualização dos processos de negócio existentes na empresa. Após este momento deu-se início à construção do data warehouse, criação das dimensões e tabelas de factos e definição dos processos de extração e carregamento dos dados para o data warehouse. Assim como a criação das diversas views. Relativamente ao impacto que esta dissertação atingiu destacam-se as diversas vantagem a nível empresarial que a empresa parceira neste trabalho retira com a implementação do data warehouse e os processos de ETL para carregamento de todas as fontes de informação. Sendo que algumas vantagens são a centralização da informação, mais flexibilidade para os gestores na forma como acedem à informação. O tratamento dos dados de forma a ser possível a extração de informação a partir dos mesmos.This thesis focuses on the problem of building a data warehouse for AdClick company who operates in the area of digital marketing. Digital marketing is a type of marketing that uses digital media, with the same purpose of the traditional marketing which results on a effective publicity of goods, services and business to attract new clients. There are several digital marketing strategies in order to achieve these objectives, highlighting organic traffic and paid traffic. Organic traffic is characterized by the development of marketing actions that do not involve any costs related to promote and / or appeal new customers. In the other hand, paid traffic is manifested by the need to invest in campaigns to boost and attract new customers. First, an approach of the state of the art of business intelligence and data warehousing is made and presented the advantages of them to the companies. The business intelligence systems are needed, because currently firms have a high volume of data rich in information, which will only be fully exploited by making use of the potential of these systems. The first step in developing a business intelligence system is to concentrate all the data in a single integrated data warehouse to support decision making. It is then that we find here the construction of the data warehouse as the unique and ideal for this kind of requirements. This dissertation begins with a survey of the data sources that will supply the data warehouse and also starts with the contextualization of existing business processes in the company. After this, we begin the construction of the data warehouse, its dimensions and facts tables, and define the processes of data extraction and loading data into the data warehouse. Just as the creation of the views. Concerning the impact that this dissertation reached to the enterprise advantage that the partner in this work takes from the implementation of data warehouse and ETL processes for loading from all sources of information. Since some advantage are, the centralization of information, more flexibility for managers in how they access information. And the opportunity to extract intelligence from data that has been spread in the operational systems

Experimentation at Industrial Setting to Improve the Effectiveness of the ETL Procedures Implementation in a Business Intelligence Environment

Business Intelligence (BI) relies on Data Warehouse (DW), a historical data repository designed to support the decision making process. Without an effective Data Warehouse, organizations cannot extract the data required for information analysis in time to enable more effective strategic, tactical, and operational insights. This paper presents an approach and a Rapid Application Development (RAD) tool to increase efficiency and effectiveness of ETL (Extract, Transform and Load) programs development. An experimental evaluation of the approach is carried out in a controlled experiment that carefully evaluated the efficiency and effectiveness of the tool in an industrial setting. The results indicate that our approach can indeed be used as method aimed at improving ETL process development