Data Warehouse Design and Management: Theory and Practice

The need to store data and information permanently, for their reuse in later stages, is a very relevant problem in the modern world and now affects a large number of people and economic agents. The storage and subsequent use of data can indeed be a valuable source for decision making or to increase commercial activity. The next step to data storage is the efficient and effective use of information, particularly through the Business Intelligence, at whose base is just the implementation of a Data Warehouse. In the present paper we will analyze Data Warehouses with their theoretical models, and illustrate a practical implementation in a specific case study on a pharmaceutical distribution companyData warehouse, database, data model.

Updating Data Warehouses with Temporal Data

There has been a growing trend to use temporal data in a data warehouse for making strategic and tactical decisions. The key idea of temporal data management is to make data available at the right time with different time intervals. The temporal data storing enables this by making all the different time slices of data available to whoever needs it. Users with different data latency needs can all be accommodated. Data can be “frozen” via a view on the proper time slice. Data as of a point in time can be obtained across multiple tables or multiple subject areas, resolving consistency and synchronization issues. This paper will discuss implementations such as temporal data updates, coexistence of load and query against the same table, performance of load and report queries, and maintenance of views against the tables with temporal data

CERN openlab Whitepaper on Future IT Challenges in Scientific Research

This whitepaper describes the major IT challenges in scientific research at CERN and several other European and international research laboratories and projects. Each challenge is exemplified through a set of concrete use cases drawn from the requirements of large-scale scientific programs. The paper is based on contributions from many researchers and IT experts of the participating laboratories and also input from the existing CERN openlab industrial sponsors. The views expressed in this document are those of the individual contributors and do not necessarily reflect the view of their organisations and/or affiliates

Presenting Business Insights on Advanced Pricing Agreements Using a Business Intelligence Framework

Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceIn companies that use advanced pricing agreements, pricing managers are responsible for setting the new and adjusted discounts from time to time. These companies are usually of great dimension and so the number of products and customers is extensive, which causes the decision-making to be challenging for the pricing managers. To aid in this process, this project report incorporates a business intelligence framework to model the data into a dimensional model that will provide the pricing managers with business insights by allowing them to have a more targeted and detailed view of the data through multiple contextual perspectives. The data sources used were provided by a client at BI4ALL and consist of two different JDE extracts: an export of the advanced pricing agreements that include all the pricing rules and an export of the sales data following those pricing rules. Both sources of data will be used to implement a business intelligence framework. The final outcome of this project report is presented in a dashboard with multiple visualizations, where the pricing manager can navigate and obtain data in a dynamic way according to the information requested. This will allow for a better analysis, and thus, for better pricing adjustment and optimization

Integration of decision support systems to improve decision support performance

Decision support system (DSS) is a well-established research and development area. Traditional isolated, stand-alone DSS has been recently facing new challenges. In order to improve the performance of DSS to meet the challenges, research has been actively carried out to develop integrated decision support systems (IDSS). This paper reviews the current research efforts with regard to the development of IDSS. The focus of the paper is on the integration aspect for IDSS through multiple perspectives, and the technologies that support this integration. More than 100 papers and software systems are discussed. Current research efforts and the development status of IDSS are explained, compared and classified. In addition, future trends and challenges in integration are outlined. The paper concludes that by addressing integration, better support will be provided to decision makers, with the expectation of both better decisions and improved decision making processes

The problems and challenges of managing crowd sourced audio-visual evidence

A number of recent incidents, such as the Stanley Cup Riots, the uprisings in the Middle East and the London riots have demonstrated the value of crowd sourced audio-visual evidence wherein citizens submit audio-visual footage captured on mobile phones and other devices to aid governmental institutions, responder agencies and law enforcement authorities to confirm the authenticity of incidents and, in the case of criminal activity, to identify perpetrators. The use of such evidence can present a significant logistical challenge to investigators, particularly because of the potential size of data gathered through such mechanisms and the added problems of time-lining disparate sources of evidence and, subsequently, investigating the incident(s). In this paper we explore this problem and, in particular, outline the pressure points for an investigator. We identify and explore a number of particular problems related to the secure receipt of the evidence, imaging, tagging and then time-lining the evidence, and the problem of identifying duplicate and near duplicate items of audio-visual evidence

Data warehouse automation trick or treat?

Data warehousing systems have been around for 25 years playing a crucial role in collecting data and transforming that data into value, allowing users to make decisions based on informed business facts. It is widely accepted that a data warehouse is a critical component to a data-driven enterprise, and it becomes part of the organisation’s information systems strategy, with a significant impact on the business. However, after 25 years, building a Data Warehouse is still painful, they are too time-consuming, too expensive and too difficult to change after deployment. Data Warehouse Automation appears with the promise to address the limitations of traditional approaches, turning the data warehouse development from a prolonged effort into an agile one, with gains in efficiency and effectiveness in data warehousing processes. So, is Data Warehouse Automation a Trick or Treat? To answer this question, a case study of a data warehousing architecture using a data warehouse automation tool, called WhereScape, was developed. Also, a survey was made to organisations that are using data warehouse automation tools, in order to understand their motivation in the adoption of this kind of tools in their data warehousing systems. Based on the results of the survey and on the case study, automation in the data warehouses building process is necessary to deliver data warehouse systems faster, and a solution to consider when modernize data warehouse architectures as a way to achieve results faster, keeping costs controlled and reduce risk. Data Warehouse Automation definitely may be a Treat.Os sistemas de armazenamento de dados existem há 25 anos, desempenhando um papel crucial na recolha de dados e na transformação desses dados em valor, permitindo que os utilizadores tomem decisões com base em fatos. É amplamente aceite, que um data warehouse é um componente crítico para uma empresa orientada a dados e se torna parte da estratégia de sistemas de informação da organização, com um impacto significativo nos negócios. No entanto, após 25 anos, a construção de um Data Warehouse ainda é uma tarefa penosa, demora muito tempo, é cara e difícil de mudar após a sua conclusão. A automação de Data Warehouse aparece com a promessa de endereçar as limitações das abordagens tradicionais, transformando o desenvolvimento da data warehouse de um esforço prolongado em um esforço ágil, com ganhos de eficiência e eficácia. Será, a automação de Data Warehouse uma doçura ou travessura? Foi desenvolvido um estudo de caso de uma arquitetura de data warehousing usando uma ferramenta de automação, designada WhereScape. Foi também conduzido um questionário a organizações que utilizam ferramentas de automação de data warehouse, para entender sua motivação na adoção deste tipo de ferramentas. Com base nos resultados da pesquisa e no estudo de caso, a automação no processo de construção de data warehouses, é necessária para uma maior agilidade destes sistemas e uma solução a considerar na modernização destas arquiteturas, pois permitem obter resultados mais rapidamente, mantendo os custos controlados e reduzindo o risco. A automação de data warehouse pode bem vir a ser uma “doçura”

Assessing the environmental impact of logistics sites through CO2eq footprint computation

The environmental sustainability of logistics facilities is widely acknowledged as an important issue, but a comprehensive standardised methodology for assessing their environmental impact is lacking. This study proposes a structured model for quantifying both consumptions and generated greenhouse gas (GHG) emissions, adopting a three-phase methodology that combines multiple methods. A literature-based conceptual framework was leveraged to design an analytical model, and in-depth interviews with 11 senior logistics managers were conducted. The study offers a replicable methodology that considers heterogeneous sources of consumption and related end-use types, further splitting consumptions and emissions by warehouses' functional areas. It offers a set of Environmental Performance Indicators (EPIs) that could bolster a clearer understanding of the warehouse environmental performance. A robust tool is offered to managers to support their decision-making processes, allowing for both internal assessments and benchmarking with competitors or other players along the supply chain, thus contributing to shape company's, or even supply chain, sustainability strategies

Leveraging Data Mining and Data Warehouse to Improve Prison Services and Operations in Nigeria

Crimes are social nuisance and cost our society dearly in several ways. In Nigeria, any research geared towards helping to solve crimes faster will be beneficial to the society at large. It has been observed that the major challenge facing all law-enforcement and intelligence-gathering organizations in Nigeria is how to accurately and efficiently analyze the growing volume of crime data. As the volume of this crime data becomes enormously large, new techniques have to be used to turn this data into valuable information and actionable knowledge so that appropriate actions can be taken accordingly. Sometimes it is usual to find that the data needed to be analyzed to produce report are scattered throughout different operational States and jurisdictions of Nigeria and must first be carefully integrated. Moreover, observations show that the process required to extract the existing data from each operational system demand so much of the system resources such that the IT professional must wait until nonoperational hours before running targeted queries required for producing operational reports. These delays are not only time-consuming and frustrating for both the IT professionals and the decision-makers they are dangerous for the sector whose primary task is to control crime spread and explosion. It should be noted that when such operational reports are finally produced, they may not be relied upon, because the data use in producing them many a times are inconsistent, inaccurate, or obsolete. This paper therefore highlights the increasing growing need for Data integration, Data warehouse and Data mining as ways to improve the operations of principal actors within the prisons sector of Nigeria. The paper explains what these Data management techniques mean and entail, and furthermore suggests ways to effectively leverage the techniques to help detect existing crime patterns and speed up the process of solving crimes. Keywords: Crime-data, data mining, data mining techniques, data warehouse, data integratio

Business Intelligence for Small and Middle-Sized Entreprises

Data warehouses are the core of decision support sys- tems, which nowadays are used by all kind of enter- prises in the entire world. Although many studies have been conducted on the need of decision support systems (DSSs) for small businesses, most of them adopt ex- isting solutions and approaches, which are appropriate for large-scaled enterprises, but are inadequate for small and middle-sized enterprises. Small enterprises require cheap, lightweight architec- tures and tools (hardware and software) providing on- line data analysis. In order to ensure these features, we review web-based business intelligence approaches. For real-time analysis, the traditional OLAP architecture is cumbersome and storage-costly; therefore, we also re- view in-memory processing. Consequently, this paper discusses the existing approa- ches and tools working in main memory and/or with web interfaces (including freeware tools), relevant for small and middle-sized enterprises in decision making