Towards Comparative Analysis of Resumption Techniques in ETL

Data warehouses are loaded with data from sources such as operational data bases. Failure of loading process or failure of any of the process such as extraction or transformation is expensive because of the non-availability of data for analysis. With the advent of e-commerce and many real time application analysis of data in real time becomes a norm and hence any misses while the data is being loaded into data warehouse needs to be handled in an efficient and optimized way. The techniques to handle failure of process to populate the data are very much important as the actual loading process. Alternative arrangement needs to be made for in case of failure so that processes of populating the data warehouse are done in time. This paper explores the various ways through which a failed process of populating the data warehouse could be resumed. Various resumption techniques are compared and a novel block based technique is proposed to improve one of the existing resumption techniques

Quality measures for ETL processes: from goals to implementation

Extraction transformation loading (ETL) processes play an increasingly important role for the support of modern business operations. These business processes are centred around artifacts with high variability and diverse lifecycles, which correspond to key business entities. The apparent complexity of these activities has been examined through the prism of business process management, mainly focusing on functional requirements and performance optimization. However, the quality dimension has not yet been thoroughly investigated, and there is a need for a more human-centric approach to bring them closer to business-users requirements. In this paper, we take a first step towards this direction by defining a sound model for ETL process quality characteristics and quantitative measures for each characteristic, based on existing literature. Our model shows dependencies among quality characteristics and can provide the basis for subsequent analysis using goal modeling techniques. We showcase the use of goal modeling for ETL process design through a use case, where we employ the use of a goal model that includes quantitative components (i.e., indicators) for evaluation and analysis of alternative design decisions.Peer ReviewedPostprint (author's final draft

Business intelligence-centered software as the main driver to migrate from spreadsheet-based analytics

Internship Report presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceNowadays, companies are handling and managing data in a way that they weren’t ten years ago. The data deluge is, as a mere consequence of that, the constant day-to-day challenge for them - having to create agile and scalable data solutions to tackle this reality. The main trigger of this project was to support the decision-making process of a customer-centered marketing team (called Customer Voice) in the Company X by developing a complete, holistic Business Intelligence solution that goes all the way from ETL processes to data visualizations based on that team’s business needs. Having this context into consideration, the focus of the internship was to make use of BI, ETL techniques to migrate their data stored in spreadsheets — where they performed data analysis — and shift the way they see the data into a more dynamic, sophisticated, and suitable way in order to help them make data-driven strategic decisions. To ensure that there was credibility throughout the development of this project and its subsequent solution, it was necessary to make an exhaustive literature review to help me frame this project in a more realistic and logical way. That being said, this report made use of scientific literature that explained the evolution of the ETL workflows, tools, and limitations across different time periods and generations, how it was transformed from manual to real-time data tasks together with data warehouses, the importance of data quality and, finally, the relevance of ETL processes optimization and new ways of approaching data integrations by using modern, cloud architectures

Discovery of association rules from medical data -classical and evolutionary approaches

The paper presents a method of association rules discovering from medical data using the evolutionary approach. The elaborated method (EGAR) uses a genetic algorithm as a tool of knowledge discovering from a set of data, in the form of association rules. The method is compared with known and common method - FPTree. The developed computer program is applied for testing the proposed method and comparing the results with those produced by FPTree. The program is the general and flexible tool for the rules generation task using different data sets and two embodied methods. The presented experiments are performed using the actual medical data from the Wroclaw Clinic

Offloading SLAM for Indoor Mobile Robots with Edge-Fog-Cloud Computing

Indoor mobile robots are widely used in industrial environments such as large logistic warehouses. They are often in charge of collecting or sorting products. For such robots, computation-intensive operations account for a significant per- centage of the total energy consumption and consequently affect battery life. Besides, in order to keep both the power con- sumption and hardware complexity low, simple micro-controllers or single-board computers are used as onboard local control units. This limits the computational capabilities of robots and consequently their performance. Offloading heavy computation to Cloud servers has been a widely used approach to solve this problem for cases where large amounts of sensor data such as real-time video feeds need to be analyzed. More recently, Fog and Edge computing are being leveraged for offloading tasks such as image processing and complex navigation algorithms involving non-linear mathematical operations. In this paper, we present a system architecture for offloading computationally expensive localization and mapping tasks to smart Edge gateways which use Fog services. We show how Edge computing brings computational capabilities of the Cloud to the robot environment without compromising operational reliability due to connection issues. Furthermore, we analyze the power consumption of a prototype robot vehicle in different modes and show how battery life can be significantly improved by moving the processing of data to the Edge layer

Offloading SLAM for Indoor Mobile Robots with Edge, Fog, Cloud Computing

Indoor mobile robots are widely used in industrial environments such as large logistic warehouses. They are often in charge of collecting or sorting products. For such robots, computation-intensive operations account for a significant per- centage of the total energy consumption and consequently affect battery life. Besides, in order to keep both the power con- sumption and hardware complexity low, simple micro-controllers or single-board computers are used as onboard local control units. This limits the computational capabilities of robots and consequently their performance. Offloading heavy computation to Cloud servers has been a widely used approach to solve this problem for cases where large amounts of sensor data such as real-time video feeds need to be analyzed. More recently, Fog and Edge computing are being leveraged for offloading tasks such as image processing and complex navigation algorithms involving non-linear mathematical operations. In this paper, we present a system architecture for offloading computationally expensive localization and mapping tasks to smart Edge gateways which use Fog services. We show how Edge computing brings computational capabilities of the Cloud to the robot environment without compromising operational reliability due to connection issues. Furthermore, we analyze the power consumption of a prototype robot vehicle in different modes and show how battery life can be significantly improved by moving the processing of data to the Edge layer

Quality Measures for ETL Processes

ETL processes play an increasingly important role for the support of modern business operations. These business processes are centred around artifacts with high variability and diverse lifecycles, which correspond to key business entities. The apparent complexity of these activities has been examined through the prism of Business Process Management, mainly focusing on functional requirements and performance optimization. However, the quality dimension has not yet been thoroughly investigated and there is a need for a more human-centric approach to bring them closer to business-users requirements. In this paper we take a first step towards this direction by defining a sound model for ETL process quality characteristics and quantitative measures for each characteristic, based on existing literature. Our model shows dependencies among quality characteristics and can provide the basis for subsequent analysis using Goal Modeling techniques

The Fripe as Urban Economy: Market- and Space-Making in Tunis

“Fripe” is the term used to refer to a heterogeneous array of imported second-hand donations and fast-fashion cast-offs in Tunisia that are part of the global second-hand trade. This thesis builds an account of the fripe as a singular urban economy in Tunis, the capital city. Firstly, it comprises a hitherto unwritten analysis of the fripe’s historical constitution as a contested urban economy; demonstrating its distinct political economy and constructing counter-histories of urban renewal that reveal the role of the fripe trade and its rural migrant constituencies in remaking post-independence Tunis. These histories expose the systems of differentiation that operate to exclude the fripe from formally delimited realms of ‘the national economy’ and ‘the planned urban order’, while also partially incorporating it into modes of government and city-making. Secondly, it investigates the entanglements of contemporary processes of market- and space-making that position the fripe economy as a central agent of urban change, as captured in the vernacular word creation “fripisation”. The ethnography of economic practices underpinning this investigation starts with the unstable commodity status of fripe imports, examining the situated processes of valuation that allow diverse garments and objects to enter renewed cycles of commodity circulation and exchange in Tunis. Centring on what is termed ‘valuation work’ by diverse market-makers in the fripe economy, emphasis lies on how the economy is enacted in urban space and becomes constitutive to socio-spatial relations. Three ‘collective enactments’ of fripe valuation demonstrate how the economy drives localised urban transformations; creates interdependencies and rhythms connecting disparate actors and sites; and allows the staging of temporary publicness. Overall, this thesis advances a perspective on the economy as operating through and as constitutive to urban space, positing the ‘urban economy’ as a tool to expand what can be brought to matter as economy in present-day cities