An automated ETL for online datasets

While using online datasets for machine learning is commonplace today, the quality of these datasets impacts on the performance of prediction algorithms. One method for improving the semantics of new data sources is to map these sources to a common data model or ontology. While semantic and structural heterogeneities must still be resolved, this provides a well established approach to providing clean datasets, suitable for machine learning and analysis. However, when there is a requirement for a close to real time usage of online data, a method for dynamic Extract-Transform-Load of new sources data must be developed. In this work, we present a framework for integrating online and enterprise data sources, in close to real time, to provide datasets for machine learning and predictive algorithms. An exhaustive evaluation compares a human built data transformation process with our system’s machine generated ETL process, with very favourable results, illustrating the value and impact of an automated approach

Proximal business intelligence on the semantic web

This is the post-print version of this article. The official version can be accessed from the link below - Copyright @ 2010 Springer.Ubiquitous information systems (UBIS) extend current Information System thinking to explicitly differentiate technology between devices and software components with relation to people and process. Adapting business data and management information to support specific user actions in context is an ongoing topic of research. Approaches typically focus on providing mechanisms to improve specific information access and transcoding but not on how the information can be accessed in a mobile, dynamic and ad-hoc manner. Although web ontology has been used to facilitate the loading of data warehouses, less research has been carried out on ontology based mobile reporting. This paper explores how business data can be modeled and accessed using the web ontology language and then re-used to provide the invisibility of pervasive access; uncovering more effective architectural models for adaptive information system strategies of this type. This exploratory work is guided in part by a vision of business intelligence that is highly distributed, mobile and fluid, adapting to sensory understanding of the underlying environment in which it operates. A proof-of concept mobile and ambient data access architecture is developed in order to further test the viability of such an approach. The paper concludes with an ontology engineering framework for systems of this type – named UBIS-ONTO