1 research outputs found
Two-level Data Staging ETL for Transaction Data
In data warehousing, Extract-Transform-Load (ETL) extracts the data from data
sources into a central data warehouse regularly for the support of business
decision-makings. The data from transaction processing systems are featured
with the high frequent changes of insertion, update, and deletion. It is
challenging for ETL to propagate the changes to the data warehouse, and
maintain the change history. Moreover, ETL jobs typically run in a sequential
order when processing the data with dependencies, which is not optimal, \eg,
when processing early-arriving data. In this paper, we propose a two-level data
staging ETL for handling transaction data. The proposed method detects the
changes of the data from transactional processing systems, identifies the
corresponding operation codes for the changes, and uses two staging databases
to facilitate the data processing in an ETL process. The proposed ETL provides
the "one-stop" method for fast-changing, slowly-changing and early-arriving
data processing