A comparison of Data Warehouse and Big Data principles and their application to systems of remote natural gas and electrical energy meter reading

Abstract

U daljinskom očitanju prirodnog plina prikuplja se velika količina podataka, zbog toga se u tvrtkama koje se bave time javlja problem kako te podatke skladištiti i obraditi. U novije vrijeme dvije osnovne tehnologije koje se pojavljuju u tu svrhu su Data Warehouse i Big Data te svaka od njih omogućuje različitu tehniku obrade podataka; Data Warehouse serijski obrađuje podatke dok Big Data koristi paralelnu obradu podataka. Za potrebe pilot projekta sastavljeni su zahtjevi koji će biti poslani potencijalnim ponuditeljima programskog rješenja pilot projekta. Prikazana je usporedba potencijalnih ponuditelja rješenja te je utvrđeno da za potrebe distributera prirodnog plina najviše odgovara ponuditelj Qlik. Navedeni rezultati također pokazuju da preostali potencijalni ponuditelji nude kvalitetna programska rješenja.In the remote reading of natural gas, a large amount of data is collected causing the problem of storing and processing such a type of data for the companies involved. More recently, two main technologies that appear for this purpose are Data Warehouse and Big Data, and each of them provides different data processing techniques; Data Warehouse uses serial data processing while Big Data uses parallel data processing. For the purpose of pilot project, in colaboration with the local natural gas distributer, the requirements for processing and storing the data collected are compiled and sent to the potential vendros. A comparison of the software packages provided by the potential vendors had shown that Qlik offers the best software package for the needs of the local natural gas distributor. The results also show that the other potential vendors also offer quality software packages