A High-Performance Data Accessing and Processing System for Campus Real-time Power Usage

With the flourishing of Internet of Things (IoT) technology, ubiquitous power data can be linked to the Internet and be analyzed for real-time monitoring requirements. Numerous power data would be accumulated to even Tera-byte level as the time goes. To approach a real-time power monitoring platform on them, an efficient and novel implementation techniques has been developed and formed to be the kernel material of this thesis. Based on the integration of multiple software subsystems in a layered manner, the proposed power-monitoring platform has been established and is composed of Ubuntu (as operating system), Hadoop (as storage subsystem), Hive (as data warehouse), and the Spark MLlib (as data analytics) from bottom to top. The generic power-data source is provided by the so-called smart meters equipped inside factories located in an enterprise practically. The data collection and storage are handled by the Hadoop subsystem and the data ingestion to Hive data warehouse is conducted by the Spark unit. On the aspect of system verification, under single-record query, these software modules: HiveQL and Impala SQL had been tested in terms of query-response efficiency. And for the performance exploration on the full-table query function. The relevant experiments have been conducted on the same software modules as well. The kernel contributions of this research work can be highlighted by two parts: the details of building an efficient real-time power-monitoring platform, and the relevant query-response efficiency for reference

Comparative Review on Information and Communication Technology Issues in Education Sector of Developed and Developing Countries: A Case Study About Pakistan

The use of information and communication technology is very beneficial in the education sector because it can enhance the quality of education. However, the implementation of ICT in the education sector of developed and developing countries is a challenging task. This paper explains the comparative study of ICT issues in the education sector of developed and developing countries. In particular, we compare issues between Pakistan and high-tech countries. Our study reveals the fact that the education sector is facing numerous ICT problems that are based on culture, finance, management, infrastructure, lack of training, lack of equipment, teacher’s refusal, and ethical issues. At the end of this paper, various issues faced by the implementation of ICT in the education sector of Pakistan have been categorized into various types, namely, infrastructure, lack of IT professionals, lack of high-speed internet and equipment. Our research is based on five key research questions related to ICT issues. We used a mixed approach where the results of this study can be used as a set of guidelines to help make the learning environment technology-oriented, fast, planned, and productive. Future directions are also given at the end of this paper

How to manage massive spatiotemporal dataset from stationary and non-stationary sensors in commercial DBMS?

The growing diffusion of the latest information and communication technologies in different contexts allowed the constitution of enormous sensing networks that form the underlying texture of smart environments. The amount and the speed at which these environments produce and consume data are starting to challenge current spatial data management technologies. In this work, we report on our experience handling real-world spatiotemporal datasets: a stationary dataset referring to the parking monitoring system and a non-stationary dataset referring to a train-mounted railway monitoring system. In particular, we present the results of an empirical comparison of the retrieval performances achieved by three different off-the-shelf settings to manage spatiotemporal data, namely the well-established combination of PostgreSQL + PostGIS with standard indexing, a clustered version of the same setup, and then a combination of the basic setup with Timescale, a storage extension specialized in handling temporal data. Since the non-stationary dataset has put much pressure on the configurations above, we furtherly investigated the advantages achievable by combining the TSMS setup with state-of-the-art indexing techniques. Results showed that the standard indexing is by far outperformed by the other solutions, which have different trade-offs. This experience may help researchers and practitioners facing similar problems managing these types of data

Big Data Analytics na energia: uma revisão sistemática da literatura

A transformação para uma sociedade mais digital tem vindo a produzir uma quantidade de dados incomparável com décadas anteriores. Big data analytics é agora um termo muito conhecido ao nível de dados financeiros, saúde, marketing, logística, entre outros. Sendo a energia uma das áreas afetadas pela transformação digital, como tem sido aplicado o conceito de big data analytics nesta área? Que aplicações surgiram devido ao volume de dados recolhidos? Que ferramentas são utilizadas? Como tem sido afetada por esta mudança? Esta dissertação pretende responder à pergunta-chave “Como tem sido aplicado big data analytics na área da energia?”. Para o efeito procede-se à realização de uma revisão sistemática da literatura, a partir da qual se elabora uma introdução das temáticas discutidas para leitores que possam estar a iniciar estudos nesta área de investigação, garantindo, ao mesmo tempo, que esta seja útil para leitores que já se encontram inseridos nestas áreas. Para a realização da revisão foi consultada a base de dados Science Direct, sendo construída uma frase booleana através das palavras-chave selecionadas durante a fase de pesquisa da base de dados sendo estas “Big Data”, “Analytics”, “Energy”, “Smart Grid”, “Smart Meters”, “Smart Sensor Network”, “Meter Data Analytics”, “Energy Management” e “Energy Consumption” tendo sido realizado um estudo bibliométrico aos resultados obtidos. Para a seleção dos estudos foram utilizados três métodos de filtragem: um primeiro centrado nos critérios de seleção; o segundo através da leitura dos títulos das publicações e dos resumos; e o terceiro através da realização de um teste de relevância. A exploração da informação permitiu concluir que big data analytics na energia tem sido aplicada em áreas como o estudo do consumidor, soluções para análise de dados recolhidos por smart meters, desenvolvimento de plataformas de análise das smart grids, segurança e gestão inteligente da energia que se refletem na forma como a energia é canalizada, aproveitada e utilizada através da associação de técnicas, processos e sistemas planeados para análise de grandes volumes de dados, resultando numa tomada de decisões mais consciente pelos fornecedores e consumidores de energi

An Open Source Cyberinfrastructure for Collecting, Processing, Storing and Accessing High Temporal Resolution Residential Water Use Data

Collecting and managing high temporal resolution residential water use data is challenging due to cost and technical requirements associated with the volume and velocity of data collected. We developed an open-source, modular, generalized architecture called Cyberinfrastructure for Intelligent Water Supply (CIWS) to automate the process from data collection to analysis and presentation of high temporal residential water use data. A prototype implementation was built using existing open-source technologies, including smart meters, databases, and services. Two case studies were selected to test functionalities of CIWS, including push and pull data models within single family and multi-unit residential contexts, respectively. CIWS was tested for scalability and performance within our design constraints and proved to be effective within both case studies. All CIWS elements and the case study data described are freely available for re-use