Comparison of lossless compression schemes for high rate electrical grid time series for smart grid monitoring and analysis

The smart power grid of the future will utilize waveform level monitoring with sampling rates in the kilohertz range for detailed grid status assessment. To this end, we address the challenge of handling large raw data amount with its quasi-periodical characteristic via lossless compression. We compare different freely available algorithms and implementations with regard to compression ratio, computation time and working principle to find the most suitable compression strategy for this type of data. Algorithms from the audio domain (ALAC, ALS, APE, FLAC & TrueAudio) and general archiving schemes (LZMA, Delfate, PPMd, BZip2 & Gzip) are tested against each other. We assemble a dataset from openly available sources (UK-DALE, MIT-REDD, EDR) and establish dataset independent comparison criteria. This combination is a first detailed open benchmark to support the development of tailored lossless compression schemes and a decision support for researchers facing data intensive smart grid measurement

Advanced Wide-Area Monitoring System Design, Implementation, and Application

Wide-area monitoring systems (WAMSs) provide an unprecedented way to collect, store and analyze ultra-high-resolution synchrophasor measurements to improve the dynamic observability in power grids. This dissertation focuses on designing and implementing a wide-area monitoring system and a series of applications to assist grid operators with various functionalities. The contributions of this dissertation are below: First, a synchrophasor data collection system is developed to collect, store, and forward GPS-synchronized, high-resolution, rich-type, and massive-volume synchrophasor data. a distributed data storage system is developed to store the synchrophasor data. A memory-based cache system is discussed to improve the efficiency of real-time situation awareness. In addition, a synchronization system is developed to synchronize the configurations among the cloud nodes. Reliability and Fault-Tolerance of the developed system are discussed. Second, a novel lossy synchrophasor data compression approach is proposed. This section first introduces the synchrophasor data compression problem, then proposes a methodology for lossy data compression, and finally presents the evaluation results. The feasibility of the proposed approach is discussed. Third, a novel intelligent system, SynchroService, is developed to provide critical functionalities for a synchrophasor system. Functionalities including data query, event query, device management, and system authentication are discussed. Finally, the resiliency and the security of the developed system are evaluated. Fourth, a series of synchrophasor-based applications are developed to utilize the high-resolution synchrophasor data to assist power system engineers to monitor the performance of the grid as well as investigate the root cause of large power system disturbances. Lastly, a deep learning-based event detection and verification system is developed to provide accurate event detection functionality. This section introduces the data preprocessing, model design, and performance evaluation. Lastly, the implementation of the developed system is discussed

Data and resource management in wireless networks via data compression, GPS-free dissemination, and learning

“This research proposes several innovative approaches to collect data efficiently from large scale WSNs. First, a Z-compression algorithm has been proposed which exploits the temporal locality of the multi-dimensional sensing data and adapts the Z-order encoding algorithm to map multi-dimensional data to a one-dimensional data stream. The extended version of Z-compression adapts itself to working in low power WSNs running under low power listening (LPL) mode, and comprehensively analyzes its performance compressing both real-world and synthetic datasets. Second, it proposed an efficient geospatial based data collection scheme for IoTs that reduces redundant rebroadcast of up to 95% by only collecting the data of interest. As most of the low-cost wireless sensors won’t be equipped with a GPS module, the virtual coordinates are used to estimate the locations. The proposed work utilizes the anchor-based virtual coordinate system and DV-Hop (Distance vector of hops to anchors) to estimate the relative location of nodes to anchors. Also, it uses circle and hyperbola constraints to encode the position of interest (POI) and any user-defined trajectory into a data request message which allows only the sensors in the POI and routing trajectory to collect and route. It also provides location anonymity by avoiding using and transmitting GPS location information. This has been extended also for heterogeneous WSNs and refined the encoding algorithm by replacing the circle constraints with the ellipse constraints. Last, it proposes a framework that predicts the trajectory of the moving object using a Sequence-to-Sequence learning (Seq2Seq) model and only wakes-up the sensors that fall within the predicted trajectory of the moving object with a specially designed control packet. It reduces the computation time of encoding geospatial trajectory by more than 90% and preserves the location anonymity for the local edge servers”--Abstract, page iv

An enhanced lossless compression with cryptography hybrid mechanism for ECG biomedical signal monitoring

Due to their use in daily life situation, demand for remote health applications and e-health monitoring equipment is growing quickly. In this phase, for fast diagnosis and therapy, information can be transferred from the patient to the distant clinic. Nowadays, the most chronic disease is cardiovascular diseases (CVDs). However, the storage and transmission of the ECG signal, consumes more energy, bandwidth and data security which is faced many challenges. Hence, in this work, we present a combined approach for ECG data compression and cryptography. The compression is performed using adaptive Huffman encoding and encrypting is done using AES (CBC) scheme with a 256-bit key. To increase the security, we include Diffie-Hellman Key exchange to authenticate the receiver, RSA key generation for encrypting and decrypting the data. Experimental results show that the proposed approach achieves better performance in terms of compression and encryption on MIT-BIH ECG dataset

Design and development of learning model for compression and processing of deoxyribonucleic acid genome sequence

Owing to the substantial volume of human genome sequence data files (from 30-200 GB exposed) Genomic data compression has received considerable traction and storage costs are one of the major problems faced by genomics laboratories. This involves a modern technology of data compression that reduces not only the storage but also the reliability of the operation. There were few attempts to solve this problem independently of both hardware and software. A systematic analysis of associations between genes provides techniques for the recognition of operative connections among genes and their respective yields, as well as understandings into essential biological events that are most important for knowing health and disease phenotypes. This research proposes a reliable and efficient deep learning system for learning embedded projections to combine gene interactions and gene expression in prediction comparison of deep embeddings to strong baselines. In this paper we preform data processing operations and predict gene function, along with gene ontology reconstruction and predict the gene interaction. The three major steps of genomic data compression are extraction of data, storage of data, and retrieval of the data. Hence, we propose a deep learning based on computational optimization techniques which will be efficient in all the three stages of data compression

Spatial Statistical Data Fusion on Java-enabled Machines in Ubiquitous Sensor Networks

Wireless Sensor Networks (WSN) consist of small, cheap devices that have a combination of sensing, computing and communication capabilities. They must be able to communicate and process data efficiently using minimum amount of energy and cover an area of interest with the minimum number of sensors. This thesis proposes the use of techniques that were designed for Geostatistics and applies them to WSN field. Kriging and Cokriging interpolation that can be considered as Information Fusion algorithms were tested to prove the feasibility of the methods to increase coverage. To reduce energy consumption, a compression method that models correlations based on variograms was developed. A second challenge is to establish the communication to the external networks and to react to unexpected events. A demonstrator that uses commercial Java-enabled devices was implemented. It is able to perform remote monitoring, send SMS alarms and deploy remote updates