Data compression in smart distribution systems via singular value decomposition

Electrical distribution systems have been experiencing many changes in recent times. Advances in metering system infrastructure and the deployment of a large number of smart meters in the grid will produce a big volume of data that will be required for many different applications. Despite the significant investments taking place in the communications infrastructure, this remains a bottleneck for the implementation of some applications. This paper presents a methodology for lossy data compression in smart distribution systems using the singular value decomposition technique. The proposed method is capable of significantly reducing the volume of data to be transmitted through the communications network and accurately reconstructing the original data. These features are illustrated by results from tests carried out using real data collected from metering devices at many different substations

Techno-economic analysis of Smart Grid pilot project- Puducherry

Smart Grid (SG) a well-known concept being rapidly introduced in the power industry. New transformation of Indian power network has begun with 14 SG pilot projects across the nation. One of such projects has been successfully commissioned in Puducherry. The motive of this research work is to analyze the techno-economic aspects of a smart distribution network before being implemented nation-wide the study case facilitating efficient planning and deployment of technology. This paper presents techno-economic analysis of Smart Grid via a case study of Puducherry pilot project. Covered in this paper are different components of investment which convey an idea about services and their proposition as well as technical advancements with their benefits. This paper discusses the gain in terms of energy and money saving through different smart technical tools. Payback analysis explains how investment in smart distribution network is justified

Comparison of lossless compression schemes for high rate electrical grid time series for smart grid monitoring and analysis

The smart power grid of the future will utilize waveform level monitoring with sampling rates in the kilohertz range for detailed grid status assessment. To this end, we address the challenge of handling large raw data amount with its quasi-periodical characteristic via lossless compression. We compare different freely available algorithms and implementations with regard to compression ratio, computation time and working principle to find the most suitable compression strategy for this type of data. Algorithms from the audio domain (ALAC, ALS, APE, FLAC & TrueAudio) and general archiving schemes (LZMA, Delfate, PPMd, BZip2 & Gzip) are tested against each other. We assemble a dataset from openly available sources (UK-DALE, MIT-REDD, EDR) and establish dataset independent comparison criteria. This combination is a first detailed open benchmark to support the development of tailored lossless compression schemes and a decision support for researchers facing data intensive smart grid measurement

The collaborative iterative search approach to multi agent path finding

PhD ThesisThis thesis presents a new approach to obtaining optimal and complete solutions to Multi Agent Path Finding (MAPF) problems called Collaborative Iterative Search (CIS). CIS employs a conflict based scheme inspired by the Conflict Based Search (CBS) algorithm and extends this to include a linear order lower level search. The structure of Planar Graphs is leveraged, permitting further optimization of the algorithm. This takes the form of reasoning-based culling of the search space, while maintaining optimality and completeness. Benchmarks provided demonstrate significant performance gains over the existing state of the art, particularly in the case of sparsely populated maps. The thesis draws to a conclusion with a summary of proposed future work

Compressão de sinais para smart grid

Trabalho de Conclusão de Curso (graduação)—Universidade de Brasília, Faculdade UnB Gama, 2019.Para modernizar o sistema elétrico atual são necessárias novas tecnologias entre elas, as tecnologias da informação e comunicação. Considerando que os sinais elétricos de uma rede devem são continuamente medidos e que a importância e quantidade dos dados gerados por estas medições ´e grande o suficiente para gerar uma demanda significativa de transmissão de dados, a compressão das informações armazenadas passa a ter grande relevância, pois a compressão destes sinais também permite o armazenamento de dados a respeito de um grande número de variáveis, por longos períodos, para análises posteriores mais aprofundadas, as quais podem resultar em grandes ganhos de eficiência e segurança nas operações do sistema elétrico. Vale ressaltar que o Brasil possui um sistema elétrico integrado, e ´e um país com muito potencial para investimentos na aplicação das redes elétricas inteligentes, dado seu tamanho e suas necessidades no setor elétrico. Este trabalho propõe um algoritmo de compressão para sinais com distúrbios. São apresentados os procedimentos que envolveram o tratamento matemático dos sinais em domínio transformado, como a quantização e alocação de bits, a codificação, a decodificação e todos os passos realizados até a reconstrução do sinal tratado, assim como os resultados dos métodos utilizados.To modernize the current electrical system, new technologies are needed, including information and communication technologies. Considering that the electrical signals of a network must be continuously measured and that the importance and quantity of the data generated by these measurements is large enough to generate a significant demand for data transmission, the compression of the stored information is of great relevance, since the compression of these signals also allows the storage of data about a large number of variables, for longer periods, for further in-depth analyzes, which can result in great gains in efficiency and safety in the operations of an electrical system. It is worth mentioning that Brazil has an integrated electrical system, and it is a country with a lot of potential for investments in the application of smart electricity grids, given the size of the country and the current needs in the electric sector. This paper proposes a compression algorithm for signals with disturbances. The procedures that involve the mathematical treatment of the signals in the transformed domain are presented, such as quantization and allocation of bits, encoding, decoding and all the steps performed until the reconstruction of the treated signal, as well as the results of the methods used

Compressão de sinais de tensão e corrente com distúrbios em redes de distribuição de energia elétrica

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2020.O desenvolvimento da sociedade pode ser observado a partir do aumento da complexidade dos sistemas de distribuição e geração de energia elétrica. A necessidade de uso de aparelhos elétricos colocou a humanidade com uma alta dependência de um sistema que compartilhe a eletricidade de maneira eficaz, isto é, com uma boa qualidade de energia. Para isso, sistemas como redes inteligentes passaram a ser desenvolvidos. As suas motivações iniciais, assim, são proporcionar a garantia de sistemas confiáveis, flexíveis e econômicos. Para isso, a manutenção de redes de distribuição de energia elétrica passou a ser fator indispensável para a garantia da qualidade de energia. Diversos aparelhos foram desenvolvidos para proporcionar a monitoração das redes elétricas, como sensores. As informações desses aparelhos são utilizadas para as análises acerca de distúrbios e outras interferências possíveis de terem ocorrido. A boa observação desses dados acarreta no desenvolvimento propício de aparelhos capazes de reagir de maneira adequada a dados acontecimentos de redes. Com a maior quantidade de observações das redes elétricas, a necessidade de armazenamento e transmissão eficientes desses dados passa a ser um desafio. Dessa forma, técnicas de compressão de sinais são apresentadas como boas alternativas para a solução deste problema. Tais técnicas devem garantir, assim, a representação do sinal com a menor quantidade de bits possível e a preservação do conteúdo original do sinal. É neste contexto que este trabalho se insere. Foi desenvolvido um sistema de compressão de dados de redes elétricas baseado em codificação por sub-banda com o objetivo de garantir um bom ganho de compressão sem a perda da informação original do sinal. Para isso, o sistema é composto de diversas etapas do algoritmo de compressão: segmentação do sinal, transformada, quantização e codificação de entropia. Este sistema apresenta, também, inovações acerca da quantização do sinal. Neste âmbito ocorre a quantização por alocação dinâmica de bits, a qual atribui uma maior quantidade de bits para os coeficientes com maior importância para o sinal. As observações dos resultados apresentados com as alterações dos parâmetros de cada etapa possibilitam a obtenção de uma configuração final do programa para garantir a eficiente compressão dos sinais de tensão e corrente. Dessa forma, este trabalho apresenta tais configurações e suas sucessivas análises. Os resultados apresentados ao final deste trabalho têm como base a utilização de sinais de tensão e corrente reais com a presença de distúrbios obtidos em redes elétricas. São obtidas, por fim, relações que comprovam a boa compressão e a sucessiva restauração propícia do sinal, a qual não acarreta em grandes alterações nos distúrbios presentes.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).The development of society can be seen from the increase in the complexity of the electricity distribution and generation systems. The need to use electrical appliances has placed humanity with a high dependence on a system that shares electricity effectively, that is, with a good quality of energy. For that, systems like smart grids started to be developed. Its initial motivations, therefore, are to provide the guarantee of reliable, flexible and economical systems. For this, the maintenance of electricity distribution networks has become an indispensable attribute for the guarantee of energy quality. Several devices have been developed to provide monitoring of electrical networks. Among them can be mentioned sensors, which promote the capture of data such as voltage and current signals. Such information is used, therefore, for the analysis of disturbances and other possible interferences that may have occurred. The good observation of these data leads to the favorable development of devices that react appropriately to data from network events. With the greater amount of observations from the electrical networks, the need for efficient storage and transmission of this data becomes a challenge. Thus, signal compression techniques are presented as good alternatives for the solution of this problem. Such techniques must guarantee the representation of the signal with as least bits as possible and the preservation of the original content of the signal. It is in this context that this work is inserted. An electrical network data compression system based on subband coding was developed in order to ensure a good compression gain without losing the original signal information. For this, the system is composed of several steps of the compression algorithm: signal segmentation, transform, quantization and entropy coding. This system also presents innovations about the quantization of the signal. In this context, quantization by dynamic bit allocation occurs, which assigns a greater amount of bits to the coefficients with greater importance for the signal. The observations of the results presented with the alterations of the parameters of each stage makes it possible to obtain a final configuration of the program to guarantee the efficient compression of the voltage and current signals. Thus, this work presents such configurations and their successive analyzes. The results presented at the end of this work are based on the use of real voltage and current signals obtained in electrical networks, which present disturbances such as transients and interruptions. Finally, relations are obtained that prove the good compression and successive favorable restoration of the signal, which does not result in major changes in the present distortions

Robust data protection and high efficiency for IoTs streams in the cloud

Remotely generated streaming of the Internet of Things (IoTs) data has become a vital category upon which many applications rely. Smart meters collect readings for household activities such as power and gas consumption every second - the readings are transmitted wirelessly through various channels and public hops to the operation centres. Due to the unusually large streams sizes, the operation centres are using cloud servers where various entities process the data on a real-time basis for billing and power management. It is possible that smart pipe projects (where oil pipes are continuously monitored using sensors) and collected streams are sent to the public cloud for real-time flawed detection. There are many other similar applications that can render the world a convenient place which result in climate change mitigation and transportation improvement to name a few. Despite the obvious advantages of these applications, some unique challenges arise posing some questions regarding a suitable balance between guaranteeing the streams security, such as privacy, authenticity and integrity, while not hindering the direct operations on those streams, while also handling data management issues, such as the volume of protected streams during transmission and storage. These challenges become more complicated when the streams reside on third-party cloud servers. In this thesis, a few novel techniques are introduced to address these problems. We begin by protecting the privacy and authenticity of transmitted readings without disrupting the direct operations. We propose two steganography techniques that rely on different mathematical security models. The results look promising - security: only the approved party who has the required security tokens can retrieve the hidden secret, and distortion effect with the difference between the original and protected readings that are almost at zero. This means the streams can be used in their protected form at intermediate hops or third party servers. We then improved the integrity of the transmitted protected streams which are prone to intentional or unintentional noise - we proposed a secure error detection and correction based stenographic technique. This allows legitimate recipients to (1) detect and recover any noise loss from the hidden sensitive information without privacy disclosure, and (2) remedy the received protected readings by using the corrected version of the secret hidden data. It is evident from the experiments that our technique has robust recovery capabilities (i.e. Root Mean Square (RMS) <0.01%, Bit Error Rate (BER) = 0 and PRD < 1%). To solve the issue of huge transmitted protected streams, two compression algorithms for lossless IoTs readings are introduced to ensure the volume of protected readings at intermediate hops is reduced without revealing the hidden secrets. The first uses Gaussian approximation function to represent IoTs streams in a few parameters regardless of the roughness in the signal. The second reduces the randomness of the IoTs streams into a smaller finite field by splitting to enhance repetition and avoiding the floating operations round errors issues. Under the same conditions, our both techniques were superior to existing models mathematically (i.e. the entropy was halved) and empirically (i.e. achieved ratio was 3.8:1 to 4.5:1). We were driven by the question ‘Can the size of multi-incoming compressed protected streams be re-reduced on the cloud without decompression?’ to overcome the issue of vast quantities of compressed and protected IoTs streams on the cloud. A novel lossless size reduction algorithm was introduced to prove the possibility of reducing the size of already compressed IoTs protected readings. This is successfully achieved by employing similarity measurements to classify the compressed streams into subsets in order to reduce the effect of uncorrelated compressed streams. The values of every subset was treated independently for further reduction. Both mathematical and empirical experiments proved the possibility of enhancing the entropy (i.e. almost reduced by 50%) and the resultant size reduction (i.e. up to 2:1)