Short Term Load Calculating for Smart Meter Data using Weighted KNN for Classification & PredictionÂ

No Abstrac

Fault detection in low voltage networks with smart meters and machine learning techniques

25th International Conference on Electricity Distribution (CIRED 2019), junio, Madrid (Spain)Smart grid data analytics and artificial intelligence techniques are playing an increasingly critical role, becoming the focal point to understanding low voltage real-time grid performance. This new point of view, (advanced analytics in combination with electrical knowledge expertise), makes flexibility and efficiency in electrical grid management approach real. HDCE (Hidrocantábrico Distribución Eléctrica) is the Electrical Distribution System Operator for EdP (Electricity of Portugal) around Spain who supplies energy to 650.000 customers. Starting from 2012, this company has nowadays replaced 99% of traditional meters by smart meters. Based on the analysis of smart metering voltage alarms, recorded from EdP LV distribution network, an automatic learning system has been implemented that groups and orders these alarms helping the grid distribution operator to drive the network technicians to the right and more urgent places where a grid failure is happening, starts to happen or will happen

An Artificial Intelligence Framework for Bidding Optimization with Uncertainty inMultiple Frequency Reserve Markets

The global ambitions of a carbon-neutral society necessitate a stable and robust smart grid that capitalises on frequency reserves of renewable energy. Frequency reserves are resources that adjust power production or consumption in real time to react to a power grid frequency deviation. Revenue generation motivates the availability of these resources for managing such deviations. However, limited research has been conducted on data-driven decisions and optimal bidding strategies for trading such capacities in multiple frequency reserves markets. We address this limitation by making the following research contributions. Firstly, a generalised model is designed based on an extensive study of critical characteristics of global frequency reserves markets. Secondly, three bidding strategies are proposed, based on this market model, to capitalise on price peaks in multi-stage markets. Two strategies are proposed for non-reschedulable loads, in which case the bidding strategy aims to select the market with the highest anticipated price, and the third bidding strategy focuses on rescheduling loads to hours on which highest reserve market prices are anticipated. The third research contribution is an Artificial Intelligence (AI) based bidding optimization framework that implements these three strategies, with novel uncertainty metrics that supplement data-driven price prediction. Finally, the framework is evaluated empirically using a case study of multiple frequency reserves markets in Finland. The results from this evaluation confirm the effectiveness of the proposed bidding strategies and the AI-based bidding optimization framework in terms of cumulative revenue generation, leading to an increased availability of frequency reserves

A New Framework for the Analysis of Large Scale Multi-Rate Power Data

A new framework for the analysis of large scale, multi-rate power data is introduced. The system comprises high rate power grid data acquisition devices, software modules for big data management and large scale time series analysis. The power grid modeling and simulation modules enable to run power flow simulations. Visualization methods support data exploration for captured, simulated and analyzed energy data. A remote software control module for the proposed tools is provided

Public transport congestion detection using incremental learning

In the past decade, intelligent transportation systems have emerged as an efficient way of improving transportation services, while machine learning has been the key driver that created scopes for numerous innovations and improvements. Still, most machine learning approaches integrate paradigms that fell short of providing cost-effective and scalable solutions. This work employs long short-term memory to detect congestion by capturing the long-term temporal dependency for short-term public bus travel speed prediction to detect congestion. In contrast to existing methods, we implement our solution as incremental learning that is superior to traditional batch learning, enabling efficient and sustainable congestion detection. We examine the real-world efficacy of our prototype implementation in Pécs, the fifth largest city of Hungary, and observed that the incrementally updated model can detect congestion of up to 82.37. Additionally, we find our solution to evolve sufficiently over time, implying diverse real-world practicability. The findings emerging from this work can serve as a basis for future improvements to develop better public transportation congestion detection

Energy Analytics for Infrastructure: An Application to Institutional Buildings

abstract: Commercial buildings in the United States account for 19% of the total energy consumption annually. Commercial Building Energy Consumption Survey (CBECS), which serves as the benchmark for all the commercial buildings provides critical input for EnergyStar models. Smart energy management technologies, sensors, innovative demand response programs, and updated versions of certification programs elevate the opportunity to mitigate energy-related problems (blackouts and overproduction) and guides energy managers to optimize the consumption characteristics. With increasing advancements in technologies relying on the ‘Big Data,' codes and certification programs such as the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE), and the Leadership in Energy and Environmental Design (LEED) evaluates during the pre-construction phase. It is mostly carried out with the assumed quantitative and qualitative values calculated from energy models such as Energy Plus and E-quest. However, the energy consumption analysis through Knowledge Discovery in Databases (KDD) is not commonly used by energy managers to perform complete implementation, causing the need for better energy analytic framework. The dissertation utilizes Interval Data (ID) and establishes three different frameworks to identify electricity losses, predict electricity consumption and detect anomalies using data mining, deep learning, and mathematical models. The process of energy analytics integrates with the computational science and contributes to several objectives which are to 1. Develop a framework to identify both technical and non-technical losses using clustering and semi-supervised learning techniques. 2. Develop an integrated framework to predict electricity consumption using wavelet based data transformation model and deep learning algorithms. 3. Develop a framework to detect anomalies using ensemble empirical mode decomposition and isolation forest algorithms. With a thorough research background, the first phase details on performing data analytics on the demand-supply database to determine the potential energy loss reduction potentials. Data preprocessing and electricity prediction framework in the second phase integrates mathematical models and deep learning algorithms to accurately predict consumption. The third phase employs data decomposition model and data mining techniques to detect the anomalies of institutional buildings.Dissertation/ThesisDoctoral Dissertation Civil, Environmental and Sustainable Engineering 201

Load forecasting on the user‐side by means of computational intelligence algorithms

Nowadays, it would be very difficult to deny the need to prioritize sustainable development through energy efficiency at all consumption levels. In this context, an energy management system (EMS) is a suitable option for continuously improving energy efficiency, particularly on the user side. An EMS is a set of technological tools that manages energy consumption information and allows its analysis. EMS, in combination with information technologies, has given rise to intelligent EMS (iEMS), which, aside from lending support to monitoring and reporting functions as an EMS does, it has the ability to model, forecast, control and diagnose energy consumption in a predictive way. The main objective of an iEMS is to continuously improve energy efficiency (on-line) as automatically as possible. The core of an iEMS is its load modeling forecasting system (LMFS). It takes advantage of historical information on energy consumption and energy-related variables in order to model and forecast load profiles and, if available, generator profiles. These models and forecasts are the main information used for iEMS applications for control and diagnosis. That is why in this thesis we have focused on the study, analysis and development of LMFS on the user side. The fact that the LMFS is applied on the user side to support an iEMS means that specific characteristics are required that in other areas of load forecasting they are not. First of all, the user-side load profiles (LPs) have a higher random behavior than others, as for example, in power system distribution or generation. This makes the modeling and forecasting process more difficult. Second, on the user side --for example an industrial user-- there is a high number and variety of places that can be monitored, modeled and forecasted, as well as their precedence or nature. Thus, on the one hand, an LMFS requires a high degree of autonomy to automatically or autonomously generate the demanded models. And on the other hand, it needs a high level of adaptability in order to be able to model and forecast different types of loads and different types of energies. Therefore, the addressed LMFS are those that do not look only for accuracy, but also adaptability and autonomy. Seeking to achieve these objectives, in this thesis work we have proposed three novel LMFS schemes based on hybrid algorithms from computational intelligence, signal processing and statistical theory. The first of them looked to improve adaptability, keeping in mind the importance of accuracy and autonomy. It was called an evolutionary training algorithm (ETA) and is based on adaptivenetwork-based-fuzzy-inference system (ANFIS) that is trained by a multi-objective genetic algorithm instead of its traditional training algorithm. As a result of this hybrid, the generalization capacity was improved (avoiding overfitting) and an easily adaptable training algorithm for new adaptive networks based on traditional ANFIS was obtained. The second scheme deals with LMF autonomy in order to build models from multiple loads automatically. Similar to the previous proposal, an ANFIS and a MOGA were used. In this case, the MOGA was used to find a near-optimal configuration for the ANFIS instead of training it. The LMFS relies on this configuration to work properly, as well as to maintain accuracy and generalization capabilities. Real data from an industrial scenario were used to test the proposed scheme and the multi-site modeling and self-configuration results were satisfactory. Furthermore, other algorithms were satisfactorily designed and tested for processing raw data in outlier detection and gap padding. The last of the proposed approaches sought to improve accuracy while keeping autonomy and adaptability. It took advantage of dominant patterns (DPs) that have lower time resolution than the target LP, so they are easier to model and forecast. The Hilbert-Huang transform and Hilbert-spectral analysis were used for detecting and selecting the DPs. Those selected were used in a proposed scheme of partial models (PM) based on parallel ANFIS or artificial neural networks (ANN) to extract the information and give it to the main PM. Therefore, LMFS accuracy improved and the user-side LP noising problem was reduced. Additionally, in order to compensate for the added complexity, versions of self-configured sub-LMFS for each PM were used. This point was fundamental since, the better the configuration, the better the accuracy of the model; and subsequently the information provided to the main partial model was that much better. Finally, and to close this thesis, an outlook of trends regarding iEMS and an outline of several hybrid algorithms that are pending study and testing are presented.En el contexto energético actual y particularmente en el lado del usuario, el concepto de sistema de gestión energética (EMS) se presenta como una alternativa apropiada para mejorar continuamente la eficiencia energética. Los EMSs en combinación con las tecnologías informáticas dan origen al concepto de iEMS, que además de soportar las funciones de los EMS, tienen la capacidad de modelar, pronosticar, controlar y supervisar los consumos energéticos. Su principal objetivo es el de realizar una mejora continua, lo más autónoma posible y predictiva de la eficiencia energética. Este tipo de sistemas tienen como núcleo fundamental el sistema de modelado y pronóstico de consumos (Load Modeling and Forecasting System, LMFS). El LMFS está habilitado para pronosticar el comportamiento futuro de cargas y, si es necesario, de generadores. Es sobre estos pronósticos sobre los cuales el iEMS puede realizar sus tareas automáticas y predictivas de optimización y supervisión. Los LMFS en el lado del usuario son el foco de esta tesis. Un LMFS en el lado del usuario, diseñado para soportar un iEMS requiere o demanda ciertas características que en otros contextos no serían tan necesarias. En primera estancia, los perfiles de los usuarios tienen un alto grado de aleatoriedad que los hace más difíciles de pronosticar. Segundo, en el lado del usuario, por ejemplo en la industria, el gran número de puntos a modelar requiere que el LMFS tenga por un lado, un nivel elevado de autonomía para generar de la manera más desatendida posible los modelos. Por otro lado, necesita un nivel elevado de adaptabilidad para que, usando la misma estructura o metodología, pueda modelar diferentes tipos de cargas cuya procedencia pude variar significativamente. Por lo tanto, los sistemas de modelado abordados en esta tesis son aquellos que no solo buscan mejorar la precisión, sino también la adaptabilidad y autonomía. En busca de estos objetivos y soportados principalmente por algoritmos de inteligencia computacional, procesamiento de señales y estadística, hemos propuesto tres algoritmos novedosos para el desarrollo de un LMFS en el lado del usuario. El primero de ellos busca mejorar la adaptabilidad del LMFS manteniendo una buena precisión y capacidad de autonomía. Denominado ETA, consiste del uso de una estructura ANFIS que es entrenada por un algoritmo genético multi objetivo (MOGA). Como resultado de este híbrido, obtenemos un algoritmo con excelentes capacidades de generalización y fácil de adaptar para el entrenamiento y evaluación de nuevas estructuras adaptativas basadas en ANFIS. El segundo de los algoritmos desarrollados aborda la autonomía del LMFS para así poder generar modelos de múltiples cargas. Al igual que en la anterior propuesta usamos un ANFIS y un MOGA, pero esta vez el MOGA en vez de entrenar el ANFIS, se utiliza para encontrar la configuración cuasi-óptima del ANFIS. Encontrar la configuración apropiada de un ANFIS es muy importante para obtener un buen funcionamiento del LMFS en lo que a precisión y generalización respecta. El LMFS propuesto, además de configurar automáticamente el ANFIS, incluyó diversos algoritmos para procesar los datos puros que casi siempre estuvieron contaminados de datos espurios y gaps de información, operando satisfactoriamente en las condiciones de prueba en un escenario real. El tercero y último de los algoritmos buscó mejorar la precisión manteniendo la autonomía y adaptabilidad, aprovechando para ello la existencia de patrones dominantes de más baja resolución temporal que el consumo objetivo, y que son más fáciles de modelar y pronosticar. La metodología desarrollada se basa en la transformada de Hilbert-Huang para detectar y seleccionar tales patrones dominantes. Además, esta metodología define el uso de modelos parciales de los patrones dominantes seleccionados, para mejorar la precisión del LMFS y mitigar el problema de aleatoriedad que afecta a los consumos en el lado del usuario. Adicionalmente, se incorporó el algoritmo de auto configuración que se presentó en la propuesta anterior para hallar la configuración cuasi-óptima de los modelos parciales. Este punto fue crucial puesto que a mejor configuración de los modelos parciales mayor es la mejora en precisión del pronóstico final. Finalmente y para cerrar este trabajo de tesis, se realizó una prospección de las tendencias en cuanto al uso de iEMS y se esbozaron varias propuestas de algoritmos híbridos, cuyo estudio y comprobación se plantea en futuros estudios

A data mining framework for electricity consumption analysis from meter data

This paper presents a novel data mining framework for the exploration and extraction of actionable knowledge from data generated by electricity meters. Although a rich source of information for energy consumption analysis, electricity meters produce a voluminous, fast-paced, transient stream of data that conventional approaches are unable to address entirely. In order to overcome these issues, it is important for a data mining framework to incorporate functionality for interim summarization and incremental analysis using intelligent techniques. The proposed Incremental Summarization and Pattern Characterization (ISPC) framework demonstrates this capability. Stream data is structured in a data warehouse based on key dimensions enabling rapid interim summarization. Independently, the IPCL algorithm incrementally characterizes patterns in stream data and correlates these across time. Eventually, characterized patterns are consolidated with interim summarization to facilitate an overall analysis and prediction of energy consumption trends. Results of experiments conducted using the actual data from electricity meters confirm applicability of the ISPC framework

An investigation into the residential electricity demand, device use and activity patterns of older people in Reading, UK

The UK residential sector accounts for about one-third of overall electricity demand. The proportion and rhythm of this demand are projected to change largely due to projected changes in household characteristics such as demographic structure, energy-relevant practices and the changes in the ordering and structure of each day during the week. The UK population is ageing and future levels of electricity demand will depend not only on energy-efficiency gains, changes in the work-life length and structure, but also more on the activities of older consumers. The relationships between device use and activity patterns can shed light on the timing of residential electricity demand. This study analysed the timing of some energy-relevant practices enacted by over 65 year-olds in Reading, UK. The aim of this study is to investigate the pattern of the energy-relevant practices by older people. This study was carried out between November 2015 and February 2017 using meter readings and appliance-use diaries. These provided a snapshot of the timing of selected device (shared) use, the pattern of kWh demand and the frequency of use for other devices for seven consecutive days. From these, I developed energy intensity indexes and time slots to enable comparison of the most significant kWh readings within and across days. The frequencies of households using the selected devices were used to indicate the synchronicity (coming together) of device use. Meters were installed in participating households for a week during which the occupant(s) filled-in time use diaries for selected devices in a mostly 4-hour time ii slot and indicated the frequency of use for other appliances. The meter readings were recorded every 60 seconds and aggregated in alignment with the diary time slots. This enabled the determination of the variation and rhythm of electricity demand and electricity demand intensity. Frequencies were employed to describe the pattern of use of the selected devices. The findings from this study suggest that the single and two-person older households differ significantly in their pattern, intensity of electricity demand and device use. For instance, two-person households seem to have higher levels of routine and internal synchronisation than single households