Robust and automatic data cleansing method for short-term load forecasting of distribution feeders

Distribution networks are undergoing fundamental changes at medium voltage level. To support growing planning and control decision-making, the need for large numbers of short-term load forecasts has emerged. Data-driven modelling of medium voltage feeders can be affected by (1) data quality issues, namely, large gross errors and missing observations (2) the presence of structural breaks in the data due to occasional network reconfiguration and load transfers. The present work investigates and reports on the effects of advanced data cleansing techniques on forecast accuracy. A hybrid framework to detect and remove outliers in large datasets is proposed; this automatic procedure combines the Tukey labelling rule and the binary segmentation algorithm to cleanse data more efficiently, it is fast and easy to implement. Various approaches for missing value imputation are investigated, including unconditional mean, Hot Deck via k-nearest neighbour and Kalman smoothing. A combination of the automatic detection/removal of outliers and the imputation methods mentioned above are implemented to cleanse time series of 342 medium-voltage feeders. A nested rolling-origin-validation technique is used to evaluate the feed-forward deep neural network models. The proposed data cleansing framework efficiently removes outliers from the data, and the accuracy of forecasts is improved. It is found that Hot Deck (k-NN) imputation performs best in balancing the bias-variance trade-off for short-term forecasting

Robust data cleaning procedure for large scale medium voltage distribution networks feeders

Relatively little attention has been given to the short-term load forecasting problem of primary substations mainly because load forecasts were not essential to secure the operation of passive distribution networks. With the increasing uptake of intermittent generations, distribution networks are becoming active since power flows can change direction in a somewhat volatile fashion. The volatility of power flows introduces operational constraints on voltage control, system fault levels, thermal constraints, systems losses and high reverse power flows. Today, greater observability of the networks is essential to maintain a safe overall system and to maximise the utilisation of existing assets. Hence, to identify and anticipate for any forthcoming critical operational conditions, networks operators are compelled to broaden their visibility of the networks to time horizons that include not only real-time information but also hour-ahead and day-ahead forecasts. With this change in paradigm, progressively, large scales of short-term load forecasters is integrated as an essential component of distribution networks' control and planning tools. The data acquisition of large scale real-world data is prone to errors; anomalies in data sets can lead to erroneous forecasting outcomes. Hence, data cleansing is an essential first step in data-driven learning techniques. Data cleansing is a labour-intensive and time-consuming task for the following reasons: 1) to select a suitable cleansing method is not trivial 2) to generalise or automate a cleansing procedure is challenging, 3) there is a risk to introduce new errors in the data. This thesis attempts to maximise the performance of large scale forecasting models by addressing the quality of the modelling data. Thus, the objectives of this research are to identify the bad data quality causes, design an automatic data cleansing procedure suitable for large scale distribution network datasets and, to propose a rigorous framework for modelling MV distribution network feeders time series with deep learning architecture. The thesis discusses in detail the challenges in handling and modelling real-world distribution feeders time series. It also discusses a robust technique to detect outliers in the presence of level-shifts, and suitable missing values imputation techniques. All the concepts have been demonstrated on large real-world distribution network data.Open Acces

Comparison of Machine Learning Methods for Electricity Demand Forecasting in Bosnia and Herzegovina

Electricity demand forecasting is one of the most important components in the power system analysis. Furthermore, it is difficult and complicated process to forecast energy consumption. This study deals with modeling of the electrical energy consumption in Bosnia and Herzegovina in order to forecast future consumption of electrical loads based on temperature variables using machine learning methods. We used three different  machine learning methods for analyzing short term forecasting. The methods were trained using historical load data, collected from JP Elektroprivreda electrical power utility in BiH, and also considering weather data which is known to have a big impact on the use of electric power. Comparing the results it was seen that prediction for 500 hours is pretty good in range from 92,92% for reactive power till 98.84% for active power. Four different parameters were analyzed mean absolute error, root mean squared error, relative absolute error and root relative square error. The best results for apparent power were gotten with linear regression and are presented as for mean absolute error 9.84, root mean squared error 13.62, relative absolute error 14.06%, root relative squared error 14.39%. It is also seen from the results that,  the short term power consumption can be predicted which is important for maintaining of the voltage at the consumer side

A Novel Hybrid Feature Selection Method for Day-Ahead Electricity Price Forecasting

The paper proposes a novel hybrid feature selection (FS) method for day-ahead electricity price forecasting. The work presents a novel hybrid FS algorithm for obtaining optimal feature set to gain optimal forecast accuracy. The performance of the proposed forecaster is compared with forecasters based on classification tree and regression tree. A hybrid FS method based on the elitist genetic algorithm (GA) and a tree-based method is applied for FS. Making use of selected features, aperformance test of the forecaster was carried out to establish the usefulness of the proposed approach. By way of analyzing and forecasts for day-ahead electricity prices in the Australian electricity markets, the proposed approach is evaluated and it has been established that, with the selected feature, the proposed forecaster consistently outperforms the forecaster with a larger feature set. The proposed method is simulated in MATLAB and WEKA software.publishedVersio

Machine learning techniques for sensor-based household activity recognition and forecasting

Thanks to the recent development of cheap and unobtrusive smart-home sensors, ambient assisted living tools promise to offer innovative solutions to support the users in carrying out their everyday activities in a smoother and more sustainable way. To be effective, these solutions need to constantly monitor and forecast the activities of daily living carried out by the inhabitants. The Machine Learning field has seen significant advancements in the development of new techniques, especially regarding deep learning algorithms. Such techniques can be successfully applied to household activity signal data to benefit the user in several applications. This thesis therefore aims to produce a contribution that artificial intelligence can make in the field of activity recognition and energy consumption. The effective recognition of common actions or the use of high-consumption appliances would lead to user profiling, thus enabling the optimisation of energy consumption in favour of the user himself or the energy community in general. Avoiding wasting electricity and optimising its consumption is one of the main objectives of the community. This work is therefore intended as a forerunner for future studies that will allow, through the results in this thesis, the creation of increasingly intelligent systems capable of making the best use of the user's resources for everyday life actions. Namely, this thesis focuses on signals from sensors installed in a house: data from position sensors, door sensors, smartphones or smart meters, and investigates the use of advanced machine learning algorithms to recognize and forecast inhabitant activities, including the use of appliances and the power consumption. The thesis is structured into four main chapters, each of which represents a contribution regarding Machine Learning or Deep Learning techniques for addressing challenges related to the aforementioned data from different sources. The first contribution highlights the importance of exploiting dimensionality reduction techniques that can simplify a Machine Learning model and increase its efficiency by identifying and retaining only the most informative and predictive features for activity recognition. In more detail, it is presented an extensive experimental study involving several feature selection algorithms and multiple Human Activity Recognition benchmarks containing mobile sensor data. In the second contribution, we propose a machine learning approach to forecast future energy consumption considering not only past consumption data, but also context data such as inhabitants’ actions and activities, use of household appliances, interaction with furniture and doors, and environmental data. We performed an experimental evaluation with real-world data acquired in an instrumented environment from a large user group. Finally, the last two contributions address the Non-Intrusive-Load-Monitoring problem. In one case, the aim is to identify the operating state (on/off) and the precise energy consumption of individual electrical loads, considering only the aggregate consumption of these loads as input. We use a Deep Learning method to disaggregate the low-frequency energy signal generated directly by the new generation smart meters being deployed in Italy, without the need for additional specific hardware. In the other case, driven by the need to build intelligent non-intrusive algorithms for disaggregating electrical signals, the work aims to recognize which appliance is activated by analyzing energy measurements and classifying appliances through Machine Learning techniques. Namely, we present a new way of approaching the problem by unifying Single Label (single active appliance recognition) and Multi Label (multiple active appliance recognition) learning paradigms. This combined approach, supplemented with an event detector, which suggests the instants of activation, would allow the development of an end-to-end NILM approach

Time Series Prediction Evolving Voronoi Regions

Time series prediction is a complex problem that consists of forecasting the future behavior of a set of data with the only information of the previous data. The main problem is the fact that most of the time series that represent real phenomena include local behaviors that cannot be modelled by global approaches. This work presents a new procedure able to find predictable local behaviors, and thus, attaining a better level of total prediction. This new method is based on a division of the input space into Voronoi regions by means of Evolution Strategies. Our method has been tested using different time series domains. One of them that represents the water demand in a water tank, through a long period of time. The other two domains are well known examples of chaotic time series (Mackey-Glass) and natural phenomenon time series (Sunspot). Results prove that, in most of cases, the proposed algorithm obtain better results than other algorithms commonly used.Publicad

Potential challenges : integrating renewable energy with the smart grid

This is the published version

Advanced Control for Energy Management of Grid-Connected Hybrid Power Systems in the Sugar Cane Industry

This work presents a process supervision and advanced control structure, based on Model Predictive Control (MPC) coupled with disturbance estimation techniques and a finite-state machine decision system, responsible for setting energy productions set-points. This control scheme is applied to energy generation optimization in a sugar cane power plant, with non-dispatchable renewable sources, such as photovoltaic and wind power generation, as well as dispatchable sources, as biomass. The energy plant is bound to produce steam in different pressures, cold water and, imperiously, has to produce and maintain an amount of electric power throughout each month, defined by contract rules with a local distribution network operator (DNO). The proposed predictive control structure uses feedforward compensation of estimated future disturbances, obtained by the Double Exponential Smoothing (DES) method. The control algorithm has the task of performing the management of which energy system to use, maximize the use of the renewable energy sources, manage the use of energy storage units and optimize energy generation due to contract rules, while aiming to maximize economic profits. Through simulation, the proposed system is compared to a MPC structure, with standard techniques, and shows improved behavior.Ministerio de Economía y Competitividad CNPq401126/2014-5Ministerio de Economía y Competitividad CNPq303702/2011-7Ministerio de Economía y Competitividad DPI2016-78338-