Transformer Training Strategies for Forecasting Multiple Load Time Series

In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy

Efficient use of deep learning and machine learning for load forecasting in South African power distribution networks

Abstract: Load forecasting, which is the act of anticipating future loads, has been shown to be important in power system network planning, operations and maintenance. Artificial Intelligence (AI) techniques have been shown to be good tools for load forecasting. Load forecasting can assist power distribution utilities maximise their revenue through optimising maintenance planning. With the dawn of the smart grid, first world countries have moved past the customer’s point of supply and use smart meters to forecast customer loads. These recent studies also utilise recent state of the art AI techniques such as deep learning techniques. Weather parameters are such as temperature, humidity and rainfall are usually used as parameters in these studies. South African load forecasting studies are outdated and recent studies are limited. Most of these studies are from 2010, and dating backwards to 1999. Hence they do not use recent state of the art AI techniques. The studies do not focus at distribution level load forecasting for optimal maintenance planning. The impact of adjusting power consumption data when there are spikes and dips in the data was not investigated in all these South African studies. These studies did not investigate the impact of weather parameters on different South African loads and hence load forecasting performance...D.Phil. (Electrical and Electronic Management

Inductive Transfer and Deep Neural Network Learning-Based Cross-Model Method for Short-Term Load Forecasting in Smarts Grids

In a real-world scenario of load forecasting, it is crucial to determine the energy consumption in electrical networks. The energy consumption data exhibit high variability between historical data and newly arriving data streams. To keep the forecasting models updated with the current trends, it is important to fine-tune the models in a timely manner. This article proposes a reliable inductive transfer learning (ITL) method, to use the knowledge from existing deep learning (DL) load forecasting models, to innovatively develop highly accurate ITL models at a large number of other distribution nodes reducing model training time. The outlier-insensitive clustering-based technique is adopted to group similar distribution nodes into clusters. ITL is considered in the setting of homogeneous inductive transfer. To solve overfitting that exists with ITL, a novel weight regularized optimization approach is implemented. The proposed novel cross-model methodology is evaluated on a real-world case study of 1000 distribution nodes of an electrical grid for one-day ahead hourly forecasting. Experimental results demonstrate that overfitting and negative learning in ITL can be avoided by the dissociated weight regularization (DWR) optimizer and that the proposed methodology delivers a reduction in training time by almost 85.6% and has no noticeable accuracy losses.Peer reviewe

Improving Clustering-Based Forecasting of Aggregated Distribution Transformer Loadings With Gradient Boosting and Feature Selection

Load forecasting is more important than ever to enable new monitor and control functionalities of distribution networks aiming to mitigate the impact of the energy transition. Load forecasting at medium voltage (MV) level is becoming more challenging, because these load profiles become more stochastic due to the increasing penetration of photovoltaic (PV) generation in distribution networks. This work combines medium to low voltage (MV/LV) transformer loadings measured with advanced metering infrastructure (AMI) and machine learning (ML) algorithms to propose a new clustering based day-ahead aggregated load forecasting approach. This four-step approach improves the day-ahead load forecast of a city. First, MV/LV transformer loadings are clustered based on the shape of their load pattern. Second, a gradient boosting algorithm is used to forecast the load of each cluster and calculate the related feature importance. Third, feature selection is applied to improve the forecast accuracy of each cluster. Finally, the day-ahead load forecast of all clusters are aggregated. The case study presented uses 519 measured MV/LV transformer loadings in a city to perform 30 day-ahead load forecasts. Compared against the day-ahead aggregated load forecast without clustering, the average normalized root mean squared error (NRMSE) reduced 12.7 %, the average mean absolute percentage error (MAPE) reduced 18.2 % and the average Pearson Correlation Coefficient (PCC) increased 0.37 %. The 95 % confidence interval of the difference between the average NRMSE, MAPE and PCC without clustering and with the proposed method indicates a statistically significant improvement in accuracy

Multi-phase state estimation featuring industrial-grade distribution network models

This paper proposes a novel implementation of a multi-phase distribution network state estimator which employs industrial-grade modeling of power components and measurements. Unlike the classical voltage-based and current-based state estimators, this paper presents the implementation details of a constrained weighted least squares state calculation method that includes standard three-phase state estimation capabilities in addition to practical modeling requirements from the industry; these requirements comprise multi-phase line configurations, unsymmetrical and incomplete transformer connections, power measurements on 4-connected loads, cumulative-type power measurements, line-to-line voltage magnitude measurements, and reversible line drop compensators. The enhanced modeling equips the estimator with capabilities that make it superior to a recently presented state-of-the-art distribution network load estimator that is currently used in real-life distribution management systems; comparative performance results demonstrate the advantage of the proposed estimator under practical measurement schemes

Application of Artificial Neural Networks for Power Load Prediction in Critical Infrastructure: A Comparative Case Study

This article aims to assess the effectiveness of state-of-the-art artificial neural network (ANN) models in time series analysis, specifically focusing on their application in prediction tasks of critical infrastructures (CIs). To accomplish this, shallow models with nearly identical numbers of trainable parameters are constructed and examined. The dataset, which includes 120,884 hourly electricity consumption records, is divided into three subsets (25%, 50%, and the entire dataset) to examine the effect of increasing training data. Additionally, the same models are trained and evaluated for univariable and multivariable data to evaluate the impact of including more features. The case study specifically focuses on predicting electricity consumption using load information from Norway. The results of this study confirm that LSTM models emerge as the best-performed model, surpassing other models as data volume and feature increase. Notably, for training datasets ranging from 2000 to 22,000 instances, GRU exhibits superior accuracy, while in the 22,000 to 42,000 range, LSTM and BiLSTM are the best. When the training dataset is within 42,000 to 360,000, LSTM and ConvLSTM prove to be good choices in terms of accuracy. Convolutional-based models exhibit superior performance in terms of computational efficiency. The convolutional 1D univariable model emerges as a standout choice for scenarios where training time is critical, sacrificing only 0.000105 in accuracy while a threefold improvement in training time is gained. For training datasets lower than 22,000, feature inclusion does not enhance any of the ANN model’s performance. In datasets exceeding 22,000 instances, ANN models display no consistent pattern regarding feature inclusion, though LSTM, Conv1D, Conv2D, ConvLSTM, and FCN tend to benefit. BiLSTM, GRU, and Transformer do not benefit from feature inclusion, regardless of the training dataset size. Moreover, Transformers exhibit inefficiency in time series forecasting due to their permutation-invariant self-attention mechanism, neglecting the crucial role of sequence order, as evidenced by their poor performance across all three datasets in this study. These results provide valuable insights into the capabilities of ANN models and their effective usage in the context of CI prediction tasks.publishedVersio

A comparative assessment of deep learning models for day-ahead load forecasting: Investigating key accuracy drivers

Short-term load forecasting (STLF) is vital for the effective and economic operation of power grids and energy markets. However, the non-linearity and non-stationarity of electricity demand as well as its dependency on various external factors renders STLF a challenging task. To that end, several deep learning models have been proposed in the literature for STLF, reporting promising results. In order to evaluate the accuracy of said models in day-ahead forecasting settings, in this paper we focus on the national net aggregated STLF of Portugal and conduct a comparative study considering a set of indicative, well-established deep autoregressive models, namely multi-layer perceptrons (MLP), long short-term memory networks (LSTM), neural basis expansion coefficient analysis (N-BEATS), temporal convolutional networks (TCN), and temporal fusion transformers (TFT). Moreover, we identify factors that significantly affect the demand and investigate their impact on the accuracy of each model. Our results suggest that N-BEATS consistently outperforms the rest of the examined models. MLP follows, providing further evidence towards the use of feed-forward networks over relatively more sophisticated architectures. Finally, certain calendar and weather features like the hour of the day and the temperature are identified as key accuracy drivers, providing insights regarding the forecasting approach that should be used per case.Comment: Keywords: Short-Term Load Forecasting, Deep Learning, Ensemble, N-BEATS, Temporal Convolution, Forecasting Accurac