Machine Learning Models and Intra-Daily Market Information for the Prediction of Italian Electricity Prices

In this paper we assess how intra-day electricity prices can improve the prediction of zonal day-ahead wholesale electricity prices in Italy. We consider linear autoregressive models with exogenous variables (ARX) with and without interactions among predictors, and non-parametric models taken from the machine learning literature. In particular, we implement Random Forests and support vector machines, which should automatically capture the relevant interactions among predictors. Given the large number of predictors, ARX models are also estimated using LASSO regularization, which improves predictions when regressors are many and selects the important variables. In addition to zonal intra-day prices, among the predictors we include also the official demand forecasts and wind generation expectations. Our results show that the prediction performance of the simple ARX model is mostly superior to those of machine learning models. The analysis of the relevance of exogenous variables, using variable importance measures, reveals that intra-day market information successfully contributes to the forecasting performance, although the impact differs among the estimated models

Development of Neurofuzzy Architectures for Electricity Price Forecasting

In 20th century, many countries have liberalized their electricity market. This power markets liberalization has directed generation companies as well as wholesale buyers to undertake a greater intense risk exposure compared to the old centralized framework. In this framework, electricity price prediction has become crucial for any market player in their decision‐making process as well as strategic planning. In this study, a prototype asymmetric‐based neuro‐fuzzy network (AGFINN) architecture has been implemented for short‐term electricity prices forecasting for ISO New England market. AGFINN framework has been designed through two different defuzzification schemes. Fuzzy clustering has been explored as an initial step for defining the fuzzy rules while an asymmetric Gaussian membership function has been utilized in the fuzzification part of the model. Results related to the minimum and maximum electricity prices for ISO New England, emphasize the superiority of the proposed model over well‐established learning‐based models

Analysis of the electric power outage data and prediction of electric power outage for major metropolitan areas in Texas using Machine Learning and Time Series Methods

With growing energy usage, power outages affect millions of households. This case study focuses on gathering power outage historical data, modifying the data to attach weather attributes, and gathering ERCOT energy market conditions for Dallas-Fort Worth and Houston metropolitan areas of Texas. The transformed data is then analyzed using machine learning algorithms including, but not limited to, Regression, Random Forests and XGBoost to consider current weather and ERCOT features and predict power outage percentage for locations. The transformed data is also trained using time series models and serially correlated models including Autoregression and Vector Autoregression. This study also focuses on traditional machine learning models that assume sample independence when compared to those that assume serial correlation. The results show machine learning models that utilize both weather features and ERCOT data yield a lower RMSE and higher prediction accuracy than using one feature-set exclusively. In addition, multivariate Vector Autoregressive models have lower RMSE compared to univariate Auto-Regressive, univariate Random Forest and univariate neural network models when weather and ERCOT data are included to predict power outages. Top performing traditional machine learning models are packaged into an external facing web application for public use in determining current power outage risk

Ensemble prediction model with expert selection for electricity price forecasting

Forecasting of electricity prices is important in deregulated electricity markets for all of the stakeholders: energy wholesalers, traders, retailers and consumers. Electricity price forecasting is an inherently difficult problem due to its special characteristic of dynamicity and non-stationarity. In this paper, we present a robust price forecasting mechanism that shows resilience towards the aggregate demand response effect and provides highly accurate forecasted electricity prices to the stakeholders in a dynamic environment. We employ an ensemble prediction model in which a group of different algorithms participates in forecasting 1-h ahead the price for each hour of a day. We propose two different strategies, namely, the Fixed Weight Method (FWM) and the Varying Weight Method (VWM), for selecting each hour’s expert algorithm from the set of participating algorithms. In addition, we utilize a carefully engineered set of features selected from a pool of features extracted from the past electricity price data, weather data and calendar data. The proposed ensemble model offers better results than the Autoregressive Integrated Moving Average (ARIMA) method, the Pattern Sequence-based Forecasting (PSF) method and our previous work using Artificial Neural Networks (ANN) alone on the datasets for New York, Australian and Spanish electricity markets

Machine Learning Applications for Load Predictions in Electrical Energy Network

In this work collected operational data of typical urban and rural energy network are analysed for predictions of energy consumption, as well as for selected region of Nordpool electricity markets. The regression techniques are systematically investigated for electrical energy prediction and correlating other impacting parameters. The k-Nearest Neighbour (kNN), Random Forest (RF) and Linear Regression (LR) are analysed and evaluated both by using continuous and vertical time approach. It is observed that for 30 minutes predictions the RF Regression has the best results, shown by a mean absolute percentage error (MAPE) in the range of 1-2 %. kNN show best results for the day-ahead forecasting with MAPE of 2.61 %. The presented vertical time approach outperforms the continuous time approach. To enhance pre-processing stage, refined techniques from the domain of statistics and time series are adopted in the modelling. Reducing the dimensionality through principal components analysis improves the predictive performance of Recurrent Neural Networks (RNN). In the case of Gated Recurrent Units (GRU) networks, the results for all the seasons are improved through principal components analysis (PCA). This work also considers abnormal operation due to various instances (e.g. random effect, intrusion, abnormal operation of smart devices, cyber-threats, etc.). In the results of kNN, iforest and Local Outlier Factor (LOF) on urban area data and from rural region data, it is observed that the anomaly detection for the scenarios are different. For the rural region, most of the anomalies are observed in the latter timeline of the data concentrated in the last year of the collected data. For the urban area data, the anomalies are spread out over the entire timeline. The frequency of detected anomalies where considerably higher for the rural area load demand than for the urban area load demand. Observing from considered case scenarios, the incidents of detected anomalies are more data driven, than exceptions in the algorithms. It is observed that from the domain knowledge of smart energy systems the LOF is able to detect observations that could not have detected by visual inspection alone, in contrast to kNN and iforest. Whereas kNN and iforest excludes an upper and lower bound, the LOF is density based and separates out anomalies amidst in the data. The capability that LOF has to identify anomalies amidst the data together with the deep domain knowledge is an advantage, when detecting anomalies in smart meter data. This work has shown that the instance based models can compete with models of higher complexity, yet some methods in preprocessing (such as circular coding) does not function for an instance based learner such as k-Nearest Neighbor, and hence kNN can not option for this kind of complexity even in the feature engineering of the model. It will be interesting for the future work of electrical load forecasting to develop solution that combines a high complexity in the feature engineering and have the explainability of instance based models.publishedVersio

A new framework for electricity price forecasting via multi-head self-attention and CNN-based techniques in the competitive electricity market

Due to recent technical improvements, the smart grid has become a feasible platform for electricity market participants to successfully regulate their bidding process based on demand-side management (DSM) perspectives. At this level, practical design, implementation, and assessment of numerous demand response mechanisms and robust short-term price forecasting development in day-ahead transactions are all critical. The accuracy and effectiveness of the day-ahead price forecasting process are crucial concerns in a deregulated market. In this market, the reason for low accuracy is the limitation of electricity generation compared to the electricity demand variations. Hence, this study proposes a suitable technique for forecasting electricity prices using a multi-head self-attention and Convolutional Neural networks (CNN) based approach. Further, this study develops a feature selection technique using mutual information (MI) and neural networks (NN) to choose suitable input variable subsets significantly affecting electricity price predictions simultaneously. The combination of MI and NN reduces the number of input features used in the model, thereby decreasing the computational complexity of the NN. The actual data sets from the Ontario electricity market in 2020 are acquired to verify the simulation results. Finally, the simulation results proved the efficiency of the proposed method by demonstrating increased accuracy by attaining the lowest average value for MAPE and RMSE with a value of 1.75% and 0.0085, respectively, and compared to results obtained by recent computational intelligence approaches. By attaining accurate electricity price results, the significance of this study can be summed up as aiding the electricity industry's operators in administering effective energy management, efficient resource allocation, and informed decision-making.© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).fi=vertaisarvioitu|en=peerReviewed

Relative evaluation of regression tools for urban area electrical energy demand forecasting

Load forecasting is the most fundamental application in Smart-Grid, which provides essential input to Demand Response, Topology Optimization and Abnormally Detection, facilitating the integration of intermittent clean energy sources. In this work, several regression tools are analyzed using larger datasets for urban area electrical load forecasting. The regression tools which are used are Random Forest Regressor, k-Nearest Neighbour Regressor and Linear Regressor. This work explores the use of regression tool for regional electric load forecasting by correlating lower distinctive categorical level (season, day of the week) and weather parameters. The regression analysis has been done on continuous time basis as well as vertical time axis approach. The vertical time approach is considering a sample time period (e.g seasonally and weekly) of data for four years and has been tested for the same time period for the consecutive year. This work has uniqueness in electrical demand forecasting using regression tools through vertical approach and it also considers the impact of meteorological parameters. This vertical approach uses less amount of data compare to continuous time-series as well as neural network techniques. A correlation study, where both the Pearson method and visual inspection, of the vertical approach depicts meaningful relation between pre-processing of data, test methods and results, for the regressors examined through Mean Absolute Percentage Error (MAPE). By examining the structure of various regressors they are compared for the lowest MAPE. Random Forest Regressor provides better short-term load prediction (30 min) and kNN offers relatively better long-term load prediction (24 h).acceptedVersio

Iberian Energy Market: Spot Price Forecast by Modelling Market Offers

Electricity is a very special commodity since it is economically non-storable, and thus requiring a constant balance between production and consumption. At the corporate level, electricity price forecasts have become a fundamental input to energy companies’ decision making mechanisms [22, 45]. Electric utilities are higly vulnerable to economical crisis, since they generally cannot pass their excess costs on the wholesale market to the retail consumers [77] and, since the price depends on variables like weather (temperature, wind speed, precipitation, etc.) and the intensity of business and everyday activities (on-peak vs. off-peak hours, weekdays vs. weekends, holidays and near-holidays, etc.) it shows specific dynamics not observed in any other market, exhibiting seasonality at the daily, weekly and annual levels, and abrupt, short-lived and generally unanticipated price spikes. These extreme price volatility make price forecasts from a few hours to a few months ahead to become of particular interest to power portfolio managers. An utility company or large industrial consumer who is able to accurately forecast the wholesale prices and it’s volatility, can adjust its bidding strategy and its own production/consumption schedule in order to reduce the risk or maximize the profits in day-ahead trading. In this work I discuss the dynamics of the Iberian electricity day-ahead market (OMIE), review the state-of-the-art forecasting techniques and introduce a new approach to Electricity Price Forecasting, by forecasting the underlying dynamics, the market demand/supply curves. With this method it is possible to predict not only the electricity prices for the next hours, but also the market curves, which can then be used for risk management and a more accurate schedule of generation units. I analyze the model results and benchmark them against other models in the industry.A eletricidade é uma commodity muito especial, uma vez que não é possível armazená-la, e por isso, requer um constante equilíbrio entre a produção e consumo. ao nível empresarial, a previsão de preços de eletricidade tornou-se um input fundamental para os mecanismos de tomada de decisão das companhias [22, 45]. As empresas de eletricidade são altamente vulneráveis a crises económicas, uma vez que, em geral, não conseguem passar os seus custos excessivos para o mercado retalhista [77] e, uma vez que o preço depende de variáveis como meteorologia (temperatura, velocidade do vento, precipitação, etc.) e da intensidade de negócio e das atividades do dia-a-dia (pico vs vazio, dias da semana vs fim-de-semana, feriados e pontes, etc.) apresenta uma dinâmica que não é observada em mais nenhum mercado, com sazonalidade diária, semanal e anual, e com picos de preço abruptos de pouca duração e, em termos gerais, impossíveis de antecipar. Esta volatilidade de preços torna a previsão de preços particularmente interessante para gestores de portfólio, seja a curto ou a longo prazo. Uma companhia de eletricidade ou grande consumidor industrial que seja capaz de prever corretamente os preços do mercado grossista e a sua volatilidade, pode ajustar a estratégia de oferta da sua produção/seu consumo de maneira a reduzir o risco ou maximizar os ganhos no mercado à vista. Neste trabalho abordo a dinâmica do mercado de eletricidade ibérico (Operador de Mercado Iberico - Polo Español (OMIE)), revendo o estado da arte dos métodos de previsão de preços de eletricidade, e introduzo uma nova técnica de previsão de preços de eletricidade, através da previsão da sua dinâmica subjacente, as curvas de mercado da procura e oferta. Com este método é possível prever, não só o preço de eletricidade para as próximas horas, mas também as próprias curvas de oferta, o que pode ser utilizado na gestão de risco ao melhor a capacidade de programar as suas unidades de geração.Os resultados do modelo são analisados e comparados com outros modelos já utilizados na industria