Development of Neurofuzzy Architectures for Electricity Price Forecasting

In 20th century, many countries have liberalized their electricity market. This power markets liberalization has directed generation companies as well as wholesale buyers to undertake a greater intense risk exposure compared to the old centralized framework. In this framework, electricity price prediction has become crucial for any market player in their decision‐making process as well as strategic planning. In this study, a prototype asymmetric‐based neuro‐fuzzy network (AGFINN) architecture has been implemented for short‐term electricity prices forecasting for ISO New England market. AGFINN framework has been designed through two different defuzzification schemes. Fuzzy clustering has been explored as an initial step for defining the fuzzy rules while an asymmetric Gaussian membership function has been utilized in the fuzzification part of the model. Results related to the minimum and maximum electricity prices for ISO New England, emphasize the superiority of the proposed model over well‐established learning‐based models

Neuro-Fuzzy Based Intelligent Approaches to Nonlinear System Identification and Forecasting

Nearly three decades back nonlinear system identification consisted of several ad-hoc approaches, which were restricted to a very limited class of systems. However, with the advent of the various soft computing methodologies like neural networks and the fuzzy logic combined with optimization techniques, a wider class of systems can be handled at present. Complex systems may be of diverse characteristics and nature. These systems may be linear or nonlinear, continuous or discrete, time varying or time invariant, static or dynamic, short term or long term, central or distributed, predictable or unpredictable, ill or well defined. Neurofuzzy hybrid modelling approaches have been developed as an ideal technique for utilising linguistic values and numerical data. This Thesis is focused on the development of advanced neurofuzzy modelling architectures and their application to real case studies. Three potential requirements have been identified as desirable characteristics for such design: A model needs to have minimum number of rules; a model needs to be generic acting either as Multi-Input-Single-Output (MISO) or Multi-Input-Multi-Output (MIMO) identification model; a model needs to have a versatile nonlinear membership function. Initially, a MIMO Adaptive Fuzzy Logic System (AFLS) model which incorporates a prototype defuzzification scheme, while utilising an efficient, compared to the Takagi–Sugeno–Kang (TSK) based systems, fuzzification layer has been developed for the detection of meat spoilage using Fourier transform infrared (FTIR) spectroscopy. The identification strategy involved not only the classification of beef fillet samples in their respective quality class (i.e. fresh, semi-fresh and spoiled), but also the simultaneous prediction of their associated microbiological population directly from FTIR spectra. In the case of AFLS, the number of memberships for each input variable was directly associated to the number of rules, hence, the “curse of dimensionality” problem was significantly reduced. Results confirmed the advantage of the proposed scheme against Adaptive Neurofuzzy Inference System (ANFIS), Multilayer Perceptron (MLP) and Partial Least Squares (PLS) techniques used in the same case study. In the case of MISO systems, the TSK based structure, has been utilized in many neurofuzzy systems, like ANFIS. At the next stage of research, an Adaptive Fuzzy Inference Neural Network (AFINN) has been developed for the monitoring the spoilage of minced beef utilising multispectral imaging information. This model, which follows the TSK structure, incorporates a clustering pre-processing stage for the definition of fuzzy rules, while its final fuzzy rule base is determined by competitive learning. In this specific case study, AFINN model was also able to predict for the first time in the literature, the beef’s temperature directly from imaging information. Results again proved the superiority of the adopted model. By extending the line of research and adopting specific design concepts from the previous case studies, the Asymmetric Gaussian Fuzzy Inference Neural Network (AGFINN) architecture has been developed. This architecture has been designed based on the above design principles. A clustering preprocessing scheme has been applied to minimise the number of fuzzy rules. AGFINN incorporates features from the AFLS concept, by having the same number of rules as well as fuzzy memberships. In spite of the extensive use of the standard symmetric Gaussian membership functions, AGFINN utilizes an asymmetric function acting as input linguistic node. Since the asymmetric Gaussian membership function’s variability and flexibility are higher than the traditional one, it can partition the input space more effectively. AGFINN can be built either as an MISO or as an MIMO system. In the MISO case, a TSK defuzzification scheme has been implemented, while two different learning algorithms have been implemented. AGFINN has been tested on real datasets related to electricity price forecasting for the ISO New England Power Distribution System. Its performance was compared against a number of alternative models, including ANFIS, AFLS, MLP and Wavelet Neural Network (WNN), and proved to be superior. The concept of asymmetric functions proved to be a valid hypothesis and certainly it can find application to other architectures, such as in Fuzzy Wavelet Neural Network models, by designing a suitable flexible wavelet membership function. AGFINN’s MIMO characteristics also make the proposed architecture suitable for a larger range of applications/problems

Load forecast on a Micro Grid level through Machine Learning algorithms

As Micro Redes constituem um sector em crescimento da indústria energética, representando uma mudança de paradigma, desde as remotas centrais de geração até à produção mais localizada e distribuída. A capacidade de isolamento das principais redes elétricas e atuar de forma independente tornam as Micro Redes em sistemas resilientes, capazes de conduzir operações flexíveis em paralelo com a prestação de serviços que tornam a rede mais competitiva. Como tal, as Micro Redes fornecem energia limpa eficiente de baixo custo, aprimoram a coordenação dos ativos e melhoram a operação e estabilidade da rede regional de eletricidade, através da capacidade de resposta dinâmica aos recursos energéticos. Para isso, necessitam de uma coordenação de gestão inteligente que equilibre todas as tecnologias ao seu dispor. Daqui surge a necessidade de recorrer a modelos de previsão de carga e de produção robustos e de confiança, que interligam a alocação dos recursos da rede perante as necessidades emergentes. Sendo assim, foi desenvolvida a metodologia HALOFMI, que tem como principal objetivo a criação de um modelo de previsão de carga para 24 horas. A metodologia desenvolvida é constituída, numa primeira fase, por uma abordagem híbrida de multinível para a criação e escolha de atributos, que alimenta uma rede neuronal (Multi-Layer Perceptron) sujeita a um ajuste de híper-parâmetros. Posto isto, numa segunda fase são testados dois modos de aplicação e gestão de dados para a Micro Rede. A metodologia desenvolvida é aplicada em dois casos de estudo: o primeiro é composto por perfis de carga agregados correspondentes a dados de clientes em Baixa Tensão Normal e de Unidades de Produção e Autoconsumo (UPAC). Este caso de estudo apresenta-se como um perfil de carga elétrica regular e com contornos muito suaves. O segundo caso de estudo diz respeito a uma ilha turística e representa um perfil irregular de carga, com variações bruscas e difíceis de prever e apresenta um desafio maior em termos de previsão a 24-horas A partir dos resultados obtidos, é avaliado o impacto da integração de uma seleção recursiva inteligente de atributos, seguido por uma viabilização do processo de redução da dimensão de dados para o operador da Micro Rede, e por fim uma comparação de estimadores usados no modelo de previsão, através de medidores de erros na performance do algoritmo.Micro Grids constitute a growing sector of the energetic industry, representing a paradigm shift from the central power generation plans to a more distributed generation. The capacity to work isolated from the main electric grid make the MG resilient system, capable of conducting flexible operations while providing services that make the network more competitive. Additionally, Micro Grids supply clean and efficient low-cost energy, enhance the flexible assets coordination and improve the operation and stability of the of the local electric grid, through the capability of providing a dynamic response to the energetic resources. For that, it is required an intelligent coordination which balances all the available technologies. With this, rises the need to integrate accurate and robust load and production forecasting models into the MG management platform, thus allowing a more precise coordination of the flexible resource according to the emerging demand needs. For these reasons, the HALOFMI methodology was developed, which focus on the creation of a precise 24-hour load forecast model. This methodology includes firstly, a hybrid multi-level approach for the creation and selection of features. Then, these inputs are fed to a Neural Network (Multi-Layer Perceptron) with hyper-parameters tuning. In a second phase, two ways of data operation are compared and assessed, which results in the viability of the network operating with a reduced number of training days without compromising the model's performance. Such process is attained through a sliding window application. Furthermore, the developed methodology is applied in two case studies, both with 15-minute timesteps: the first one is composed by aggregated load profiles of Standard Low Voltage clients, including production and self-consumption units. This case study presents regular and very smooth load profile curves. The second case study concerns a touristic island and represents an irregular load curve with high granularity with abrupt variations. From the attained results, it is evaluated the impact of integrating a recursive intelligent feature selection routine, followed by an assessment on the sliding window application and at last, a comparison on the errors coming from different estimators for the model, through several well-defined performance metrics

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Iberian Energy Market: Spot Price Forecast by Modelling Market Offers

Electricity is a very special commodity since it is economically non-storable, and thus requiring a constant balance between production and consumption. At the corporate level, electricity price forecasts have become a fundamental input to energy companies’ decision making mechanisms [22, 45]. Electric utilities are higly vulnerable to economical crisis, since they generally cannot pass their excess costs on the wholesale market to the retail consumers [77] and, since the price depends on variables like weather (temperature, wind speed, precipitation, etc.) and the intensity of business and everyday activities (on-peak vs. off-peak hours, weekdays vs. weekends, holidays and near-holidays, etc.) it shows specific dynamics not observed in any other market, exhibiting seasonality at the daily, weekly and annual levels, and abrupt, short-lived and generally unanticipated price spikes. These extreme price volatility make price forecasts from a few hours to a few months ahead to become of particular interest to power portfolio managers. An utility company or large industrial consumer who is able to accurately forecast the wholesale prices and it’s volatility, can adjust its bidding strategy and its own production/consumption schedule in order to reduce the risk or maximize the profits in day-ahead trading. In this work I discuss the dynamics of the Iberian electricity day-ahead market (OMIE), review the state-of-the-art forecasting techniques and introduce a new approach to Electricity Price Forecasting, by forecasting the underlying dynamics, the market demand/supply curves. With this method it is possible to predict not only the electricity prices for the next hours, but also the market curves, which can then be used for risk management and a more accurate schedule of generation units. I analyze the model results and benchmark them against other models in the industry.A eletricidade é uma commodity muito especial, uma vez que não é possível armazená-la, e por isso, requer um constante equilíbrio entre a produção e consumo. ao nível empresarial, a previsão de preços de eletricidade tornou-se um input fundamental para os mecanismos de tomada de decisão das companhias [22, 45]. As empresas de eletricidade são altamente vulneráveis a crises económicas, uma vez que, em geral, não conseguem passar os seus custos excessivos para o mercado retalhista [77] e, uma vez que o preço depende de variáveis como meteorologia (temperatura, velocidade do vento, precipitação, etc.) e da intensidade de negócio e das atividades do dia-a-dia (pico vs vazio, dias da semana vs fim-de-semana, feriados e pontes, etc.) apresenta uma dinâmica que não é observada em mais nenhum mercado, com sazonalidade diária, semanal e anual, e com picos de preço abruptos de pouca duração e, em termos gerais, impossíveis de antecipar. Esta volatilidade de preços torna a previsão de preços particularmente interessante para gestores de portfólio, seja a curto ou a longo prazo. Uma companhia de eletricidade ou grande consumidor industrial que seja capaz de prever corretamente os preços do mercado grossista e a sua volatilidade, pode ajustar a estratégia de oferta da sua produção/seu consumo de maneira a reduzir o risco ou maximizar os ganhos no mercado à vista. Neste trabalho abordo a dinâmica do mercado de eletricidade ibérico (Operador de Mercado Iberico - Polo Español (OMIE)), revendo o estado da arte dos métodos de previsão de preços de eletricidade, e introduzo uma nova técnica de previsão de preços de eletricidade, através da previsão da sua dinâmica subjacente, as curvas de mercado da procura e oferta. Com este método é possível prever, não só o preço de eletricidade para as próximas horas, mas também as próprias curvas de oferta, o que pode ser utilizado na gestão de risco ao melhor a capacidade de programar as suas unidades de geração.Os resultados do modelo são analisados e comparados com outros modelos já utilizados na industria

Deep learning architectures applied to wind time series multi-step forecasting

Forecasting is a critical task for the integration of wind-generated energy into electricity grids. Numerical weather models applied to wind prediction, work with grid sizes too large to reproduce all the local features that influence wind, thus making the use of time series with past observations a necessary tool for wind forecasting. This research work is about the application of deep neural networks to multi-step forecasting using multivariate time series as an input, to forecast wind speed at 12 hours ahead. Wind time series are sequences of meteorological observations like wind speed, temperature, pressure, humidity, and direction. Wind series have two statistically relevant properties; non-linearity and non-stationarity, which makes the modelling with traditional statistical tools very inaccurate. In this thesis we design, test and validate novel deep learning models for the wind energy prediction task, applying new deep architectures to the largest open wind data repository available from the National Renewable Laboratory of the US (NREL) with 126,692 wind sites evenly distributed on the US geography. The heterogeneity of the series, obtained from several data origins, allows us to obtain conclusions about the level of fitness of each model to time series that range from highly stationary locations to variable sites from complex areas. We propose Multi-Layer, Convolutional and recurrent Networks as basic building blocks, and then combined into heterogeneous architectures with different variants, trained with optimisation strategies like drop and skip connections, early stopping, adaptive learning rates, filters and kernels of different sizes, between others. The architectures are optimised by the use of structured hyper-parameter setting strategies to obtain the best performing model across the whole dataset. The learning capabilities of the architectures applied to the various sites find relationships between the site characteristics (terrain complexity, wind variability, geographical location) and the model accuracy, establishing novel measures of site predictability relating the fit of the models with indexes from time series spectral or stationary analysis. The designed methods offer new, and superior, alternatives to traditional methods.La predicció de vent és clau per a la integració de l'energia eòlica en els sistemes elèctrics. Els models meteorològics es fan servir per predicció, però tenen unes graelles geogràfiques massa grans per a reproduir totes les característiques locals que influencien la formació de vent, fent necessària la predicció d'acord amb les sèries temporals de mesures passades d'una localització concreta. L'objectiu d'aquest treball d'investigació és l'aplicació de xarxes neuronals profundes a la predicció \textit{multi-step} utilitzant com a entrada series temporals de múltiples variables meteorològiques, per a fer prediccions de vent d'ací a 12 hores. Les sèries temporals de vent són seqüències d'observacions meteorològiques tals com, velocitat del vent, temperatura, humitat, pressió baromètrica o direcció. Les sèries temporals de vent tenen dues propietats estadístiques rellevants, que són la no linearitat i la no estacionalitat, que fan que la modelització amb eines estadístiques sigui poc precisa. En aquesta tesi es validen i proven models de deep learning per la predicció de vent, aquests models d'arquitectures d'autoaprenentatge s'apliquen al conjunt de dades de vent més gran del món, que ha produït el National Renewable Laboratory dels Estats Units (NREL) i que té 126,692 ubicacions físiques de vent distribuïdes per total la geografia de nord Amèrica. L'heterogeneïtat d'aquestes sèries de dades permet establir conclusions fermes en la precisió de cada mètode aplicat a sèries temporals generades en llocs geogràficament molt diversos. Proposem xarxes neuronals profundes de tipus multi-capa, convolucionals i recurrents com a blocs bàsics sobre els quals es fan combinacions en arquitectures heterogènies amb variants, que s'entrenen amb estratègies d'optimització com drops, connexions skip, estratègies de parada, filtres i kernels de diferents mides entre altres. Les arquitectures s'optimitzen amb algorismes de selecció de paràmetres que permeten obtenir el model amb el millor rendiment, en totes les dades. Les capacitats d'aprenentatge de les arquitectures aplicades a ubicacions heterogènies permet establir relacions entre les característiques d'un lloc (complexitat del terreny, variabilitat del vent, ubicació geogràfica) i la precisió dels models, establint mesures de predictibilitat que relacionen la capacitat dels models amb les mesures definides a partir d'anàlisi espectral o d'estacionalitat de les sèries temporals. Els mètodes desenvolupats ofereixen noves i superiors alternatives als algorismes estadístics i mètodes tradicionals.Arquitecturas de aprendizaje profundo aplicadas a la predición en múltiple escalón de series temporales de viento. La predicción de viento es clave para la integración de esta energía eólica en los sistemas eléctricos. Los modelos meteorológicos tienen una resolución geográfica demasiado amplia que no reproduce todas las características locales que influencian en la formación del viento, haciendo necesaria la predicción en base a series temporales de cada ubicación concreta. El objetivo de este trabajo de investigación es la aplicación de redes neuronales profundas a la predicción multi-step usando como entrada series temporales de múltiples variables meteorológicas, para realizar predicciones de viento a 12 horas. Las series temporales de viento son secuencias de observaciones meteorológicas tales como, velocidad de viento, temperatura, humedad, presión barométrica o dirección. Las series temporales de viento tienen dos propiedades estadísticas relevantes, que son la no linealidad y la no estacionalidad, lo que implica que su modelización con herramientas estadísticas sea poco precisa. En esta tesis se validan y verifican modelos de aprendizaje profundo para la predicción de viento, estos modelos de arquitecturas de aprendizaje automático se aplican al conjunto de datos de viento más grande del mundo, que ha sido generado por el National Renewable Laboratory de los Estados Unidos (NREL) y que tiene 126,682 ubicaciones físicas de viento distribuidas por toda la geografía de Estados Unidos. La heterogeneidad de estas series de datos permite establecer conclusiones válidas sobre la validez de cada método al ser aplicado en series temporales generadas en ubicaciones físicas muy diversas. Proponemos redes neuronales profundas de tipo multi capa, convolucionales y recurrentes como tipos básicos, sobre los que se han construido combinaciones en arquitecturas heterogéneas con variantes de entrenamiento como drops, conexiones skip, estrategias de parada, filtros y kernels de distintas medidas, entre otros. Las arquitecturas se optimizan con algoritmos de selección de parámetros que permiten obtener el mejor modelo buscando el mejor rendimiento, incluyendo todos los datos. Las capacidades de aprendizaje de las arquitecturas aplicadas a localizaciones físicas muy variadas permiten establecer relaciones entre las características de una ubicación (complejidad del terreno, variabilidad de viento, ubicación geográfica) y la precisión de los modelos, estableciendo medidas de predictibilidad que relacionan la capacidad de los algoritmos con índices que se definen a partir del análisis espectral o de estacionalidad de las series temporales. Los métodos desarrollados ofrecen nuevas alternativas a los algoritmos estadísticos tradicionales.Postprint (published version

Short Term Load Forecasting for Smart Grids Using Apache Spark and a Modified Transformer Model

Smart grid is an advanced electrical grid that enables more efficient distribution of electricity. It counters many of the problems presented by renewable energy sources such as variability in production through techniques like load forecasting and dynamic pricing. Smart grid generates massive amounts of data through smart meters, this data is used to forecast future load to adjust distribution. To process all this data, big data analysis is necessary. Most existing schemes use Apache Hadoop for big data processing and various techniques for load forecasting that include methods based on statistical theory, machine learning and deep learning. This paper proposes using Apache Spark for big data analysis and a modified version of the transformer model for forecasting load profiles of households. The modified transformer model has been tested against several state-of-the-art machine learning models. The proposed scheme was tested against several baseline and state-of-the-art machine learning models and evaluated in terms of the RMSE, MAE, MedAE and R2 scores. The obtained results show that the proposed model has better performance in terms of RMSE and R2 which are the preferred metrics when evaluating a regression model on data with a large number of outliers