Using Bayesian deep learning to capture uncertainty for residential net load forecasting

Decarbonization of electricity systems drives significant and continued investments in distributed energy sources to support the cost-effective transition to low-carbon energy systems. However, the rapid integration of distributed photovoltaic (PV) generation presents great challenges in obtaining reliable and secure grid operations because of its limited visibility and intermittent nature. Under this reality, net load forecasting is facing unprecedented difficulty in answering the following question: how can we accurately predict the net load while capturing the massive uncertainties arising from distributed PV generation and load, especially in the context of high PV penetration? This paper proposes a novel probabilistic day-ahead net load forecasting method to capture both epistemic uncertainty and aleatoric uncertainty using Bayesian deep learning, which is a new field that combines Bayesian probability theory and deep learning. The proposed methodological framework employs clustering in subprofiles and considers residential rooftop PV outputs as input features to enhance the performance of aggregated net load forecasting. Numerical experiments have been carried out based on fine-grained smart meter data from the Australian grid with separately recorded measurements of rooftop PV generation and loads. The results demonstrate the superior performance of the proposed scheme compared with a series of state-of-theart methods and indicate the importance and effectiveness of subprofile clustering and high PV visibility

Machine Learning based Wind Power Forecasting for Operational Decision Support

To utilize renewable energy efficiently to meet the needs of mankind's living demands becomes an extremely hot topic since global warming is the most serious global environmental problem that human beings are facing today. Burning of fossil fuels, such as coal and oil directly for generating electricity leads to environment pollution and exacerbates global warning. However, large-scale development of hydropower increases greenhouse gas emissions and greenhouse effects. This research is related to knowledge of wind power forecasting (WPF) and machine learning (ML). This research is built around one central research question: How to improve the accuracy of WPF by using AI methods? A pilot conceptual system combining meteorological information and operations management has been formulated. The main contribution is visualized in a proposed new framework, named Meteorological Information Service Decision Support System, consisting of a meteorological information module, wind power prediction module and operations management module. This conceptual framework has been verified by quantitative analysis in empirical cases. This system utilizes meteorological information for decision-making based on condition-based maintenance in operations and management for the purpose of optimizing energy management. It aims to analyze and predict the variation of wind power for the next day or the following week to develop scheduling planning services for WPEs based on predicting wind speed for every six hours, which is short-term wind speed prediction, through training, validating, and testing dataset. Accurate prediction of wind speed is crucial for weather forecasting service and WPF. This study presents a carefully designed wind speed prediction model which combines fully-connected neural network (FCNN), long short-term memory (LSTM) algorithm with eXtreme Gradient Boosting (XGBoost) technique, to predict wind speed. The performance of each model is tested by using reanalysis data from European Center for Medium-Range Weather Forecasts (ECMWF) for Meteorological observatory located in Vaasa in Finland. The results show that XGBoost algorithm has similar improved prediction performance as LSTM algorithm, in terms of RMSE, MAE and R2 compared to the commonly used traditional FCNN model. On the other hand, the XGBoost algorithm has a significant advantage on training time while comparing to the other algorithms in this case study. Additionally, this sensitivity analysis indicates great potential of the optimized deep learning (DL) method, which is a subset of machine learning (ML), in improving local weather forecast on the coding platform of Python. The results indicate that, by using Meteorological Information Service Decision Support System, it is possible to support effective decision-making and create timely actions within the WPEs. Findings from this research contribute to WPF in WPEs. The main contribution of this research is achieving decision optimization on a decision support system by using ML. It was concluded that the proposed system is very promising for potential applications in wind (power) energy management

Towards intelligent operation of future power system: bayesian deep learning based uncertainty modelling technique

The increasing penetration level of renewable energy resources (RES) in the power system brings fundamental changes of the system operating paradigms. In the future, the intermittent nature of RES and the corresponding smart grid technologies will lead to a much more volatile power system with higher level uncertainties. At the same time, as a result of the larger scale installation of advanced sensor devices in power system, power system engineers for the first time have the opportunity to gain insights from the influx of massive data sets in order to improve the system performance in various aspects. To this end, it is imperative to explore big data methodologies with the aim of exploring the uncertainty space within such complex data sets and thus supporting real-time decision-making in future power system. In this thesis, Bayesian Deep learning is investigated with the aim of exploring data-driven methodologies to deal with uncertainties which is in the following three aspects. (1) The first part of this thesis proposes a novel probabilistic day-ahead net load forecasting method to capture both epistemic uncertainty and aleatoric uncertainty using Bayesian deep long short-term memory network. The proposed methodological framework employs clustering in sub-profiles and considers residential rooftop PV outputs as input features to enhance the performance of aggregated net load forecasting. Numerical experiments have been carried out based on fine-grained smart meter data from the Australian grid with separately recorded measurements of rooftop PV generation and loads. The results demonstrate the superior performance of the proposed scheme compared with a series of state-of-the-art methods and indicate the importance and effectiveness of sub-profile clustering and high PV visibility. (2) The second part of this thesis studies a novel Conditional Bayesian Deep Auto-Encoder (CBDAC) based security assessment framework to compute a confidence metric of the prediction. This informs not only the operator to judge whether the prediction can be trusted, but it also allows for judging whether the model needs updating. A case study based on IEEE 68-bus system demonstrates that CBDAC outperforms the state-of-the-art machine learning-based DSA methods and the models that need updating under different topologies can be effectively identified. Furthermore, the case study verifies that effective updating of the models is possible even with very limited data. (3) The last part of this thesis proposes a novel Bayesian Deep Reinforcement Learning-based resilient control approach for multi-energy micro-grid. In particular, the proposed approach replaces deterministic network in traditional Reinforcement Learning with Bayesian probabilistic network in order to obtain an approximation of the value function distribution, which effectively solves Q-value overestimation issue. The proposed model is able to provide both energy management during normal operating conditions and resilient control during extreme events in a multi-energy micro-grid system. Comparing with naive DDPG method and optimisation method, the effectiveness and importance of employing Bayesian Reinforcement Learning approach is investigated and illustrated across different operating scenarios. Case studies have shown that by using the Monte Carlo posterior mean of the Bayesian value function distribution instead of a deterministic estimation, the proposed BDDPG method achieves a near-optimum policy in a more stable process, which verifies the robustness and the practicability of the proposed approach.Open Acces

A review of probabilistic forecasting and prediction with machine learning

Predictions and forecasts of machine learning models should take the form of probability distributions, aiming to increase the quantity of information communicated to end users. Although applications of probabilistic prediction and forecasting with machine learning models in academia and industry are becoming more frequent, related concepts and methods have not been formalized and structured under a holistic view of the entire field. Here, we review the topic of predictive uncertainty estimation with machine learning algorithms, as well as the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions. The review covers a time period spanning from the introduction of early statistical (linear regression and time series models, based on Bayesian statistics or quantile regression) to recent machine learning algorithms (including generalized additive models for location, scale and shape, random forests, boosting and deep learning algorithms) that are more flexible by nature. The review of the progress in the field, expedites our understanding on how to develop new algorithms tailored to users' needs, since the latest advancements are based on some fundamental concepts applied to more complex algorithms. We conclude by classifying the material and discussing challenges that are becoming a hot topic of research.Comment: 83 pages, 5 figure

Deep Attentive Time Series Modelling for Quantitative Finance

Mención Internacional en el título de doctorTime series modelling and forecasting is a persistent problem with extensive implications in scientific, business, industrial, and economic areas. This thesis’s contribution is twofold. Firstly, we propose a novel probabilistic time series forecasting methodology that introduces the use of Fourier domain-based attention models, merging classic signal processing spectral filtering techniques with machine learning architectures. Secondly, we take advantage of the abundance of financial intraday high-frequency data to develop deep learning-based solutions for modelling financial time series. Machine learning methods can potentially enhance the performance of traditional methodologies used by practitioners. Deep neural networks’ feature extraction capabilities, which can benefit from the rising accessibility of highfrequency data, and attention mechanisms, which help to model temporal patterns, are mostly to blame for this. Concerning our first major contribution, this thesis empirically demonstrates that spectral domain-based machine learning models can learn the properties of time series datasets and integrate this information to improve the forecasting accuracy. Simultaneously, Fourier domain-based models alleviate some of the inconveniences commonly associated with deep autoregressive models. These architectures, prone to prioritising recent past data, often ignore critical global information not contained in previous time steps. Additionally, they are susceptible to error accumulation and propagation and may not yield illustrative results. The proposed model, the Spectral Attention Autoregressive Model (SAAM), mitigates these problems by combining deep autoregressive models with a Spectral Attention (SA) module. This module uses two attention models operating over the Fourier domain representation of the time series’ embedding. Through spectral filtering, SAAM differentiates between the components of the frequency domain that should be considered noise and subsequently filtered out, and the global patterns that are relevant and should be incorporated into the predictions. Empirical evaluation proves how the proposed Spectral Attention module can be integrated into various deep autoregressive models, consistently improving the results of these base architectures and achieving state-of-the-art performance. Afterwards, this thesis shifts toward showcasing the benefits of machine learning solutions in two different quantitative finance scenarios, proving how attention-based deep learning approaches compare favourably to classic parametric-based models and providing solutions for various algorithmic and high-frequency trading problems. In the context of volatility forecasting, which plays a central role among equity risk measures, we show that Dilated Causal Convolutional-based neural networks offer significant performance gains compared to well-established volatility-oriented parametric models. The proposed model, called DeepVol, showcases how data- driven models can avoid the limitations of classical methods by taking advantage of the abundance of high-frequency data. DeepVol outperforms baseline methods while exhibiting robustness in the presence of volatility shocks, showing its ability to extract universal features and transfer learning to out-of-distribution data. Consequently, data-driven approaches should be carefully considered in the context of volatility forecasting, as they can be instrumental in the valuation of financial derivatives, risk management, and the formation of investment portfolios. Finally, this thesis presents a survival analysis model for estimating the distri- bution of fill times for limit orders posted in the Limit Order Book (LOB). The proposed model, which does not make assumptions about the underlying stochastic processes, employs a convolutional-Transformer encoder and a monotonic neural network decoder to relate the time-varying features of the LOB to the distribution of fill times. It grants practitioners the capability of making informed decisions between market orders and limit orders, which in practice entails a trade-off between immediate execution and price premium. We offer an exhaustive comparison of the survival functions resulting from different order placement strategies, offering insight into the fill probability of orders placed within the spread. Empirical evaluation reveals the superior performance of the monotonic encoder-decoder convolutional- Transformer compared to state-of-the-art benchmarks, leading to more accurate predictions and improved economic value.El modelado y predicción de series temporales es un problema persistente con amplias implicaciones en áreas científicas, comerciales, industriales y económicas. Esta tesis propone una doble contribución en este ámbito. En primer lugar, formulamos una novedosa metodología para la predicción probabilística de series temporales que introduce el uso de modelos de atención basados en el dominio de la frecuencia, con la transformada de Fourier desempeñando un papel fundamental. El modelo propuesto fusiona técnicas clásicas de filtrado espectral, pertenecientes al campo del procesado de señal, con modelos de aprendizaje automático. En segundo lugar, desarrollamos varias soluciones basadas en aprendizaje profundo para el modelado de datos financieros intradía, aprovechando la cada vez mayor disponibilidad de los mismos. Los métodos de aprendizaje automático poseen el potencial para mejorar los resultados obtenidos por las metodologías clásicas que los profesionales del ámbito de las finanzas cuantitativas acostumbran a utilizar. La capacidad de extracción de características de las redes neuronales, que pueden aprovechar la creciente accesibilidad a los datos financieros de alta frecuencia, y el uso de los mecanismos de atención para el modelado temporal, son los principales responsables de ésto. En lo relativo a la primera de las contribuciones mencionadas anteriormente, es decir, el uso de modelos de aprendizaje automático que operan sobre el dominio de la frecuencia, esta tesis demuestra de manera empírica que los modelos de aprendizaje profundo basados en el dominio espectral pueden aprender de forma más eficiente las propiedades de las series temporales a predecir. De esta manera, logran mejorar la precisión de las predicciones a la vez que solventan varios de los problemas que lastran el rendimiento de los modelos autoregresivos. Estas arquitecturas son propensas a sobreponderar los datos del pasado inmediato, ignorando a menudo valiosa información global que no está contenida en estas observaciones recientes. Además, son susceptibles a la acumulación y propagación de errores. Finalmente, los resultados que producen son difícilmente interpretables. Proponemos un nuevo modelo, llamado “Spectral Attention Autoregressive Model”(SAAM) (Modelo Autorregresivo con Atención Espectral), que mitiga estos problemas combinando modelos autorregresivos basados en aprendizaje profundo con un módulo de Atención Espectral. Dicho módulo contiene dos modelos de atención que operan sobre la representación en el dominio de Fourier del “embedding” obtenido a partir de la serie temporal a predecir. Usando técnicas de filtrado espectral, SAAM diferencia entre los componentes del espectro que deben ser considerados ruido, y por consiguiente deben ser filtrados, y aquellos patrones globales que son relevantes y deben ser incorporados en las predicciones. Mediante una exhaustiva evaluación empírica, demostramos que nuestro modelo de Atención Espectral puede ser integrado en diversos modelos autorregresivos que forman parte del estado del arte actual, mejorando de forma consistente los resultados obtenidos. En lo relativo a la segunda contribución principal de esta tesis doctoral, demostramos los beneficios que las metodologías de aprendizaje automático basadas en modelos de atención pueden aportar en dos problemas propios de las finanzas cuantitativas. Diversos experimentos demuestran cómo este tipo de modelos pueden mejorar los resultados obtenidos por los modelos clásicos empleados en este campo, proporcionando soluciones innovadoras para diversos problemas recurrentes dentro del trading algorítmico de alta frecuencia. La predicción de volatilidad en mercados financieros es el primero de estos problemas en ser abordado en la presente tesis. La estimación de volatilidad desempeña un papel central entre las medidas de riesgo utilizadas en los mercados de renta variable. En esta tesis demostramos que las redes neuronales basadas en “Dilated Causal Convolutions” (Convolucionales Causales Dilatadas) ofrecen ganancias significativas en comparación con los modelos paramétricos clásicos desarrollados única y exclusivamente para predicción de volatilidad. El modelo propuesto, llamado DeepVol, evidencia que el uso de modelos de aprendizaje profundo puede evitar las numerosas limitaciones propias de los métodos clásicos, logrando aprovechar la abundancia de datos de alta frecuencia para aprender las funciones deseadas. DeepVol supera a todos los modelos de referencia usados como comparativa, a la vez que exhibe robustez en períodos que contienen shocks de volatilidad, demostrando su capacidad para extraer características universales comunes a diferentes instrumentos financieros. Los resultados obtenidos en esta parte de la tesis nos llevan a concluir que los modelos de aprendizaje automático deben considerarse cuidadosamente en el contexto de predicción de volatilidad, pudiendo ser especialmente relevantes en la valoración de derivados financieros, gestión del riesgo, y creación de carteras de inversión. Para terminar, esta tesis presenta un modelo de análisis de supervivencia para estimar la distribución de probabilidad de ejecución subyacente a órdenes limitadas publicadas en el conocido como “Limit Order Book” (Libro de Órdenes Limitadas). El modelo propuesto, que no necesita partir de suposiciones sobre los procesos estocásticos subyacentes, emplea una arquitectura codificador/decodificador que utiliza un “Transformer” convolutional para codificar la información del libro de órdenes y una red monotónica que decodifica la función de supervivencia a estimar.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidente: Juan José Murillo Fuentes.- Secretario: Emilio Parrado Hernández.- Vocal: Manuel Gómez Rodrígue

Predicting the Future

Due to the increased capabilities of microprocessors and the advent of graphics processing units (GPUs) in recent decades, the use of machine learning methodologies has become popular in many fields of science and technology. This fact, together with the availability of large amounts of information, has meant that machine learning and Big Data have an important presence in the field of Energy. This Special Issue entitled “Predicting the Future—Big Data and Machine Learning” is focused on applications of machine learning methodologies in the field of energy. Topics include but are not limited to the following: big data architectures of power supply systems, energy-saving and efficiency models, environmental effects of energy consumption, prediction of occupational health and safety outcomes in the energy industry, price forecast prediction of raw materials, and energy management of smart buildings

Alternative Sources of Energy Modeling, Automation, Optimal Planning and Operation

An economic development model analyzes the adoption of alternative strategy capable of leveraging the economy, based essentially on RES. The combination of wind turbine, PV installation with new technology battery energy storage, DSM network and RES forecasting algorithms maximizes RES integration in isolated islands. An innovative model of power system (PS) imbalances is presented, which aims to capture various features of the stochastic behavior of imbalances and to reduce in average reserve requirements and PS risk. Deep learning techniques for medium-term wind speed and solar irradiance forecasting are presented, using for first time a specific cloud index. Scalability-replicability of the FLEXITRANSTORE technology innovations integrates hardware-software solutions in all areas of the transmission system and the wholesale markets, promoting increased RES. A deep learning and GIS approach are combined for the optimal positioning of wave energy converters. An innovative methodology to hybridize battery-based energy storage using supercapacitors for smoother power profile, a new control scheme and battery degradation mechanism and their economic viability are presented. An innovative module-level photovoltaic (PV) architecture in parallel configuration is introduced maximizing power extraction under partial shading. A new method for detecting demagnetization faults in axial flux permanent magnet synchronous wind generators is presented. The stochastic operating temperature (OT) optimization integrated with Markov Chain simulation ascertains a more accurate OT for guiding the coal gasification practice

Machine Learning Approaches for Traffic Flow Forecasting

Intelligent Transport Systems (ITS) as a field has emerged quite rapidly in the recent years. A competitive solution coupled with big data gathered for ITS applications needs the latest AI to drive the ITS for the smart and effective public transport planning and management. Although there is a strong need for ITS applications like Advanced Route Planning (ARP) and Traffic Control Systems (TCS) to take the charge and require the minimum of possible human interventions. This thesis develops the models that can predict the traffic link flows on a junction level such as road traffic flows for a freeway or highway road for all traffic conditions. The research first reviews the state-of-the-art time series data prediction techniques with a deep focus in the field of transport Engineering along with the existing statistical and machine leaning methods and their applications for the freeway traffic flow prediction. This review setup a firm work focussed on the view point to look for the superiority in term of prediction performance of individual statistical or machine learning models over another. A detailed theoretical attention has been given, to learn the structure and working of individual chosen prediction models, in relation to the traffic flow data. In modelling the traffic flows from the real-world Highway England (HE) gathered dataset, a traffic flow objective function for highway road prediction models is proposed in a 3-stage framework including the topological breakdown of traffic network into virtual patches, further into nodes and to the basic links flow profiles behaviour estimations. The proposed objective function is tested with ten different prediction models including the statistical, shallow and deep learning constructed hybrid models for bi-directional links flow prediction methods. The effectiveness of the proposed objective function greatly enhances the accuracy of traffic flow prediction, regardless of the machine learning model used. The proposed prediction objective function base framework gives a new approach to model the traffic network to better understand the unknown traffic flow waves and the resulting congestions caused on a junction level. In addition, the results of applied Machine Learning models indicate that RNN variant LSTMs based models in conjunction with neural networks and Deep CNNs, when applied through the proposed objective function, outperforms other chosen machine learning methods for link flow predictions. The experimentation based practical findings reveal that to arrive at an efficient, robust, offline and accurate prediction model apart from feeding the ML mode with the correct representation of the network data, attention should be paid to the deep learning model structure, data pre-processing (i.e. normalisation) and the error matrices used for data behavioural learning. The proposed framework, in future can be utilised to address one of the main aims of the smart transport systems i.e. to reduce the error rates in network wide congestion predictions and the inflicted general traffic travel time delays in real-time