The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie

Probabilistic short-term wind power forecasting based on kernel density estimators

International audienceShort-term wind power forecasting tools have been developed for some time. The majority of such tools usually provide single-valued (spot) predictions. Such predictions are however often not adequate when the aim is decision-making under uncertainty. In that case there is a clear requirement by end-users to have additional information on the uncertainty of the predictions for performing efficiently functions such as reserves estimation, unit commitment, trading in electricity markets, a.o. In this paper, we propose a method for producing the complete predictive probability density function (PDF) for each time step of the prediction horizon based on the kernel density estimation technique. The performance of the proposed approach is demonstrated using real data from several wind farms. Comparisons to state-of-the-art methods from both outside and inside the wind power forecasting community are presented illustrating the performances of the proposed method

Towards the Efficient Probabilistic Characterization of Tropical Cyclone-Generated Storm Surge Hazards Under Stationary and Nonstationary Conditions

The scarcity of observations at any single location confounds the probabilistic characterization of tropical cyclone-generated storm surge hazards using annual maxima and peaks-over-threshold methods. The EST and the JPM are indirect approaches aimed at estimating the probability distribution of the response variable of interest (i.e. storm surge) using the probability distributions of predictor variables (e.g. storm size, storm intensity etc.). In the first part of this work, the relative performance of the empirical simulation technique (EST; Borgman et al., 1992) and the joint probability method (JPM; Myers, 1970) is evaluated via stochastic simulation methods. It is shown that the JPM has greater predictive capability for the estimation of the frequency of tropical cyclone winds, an efficient proxy for storm surge. The traditional attractions of the EST have been its economy and ease of implementation; more efficient numerical approximation schemes such as Bayesian quadrature now exist, which allows for more cost effective implementation of the JPM. In addition, typical enhancements of the original EST approach, such as the introduction of synthetic storms to complement the historical sample, are largely ineffective. These observations indicate that the EST should no longer be considered a practical approach for the robust and reliable estimation of the exceedance probabilities of storm surge levels, as required for actuarial purposes, engineering design and flood risk management in tropical cyclone-prone regions. The JPM is, however, not applicable to extratropical storm-prone regions and nonstationary phenomena. Additionally, the JPM requires the evaluation of a multidimensional integral composed of the product of marginal and conditional probability distributions of storm descriptors. This integral is typically approximated as a weighted summation of discrete function evaluations in each dimension and extended to D-dimensions by tensor product rules. To adequately capture the dynamics of the underlying physical process—storm surge driven by tropical cyclone wind fields—one must maintain a large number of explanatory variables in the integral. The complexity and cost of the joint probability problem, however, increases exponentially with dimension, precluding the inclusion of more than a few (≤4) stochastic variables. In the second part of the work, we extend stochastic simulation approaches to the classical joint probability problem. The successful implementation of stochastic simulation to the storm surge frequency problem requires the introduction of a new paradigm: the use of a regression function constructed by the careful selection of an optimal training set from the storm sample space such that the growth of support nodes required for efficient interpolation remains nonexponential while preserving the performance of a product grid equivalent. Apart from retaining the predictive capability of the JPM, the stochastic simulation approach also allows for nonstationary phenomena such as the effects of climate change on tropical cyclone activity to be efficiently modeled. A great utility of the stochastic approach is that the random sampling scheme is readily modified so that it conducts empirical simulation if required in place of parametric simulation. The enhanced empirical simulation technique attains predictive capabilities that are comparable with the JPM and the parametric simulation approach, while also retaining the suitability of empirical methods for application to situations that confound parametric methods, such as, application to extratropical cyclones and complexly distributed data. The parametric and empirical simulation techniques, together, will enable seamless flood hazard estimation for the entire coastline of the United States, with simple elaborations where needed to allow for the joint occurrence of both tropical and extratropical storms as compound stochastic processes. The stochastic approaches proposed hold great promise for the efficient probabilistic modeling of other multi-parameter systems such as earthquakes and riverine floods

Aprendizaje automático aplicado al modelado de viento y olas

Trabajo de Fin de Grado en Ingeniería del Software, Facultad de Informática UCM, Departamento de Arquitectura de Computadores y Automática, Curso 2019/2020In the fight against climate change, Offshore wind energy is at the forefront, in the development phase. The problem with turbines anchored to the seabed lies in the enormous cost of installation and maintenance, leading to the theoretical approach of floating offshore wind turbines. However, floating turbines are exposed to new wave loads and stronger wind loads. To enable their implementation while maximizing the electricity production and ensuring the protection of the structure, more accurate predictive models than the physical and statistical ones found in the literature are needed for the metocean (meteorological and oceanographic) variables involved. This project aims to model the wind speed in the time domain, the significant waves height in the frequency domain and the misalignment between wind and waves direction in the time domain, applying Machine Learning techniques. Offshore data collection as well as an exploratory data analysis and data cleaning phases have been carried out. Subsequently, the following algorithms were applied to train the models: Linear Regression, Support Vector Machines for Regression, Gaussian Process Regression and Neural Networks. Nonlinear Autoregressive with exogenous input neural networks (NARX) have proved to be the best algorithm both for wind speed and misalignment forecasting and the most accurate predictive model for significant waves height prediction has been the Gaussian Process Regression (GPR). In this project we demonstrated the ability of Machine Learning algorithms to model wind variables of a stochastic nature and waves. We emphasize the importance of evaluating the models through techniques such as Learning Curves to make better decisions to optimize them. This work not only makes predictive models available for later use, but it is also a pioneer in misalignment modelling, leaving a door open for future research.En la lucha contra el cambio climático, la energía eólica marina se sitúa en cabeza encontrándose en fase de desarrollo. El problema de las turbinas ancladas al lecho marino reside en el enorme coste de instalación y mantenimiento, llevando al planteamiento teórico de turbinas eólicas marinas flotantes. Estas, sin embargo, están expuestas a nuevas cargas de olas y cargas de viento más fuertes. Para hacer posible su implantación maximizando la producción eléctrica a la vez que asegurando la protección de la estructura, se necesita disponer de modelos predictivos más precisos que los físicos y estadísticos de la literatura para las variables metoceánicas (meteorológicas y oceánicas) implicadas. El objetivo de este proyecto es modelar la velocidad del viento en el dominio del tiempo, la altura significativa de la ola en el dominio de la frecuencia y la desalineación entre la dirección del viento y de las olas en el dominio temporal, aplicando técnicas de Aprendizaje Automático. Se ha llevado a cabo una fase de recopilación de datos medidos en alta mar, así como el análisis exploratorio y limpieza de los mismos. Posteriormente, para el entrenamiento de los modelos se aplicaron los algoritmos: Regresión Lineal, Máquinas de Vectores Soporte para Regresión, Proceso de Regresión Gausiano y Redes Neuronales. Las redes neuronales autorregresivas no lineales con entrada externa (NARX) han resultado ser el mejor algoritmo tanto para la predicción de la velocidad del viento como para la desalineación y para la altura significativa de la ola el modelo predictivo más preciso ha sido el proceso regresivo gausiano (GPR). En este proyecto demostramos la capacidad de los algoritmos de Aprendizaje Automático para modelar las variables del viento de naturaleza estocástica y del oleaje. Destacamos la importancia de la evaluación de los modelos mediante técnicas como las Curvas de Aprendizaje para tomar mejores decisiones en la optimización de los mismos. Este trabajo no pone solo a disposición modelos predictivos para su posterior uso, además es pionero en el modelado de la desalineación dejando una puerta abierta a futuras investigaciones.Depto. de Arquitectura de Computadores y AutomáticaFac. de InformáticaTRUEunpu

Forecasting Models for Integration of Large-Scale Renewable Energy Generation to Electric Power Systems

Amid growing concerns about climate change and non-renewable energy sources deple¬tion, vari¬able renewable energy sources (VRESs) are considered as a feasible substitute for conventional environment-polluting fossil fuel-based power plants. Furthermore, the transition towards clean power systems requires additional transmission capacity. Dynamic thermal line rating (DTLR) is being considered as a potential solution to enhance the current transmission line capacity and omit/postpone transmission system expansion planning, while DTLR is highly dependent on weather variations. With increasing the accommodation of VRESs and application of DTLR, fluctuations and variations thereof impose severe and unprecedented challenges on power systems operation. Therefore, short-term forecasting of large-scale VERSs and DTLR play a crucial role in the electric power system op¬eration problems. To this end, this thesis devotes on developing forecasting models for two large-scale VRESs types (i.e., wind and tidal) and DTLR. Deterministic prediction can be employed for a variety of power system operation problems solved by deterministic optimization. Also, the outcomes of deterministic prediction can be employed for conditional probabilistic prediction, which can be used for modeling uncertainty, used in power system operation problems with robust optimization, chance-constrained optimization, etc. By virtue of the importance of deterministic prediction, deterministic prediction models are developed. Prevalently, time-frequency decomposition approaches are adapted to decompose the wind power time series (TS) into several less non-stationary and non-linear components, which can be predicted more precisely. However, in addition to non-stationarity and nonlinearity, wind power TS demonstrates chaotic characteristics, which reduces the predictability of the wind power TS. In this regard, a wind power generation prediction model based on considering the chaosity of the wind power generation TS is addressed. The model consists of a novel TS decomposition approach, named multi-scale singular spectrum analysis (MSSSA), and least squares support vector machines (LSSVMs). Furthermore, deterministic tidal TS prediction model is developed. In the proposed prediction model, a variant of empirical mode decomposition (EMD), which alleviates the issues associated with EMD. To further improve the prediction accuracy, the impact of different components of wind power TS with different frequencies (scales) in the spatiotemporal modeling of the wind farm is assessed. Consequently, a multiscale spatiotemporal wind power prediction is developed, using information theory-based feature selection, wavelet decomposition, and LSSVM. Power system operation problems with robust optimization and interval optimization require prediction intervals (PIs) to model the uncertainty of renewables. The advanced PI models are mainly based on non-differentiable and non-convex cost functions, which make the use of heuristic optimization for tuning a large number of unknown parameters of the prediction models inevitable. However, heuristic optimization suffers from several issues (e.g., being trapped in local optima, irreproducibility, etc.). To this end, a new wind power PI (WPPI) model, based on a bi-level optimization structure, is put forward. In the proposed WPPI, the main unknown parameters of the prediction model are globally tuned based on optimizing a convex and differentiable cost function. In line with solving the non-differentiability and non-convexity of PI formulation, an asymmetrically adaptive quantile regression (AAQR) which benefits from a linear formulation is proposed for tidal uncertainty modeling. In the prevalent QR-based PI models, for a specified reliability level, the probabilities of the quantiles are selected symmetrically with respect the median probability. However, it is found that asymmetrical and adaptive selection of quantiles with respect to median can provide more efficient PIs. To make the formulation of AAQR linear, extreme learning machine (ELM) is adapted as the prediction engine. Prevalently, the parameters of activation functions in ELM are selected randomly; while different sets of random values might result in dissimilar prediction accuracy. To this end, a heuristic optimization is devised to tune the parameters of the activation functions. Also, to enhance the accuracy of probabilistic DTLR, consideration of latent variables in DTLR prediction is assessed. It is observed that convective cooling rate can provide informative features for DTLR prediction. Also, to address the high dimensional feature space in DTLR, a DTR prediction based on deep learning and consideration of latent variables is put forward. Numerical results of this thesis are provided based on realistic data. The simulations confirm the superiority of the proposed models in comparison to traditional benchmark models, as well as the state-of-the-art models

Large Scale Kernel Methods for Fun and Profit

Kernel methods are among the most flexible classes of machine learning models with strong theoretical guarantees. Wide classes of functions can be approximated arbitrarily well with kernels, while fast convergence and learning rates have been formally shown to hold. Exact kernel methods are known to scale poorly with increasing dataset size, and we believe that one of the factors limiting their usage in modern machine learning is the lack of scalable and easy to use algorithms and software. The main goal of this thesis is to study kernel methods from the point of view of efficient learning, with particular emphasis on large-scale data, but also on low-latency training, and user efficiency. We improve the state-of-the-art for scaling kernel solvers to datasets with billions of points using the Falkon algorithm, which combines random projections with fast optimization. Running it on GPUs, we show how to fully utilize available computing power for training kernel machines. To boost the ease-of-use of approximate kernel solvers, we propose an algorithm for automated hyperparameter tuning. By minimizing a penalized loss function, a model can be learned together with its hyperparameters, reducing the time needed for user-driven experimentation. In the setting of multi-class learning, we show that – under stringent but realistic assumptions on the separation between classes – a wide set of algorithms needs much fewer data points than in the more general setting (without assumptions on class separation) to reach the same accuracy. The first part of the thesis develops a framework for efficient and scalable kernel machines. This raises the question of whether our approaches can be used successfully in real-world applications, especially compared to alternatives based on deep learning which are often deemed hard to beat. The second part aims to investigate this question on two main applications, chosen because of the paramount importance of having an efficient algorithm. First, we consider the problem of instance segmentation of images taken from the iCub robot. Here Falkon is used as part of a larger pipeline, but the efficiency afforded by our solver is essential to ensure smooth human-robot interactions. In the second instance, we consider time-series forecasting of wind speed, analysing the relevance of different physical variables on the predictions themselves. We investigate different schemes to adapt i.i.d. learning to the time-series setting. Overall, this work aims to demonstrate, through novel algorithms and examples, that kernel methods are up to computationally demanding tasks, and that there are concrete applications in which their use is warranted and more efficient than that of other, more complex, and less theoretically grounded models

Machine Learning Tool for Transmission Capacity Forecasting of Overhead Lines based on Distributed Weather Data

Die Erhöhung des Anteils intermittierender erneuerbarer Energiequellen im elektrischen Energiesystem ist eine Herausforderung für die Netzbetreiber. Ein Beispiel ist die Zunahme der Nord-Süd Übertragung von Windenergie in Deutschland, die zu einer Erhöhung der Engpässe in den Freileitungen führt und sich direkt in den Stromkosten der Endverbraucher niederschlägt. Neben dem Ausbau neuer Freileitungen ist ein witterungsabhängiger Freileitungsbetrieb eine Lösung, um die aktuelle Auslastung des Systems zu verbessern. Aus der Analyse in einer Probeleitung in Deutschland wurde gezeigt, dass einen Zuwachs von ca. 28% der Stromtragfähigkeit eine Reduzierung der Kosten für Engpassmaßnahmen um ca. 55% bedeuten kann. Dieser Vorteil kann nur vom Netzbetreiber wahrgenommen werden, wenn eine Belastbarkeitsprognose für die Stromerzeugunsgplanung der konventionellen Kraftwerke zur Verfügung steht. Das in dieser Dissertation vorgestellte System prognostiziert die Belastbarkeit von Freileitungen für 48 Stunden, mit einer Verbesserung der Prognosegenauigkeit im Vergleich zum Stand-der-Technik von 6,13% in Durchschnitt. Der Ansatz passt die meteorologischen Vorhersagen an die lokale Wettersituation entlang der Leitung an. Diese Anpassungen sind aufgrund von Veränderungen der Topographie entlang der Leitungstrasse und Windschatten der umliegenden Bäume notwendig, da durch die meteorologischen Modelle diese nicht beschrieben werden können. Außerdem ist das in dieser Dissertation entwickelte Modell in der Lage die Genauigkeitsabweichungen der Wettervorhersage zwischen Tag und Nacht abzugleichen, was vorteilhaft für die Strombelastbarkeitsprognose ist. Die Zuverlässigkeit und deswegen auch die Effizienz des Stromerzeugungsplans für den nächsten 48 Stunden wurde um 10% gegenüber dem Stand der Technik erhöht. Außerdem wurde in Rahmen dieser Arbeit ein Verfahren für die Positionierung der Wetterstationen entwickelt, um die wichtigsten Stellen entlang der Leitung abzudecken und gleichzeitig die Anzahl der Wetterstationen zu minimieren. Wird ein verteiltes Sensornetzwerk in ganz Deutschland umgesetzt, wird die Einsparung von Redispatchingkosten eine Kapitalrendite von ungefähr drei Jahren bedeuten. Die Durchführung einer transienten Analyse ist im entwickelten System ebenfalls möglich, um Engpassfälle für einige Minuten zu lösen, ohne die maximale Leitertemperatur zu erreichen. Dieses Dokument versucht, die Vorteile der Freileitungsmonitoringssysteme zu verdeutlichen und stellt eine Lösung zur Unterstützung eines flexiblen elektrischen Netzes vor, die für eine erfolgreiche Energiewende erforderlich ist