Random Forest Ensemble of Support Vector Regression Models for Solar Power Forecasting

To mitigate the uncertainty of variable renewable resources, two off-the-shelf machine learning tools are deployed to forecast the solar power output of a solar photovoltaic system. The support vector machines generate the forecasts and the random forest acts as an ensemble learning method to combine the forecasts. The common ensemble technique in wind and solar power forecasting is the blending of meteorological data from several sources. In this study though, the present and the past solar power forecasts from several models, as well as the associated meteorological data, are incorporated into the random forest to combine and improve the accuracy of the day-ahead solar power forecasts. The performance of the combined model is evaluated over the entire year and compared with other combining techniques.Comment: This is a preprint of the full paper that published in Innovative Smart Grid Technologies, North America Conference, 201

Predicting Global Irradiance Combining Forecasting Models Through Machine Learning

This paper has been presented at : 13th International Conference on Hybrid Artificial Intelligence Systems (HAIS 2018)Predicting solar irradiance is an active research problem, with many physical models having being designed to accurately predict Global Horizontal Irradiance. However, some of the models are better at short time horizons, while others are more accurate for medium and long horizons. The aim of this research is to automatically combine the predictions of four different models (Smart Persistence, Satellite, Cloud Index Advection and Diffusion, and Solar Weather Research and Forecasting) by means of a state-of-the-art machine learning method (Extreme Gradient Boosting). With this purpose, the four models are used as inputs to the machine learning model, so that the output is an improved Global Irradiance forecast. A 2-year dataset of predictions and measures at one radiometric station in Seville has been gathered to validate the method proposed. Three approaches are studied: a general model, a model for each horizon, and models for groups of horizons. Experimental results show that the machine learning combination of predictors is, on average, more accurate than the predictors themselves.The authors are supported by the Spanish Ministry of Economy and Competitiveness, projects ENE2014-56126-C2-1-R and ENE2014-56126-C2-2-R and FEDER funds. Some of the authors are also funded by the Junta de Andalucía (research group TEP-220)

Evolutionary-based prediction interval estimation by blending solar radiation forecasting models using meteorological weather types

Recent research has shown that the integration or blending of different forecasting models is able to improve the predictions of solar radiation. However, most works perform model blending to improve point forecasts, but the integration of forecasting models to improve probabilistic forecasting has not received much attention. In this work the estimation of prediction intervals for the integration of four Global Horizontal Irradiance (GHI) forecasting models (Smart Persistence, WRF-solar, CIADcast, and Satellite) is addressed. Several short-term forecasting horizons, up to one hour ahead, have been analyzed. Within this context, one of the aims of the article is to study whether knowledge about the synoptic weather conditions, which are related to the stability of weather, might help to reduce the uncertainty represented by prediction intervals. In order to deal with this issue, information about which weather type is present at the time of prediction, has been used by the blending model. Four weather types have been considered. A multi-objective variant of the Lower Upper Bound Estimation approach has been used in this work for prediction interval estimation and compared with two baseline methods: Quantile Regression (QR) and Gradient Boosting (GBR). An exhaustive experimental validation has been carried out, using data registered at Seville in the Southern Iberian Peninsula. Results show that, in general, using weather type information reduces uncertainty of prediction intervals, according to all performance metrics used. More specifically, and with respect to one of the metrics (the ratio between interval coverage and width), for high-coverage (0.90, 0.95) prediction intervals, using weather type enhances the ratio of the multi-objective approach by 2%¿. Also, comparing the multi-objective approach versus the two baselines for high-coverage intervals, the improvement is 11%¿% over QR and 10%¿% over GBR. Improvements for low-coverage intervals (0.85) are smaller.The authors are supported by projects funded by Agencia Estatal de Investigación, Spain (PID2019-107455RB-C21 and PID2019-107455RB-C22/AEI/10.13039/501100011033). Also supported by Spanish Ministry of Economy and Competitiveness, project ENE2014-56126-C2-1-R and ENE2014-56126-C2-2-R (http://prosol.uc3m.es). The University of Jaén team is also supported by FEDER, Spain funds and by the Junta de Andalucía, Spain (Research group TEP-220

Operational solar forecasting for the real-time market

Despite the significant progress made in solar forecasting over the last decade, most of the proposed models cannot be readily used by independent system operators (ISOs). This article proposes an operational solar forecasting algorithm that is closely aligned with the real-time market (RTM) forecasting requirements of the California ISO (CAISO). The algorithm first uses the North American Mesoscale (NAM) forecast system to generate hourly forecasts for a 5-h period that are issued 12 h before the actual operating hour, satisfying the lead-time requirement. Subsequently, the world's fastest similarity search algorithm is adopted to downscale the hourly forecasts generated by NAM to a 15-min resolution, satisfying the forecast-resolution requirement. The 5-h-ahead forecasts are repeated every hour, following the actual rolling update rate of CAISO. Both deterministic and probabilistic forecasts generated using the proposed algorithm are empirically evaluated over a period of 2 years at 7 locations in 5 climate zones

AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been increasing interest from technology companies, government institutions, and meteorological agencies in building digital twins of the Earth. Recent approaches using transformers, physics-informed machine learning, and graph neural networks have demonstrated state-of-the-art performance on relatively narrow spatiotemporal scales and specific tasks. With the recent success of generative artificial intelligence (AI) using pre-trained transformers for language modeling and vision with prompt engineering and fine-tuning, we are now moving towards generalizable AI. In particular, we are witnessing the rise of AI foundation models that can perform competitively on multiple domain-specific downstream tasks. Despite this progress, we are still in the nascent stages of a generalizable AI model for global Earth system models, regional climate models, and mesoscale weather models. Here, we review current state-of-the-art AI approaches, primarily from transformer and operator learning literature in the context of meteorology. We provide our perspective on criteria for success towards a family of foundation models for nowcasting and forecasting weather and climate predictions. We also discuss how such models can perform competitively on downstream tasks such as downscaling (super-resolution), identifying conditions conducive to the occurrence of wildfires, and predicting consequential meteorological phenomena across various spatiotemporal scales such as hurricanes and atmospheric rivers. In particular, we examine current AI methodologies and contend they have matured enough to design and implement a weather foundation model.Comment: 44 pages, 1 figure, updated Fig.

Weather & Climate Services for the Energy Industry

Climate Change; Climate Forecasting; Energy Industry; Climate Risk Management; Meteorolog

Deep learning-based hybrid short-term solar forecast using sky images and meteorological data

The global growth of solar power generation is rapid, yet the complex nature of cloud movement introduces significant uncertainty to short-term solar irradiance, posing challenges for intelligent power systems. Accurate short-term solar irradiance and photovoltaic power generation predictions under cloudy skies are critical for sub-hourly electricity markets. Ground-based image (GSI) analysis using convolutional neural network (CNN) algorithms has emerged as a promising method due to advancements in machine vision models based on deep learning networks. In this work, a novel deep network, ”ViT-E,” based on an attention mechanism Transformer architecture for short-term solar irradiance forecasting has been proposed. This innovative model enables cross-modality data parsing by establishing mapping relationships within GSI and between GSI, meteorological data, historical irradiation, clear sky irradiation, and solar angles. The feasibility of the ViT-E network was assessed the Folsom dataset from California, USA. Quantitative analysis showed that the ViT-E network achieved RMSE values of 81.45 W/m2 , 98.68 W/m2 , and 104.91 W/m2 for 2, 6, and 10-minute forecasts, respectively, outperforming the persistence model by 4.87%, 16.06%, and 19.09% and displaying performance comparable to CNN-based models. Qualitative analysis revealed that the ViT-E network successfully predicted 20.21%, 33.26%, and 36.87% of solar slope events at 2, 6, and 10 minutes in advance, respectively, significantly surpassing the persistence model and currently prevalent CNN-based model by 9.43%, 3.91%, and -0.55% for 2, 6, and 10-minute forecasts, respectively. Transfer learning experiments were conducted to test the ViT-E model’s generalisation under different climatic conditions and its performance on smaller datasets. We discovered that the weights learned from the three-year Folsom dataset in the United States could be transferred to a half-year local dataset in Nottingham, UK. Training with a dataset one-fifth the size of the original dataset achieved baseline accuracy standards and reduced training time by 80.2%. Additionally, using a dataset equivalent to only 4.5% of the original size yielded a model with less than 2% accuracy below the baseline. These findings validated the generalisation and robustness of the model’s trained weights. Finally, the ViT-E model architecture and hyperparameters were optimised and searched. Our investigation revealed that directly applying migrated deep vision models leads to redundancy in solar forecasting. We identified the best hyperparameters for ViT-E through manual hyperparameter space exploration. As a result, the model’s computational efficiency improved by 60%, and prediction performance increased by 2.7%