Cloud Segmentation and Classification from All-Sky Images Using Deep Learning

For transforming the energy sector towards renewable energies, solar power is regarded as one of the major resources. However, it is not uniformly available all the time, leading to fluctuations in power generation. Clouds have the highest impact on short-term temporal and spatial variability. Thus, forecasting solar irradiance strongly depends on current cloudiness conditions. As the share of solar energy in the electrical grid is increasing, so-called nowcasts (intra-minute to intra-hour forecasts) are beneficial for grid control and for reducing required storage capacities. Furthermore, the operation of concentrating solar power (CSP) plants can be optimized with high resolution spatial solar irradiance data. A common nowcast approach is to analyze ground-based sky images from All-Sky Imagers. Clouds within these images are detected and tracked to estimate current and immediate future irradiance, whereas the accuracy of these forecasts depends primarily on the quality of pixel-level cloud recognition. State-of-the-art methods are commonly restricted to binary segmentation, distinguishing between cloudy and cloudless pixels. Thereby the optical properties of different cloud types are ignored. Also, most techniques rely on threshold-based detection showing difficulties under certain atmospheric conditions. In this thesis, two deep learning approaches are presented to automatically determine cloud conditions. To identify cloudiness characteristics like a free sun disk, a multi-label classifier was implemented assigning respective labels to images. In addition, a segmentation model was developed, classifying images pixel-wise into three cloud types and cloud-free sky. For supervised training, a new dataset of 770 images was created containing ground truth labels and segmentation masks. Moreover, to take advantage of large amounts of raw data, self-supervised pretraining was applied. By defining suitable pretext tasks, representations of image data can be learned facilitating the distinction of cloud types. Two successful techniques were chosen for self-supervised learning: Inpainting- uperresolution and DeepCluster. Afterwards, the pretrained models were fine-tuned on the annotated dataset. To assess the effectiveness of self-supervision, a comparison with random initialization and pretrained ImageNet weights was conducted. Evaluation shows that segmentation in particular benefits from self-supervised learning, improving accuracy and IoU about 3% points compared to ImageNet pretraining. The best segmentation model was also evaluated on binary segmentation. Achieving an overall accuracy of 95.15%, a state-of-the art Clear-Sky-Library (CSL) is outperformed significantly by over 7% points

PyranoCam: Simple measurement system for all components of solar irradiance in arbitrary planes

Accurate, robust and cost-efficient measurements of different solar irradiance components in arbitrary planes are of great interest for solar energy applications. A wide range of costeffective and robust measurement systems are currently available on the market. Available measurement techniques exhibit at least one of these shortcomings: intensive maintenance, high acquisition cost, increased deviations or restrictions to single planes (global tilted irradiance). PyranoCam is a robust and inexpensive setup of a thermopile pyranometer and an all-sky imager (ASI) for measurements of GHI, DHI, DNI and GTI (for any arbitrary plane

All-sky imager based irradiance nowcasts: combining a physical and a deep learning mdoel

Improved solar irradiance nowcasts based on all-sky imagers. Hybrid physical and end-to-end machine learning (ML) model. The ML model is based on an multi-modal deep learning model combining an vision transformer (for images) with an time series transformer (for time series data). Skill score improvements >12% points are achieved. Correct detection of cloud ramp rates improved by >8% points

Cloud Detection and Tracking Based on Object Detection with Convolutional Neural Networks

Due to the need to know the availability of solar resources for the solar renewable technologies in advance, this paper presents a new methodology based on computer vision and the object detection technique that uses convolutional neural networks (EfficientDet-D2 model) to detect clouds in image series. This methodology also calculates the speed and direction of cloud motion, which allows the prediction of transients in the available solar radiation due to clouds. The convolutional neural network model retraining and validation process finished successfully, which gave accurate cloud detection results in the test. Also, during the test, the estimation of the remaining time for a transient due to a cloud was accurate, mainly due to the precise cloud detection and the accuracy of the remaining time algorithm

Applying self-supervised learning for semantic cloud segmentation of all-sky images

This work presents a new approach to exploit unlabeled image data from ground-based sky observations to train neural networks. We show that our model can detect cloud classes within images more accurately than models trained with conventional methods using small, labeled datasets only. Novel machine learning techniques as applied in this work enable training with much larger datasets, leading to improved accuracy in cloud detection and less need for manual image labeling

Combining deep learning and physical models for solar nowcasting

Sudden changes in solar irradiance on a local scale can significantly influence solar power generation. This intermittent characteristic of the solar resource is mainly caused by passing clouds and represents a challenge when solar energy is integrated into the power system. By making use of intra hour nowcasts (very short-term forecasts), changing conditions on solar irradiance can be anticipated, allowing optimized power plant operation and grid integration. All-sky imagers, capturing sky conditions at high spatial and temporal resolution, can be the basis of such nowcasting systems. However, the benefit of these nowcasting systems heavily depends on the accuracy of the predictions. In a previous work, a hybrid model combining physics-based and persistence nowcasts has proven to be advantageous. In this work, we present a novel deep learning (DL) model based on the transformer architecture for solar irradiance nowcasts and show that integrating this model into the hybrid model further improves the nowcast quality significantly. While the physics-based nowcasts are derived from a pipeline of processing steps to model clouds and anticipating their impact on solar irradiance, the DL model is completely data-driven and trained end-to-end using sequences of sky images and groundbased irradiance measurements as input. For comparison to the literature, evaluation is carried out on a benchmark dataset of 2019 from the same site. First, the nowcast quality of the DL model is analyzed independently on standard forecasting error metrics like root mean square error (RMSE), mean absolute error (MAE), mean bias error (MBE) and forecast skill. For computing the forecast skill, we used the so-called smart persistence (SP) as reference model. Reaching scores of over 28%, the DL model itself already outperforms the previous hybrid model in terms of RMSE. Next, the hybrid model, combining physics-based, DL and SP nowcasts, is evaluated on the same dataset using the same metrics. Compared to the previous hybrid model, the new hybrid model shows significant improvement over all metrics

A Benchmark of Simple Measurement Systems for Direct Irradiance

Accurate direct normal irradiance (DNI) measurements are essential for the design and the operation of concentrating solar power systems. Several measurement systems for DNI are available to users, but all commonly used systems still have drawbacks. Sun trackers with pyranometers and a pyrheliometer are expensive and require permanent checks and maintenance by qualified personnel, for example due to tracking errors and soiling effects. Simpler, i.e. more economic and robust sensors may have shortcomings regarding accuracy under various atmospheric conditions and might not be significantly less susceptible to soiling and user errors. Validations and benchmarking of simple radiometers for solar energy applications have been presented. To the best of our knowledge, no benchmarking study is available which evaluates some more recent simple measurement systems which are relevant for solar applications in 2023. Furthermore, most previous benchmarking studies did not measure atmospheric parameters like circumsolar irradiance which may directly influence the measurements of these sensors. We close this gap by benchmarking relevant measurement systems (Rotating Shadowband Irradiometer RSI and Rotating Shadowband Pyranometer RSP 4G; Delta-T SPN1, EKO MS-90, PyranoCam, Sunto CaptPro) at multiple sites. We also evaluate the influence of relevant atmospheric parameters which we measure with dedicated instruments at one site. We include the PyranoCam system in our benchmarking, a novel radiometer system suitable for all solar irradiance components including DNI. It consists of a pyranometer and a fisheye camera that takes photos of the whole sky and employs a combined physical and machine-learning model. The results of the study provide improved estimates of the sensors’ accuracies for a specific application and climatic condition and can assist in the development of corrections for the sensor technologies

Accelerating Energy-Economic Simulation Models via Machine Learning-Based Emulation and Time Series Aggregation

Energy-economic simulation models with high levels of detail, high time resolutions, or large populations (e.g., distribution networks, households, electric vehicles, energy communities) are often limited due to their computational complexity. This paper introduces a novel methodology, combining cluster-based time series aggregation and sampling methods, to efficiently emulate simulation models using machine learning and significantly reduce both simulation and training time. Machine learning-based emulation models require sufficient and high-quality data to generalize the dataset. Since simulations are computationally complex, their maximum number is limited. Sampling methods come into play when selecting the best parameters for a limited number of simulations ex ante. This paper introduces and compares multiple sampling methods on three energy-economic datasets and shows their advantage over a simple random sampling for small sample-sizes. The results show that a k-means cluster sampling approach (based on unsupervised learning) and adaptive sampling (based on supervised learning) achieve the best results especially for small sample sizes. While a k-means cluster sampling is simple to implement, it is challenging to increase the sample sizes if the emulation model does not achieve sufficient accuracy. The iterative adaptive sampling is more complex during implementation, but can be re-applied until a certain accuracy threshold is met. Emulation is then applied on a case study, emulating an energy-economic simulation framework for peer-to-peer pricing models in Germany. The evaluated pricing models are the “supply and demand ratio” (SDR) and “mid-market rate pricing” (MMR). A time series aggregation can reduce time series data of municipalities by 99.4% with less than 5% error for 98.2% (load) and 95.5% (generation) of all municipalities and hence decrease the simulation time needed to create sufficient training data. This paper combines time series aggregation and emulation in a novel approach and shows significant acceleration by up to 88.9% of the model’s initial runtime for the simulation of the entire population of around 12,000 municipalities. The time for re-calculating the population (e.g., for different scenarios or sensitivity analysis) can be increased by a factor of 1100 while still retaining high accuracy. The analysis of the simulation time shows that time series aggregation and emulation, considered individually, only bring minor improvements in the runtime but can, however, be combined effectively. This can significantly speed up both the simulation itself and the training of the emulation model and allows for flexible use, depending on the capabilities of the models and the practitioners. The results of the peer-to-peer pricing for approximately 12,000 German municipalities show great potential for energy communities. The mechanisms offer good incentives for the addition of necessary flexibility

Nowcasting systems for irradiance ramp event detection

For the operation of a CSP plant, short-term variability of incoming solar irradiance due to passing clouds can be challenging. Scattered cloud conditions, in particular, result in rapidly and continuously changing direct normal irradiance (DNI) distributions across the solar field, which impede optimal plant control. Therefore, the anticipation of such ramp events is crucial for optimized and fully automated control approaches. While recent works based on data-driven methods and hybridization have shown significant improvements in standard forecasting metrics, ramp event detection has often been neglected. In this work we refer to a simple definition for ramp event detection based on a predefined threshold value of change rate in DNI. We present the validation results of a state-of-the-art hybrid nowcasting system, combining physics-based and data-driven nowcasts and a novel generative deep learning approach to detect ramp events. The results confirm that data-driven and hybrid models optimzed on reducing RMSE do not work well to predict ramp events whereas the novel generative model shows good results in anticipating such ramp events