Big Data Analysis application in the renewable energy market: wind power

Entre as enerxías renovables, a enerxía eólica e unha das tecnoloxías mundiais de rápido crecemento. Non obstante, esta incerteza debería minimizarse para programar e xestionar mellor os activos de xeración tradicionais para compensar a falta de electricidade nas redes electricas. A aparición de técnicas baseadas en datos ou aprendizaxe automática deu a capacidade de proporcionar predicións espaciais e temporais de alta resolución da velocidade e potencia do vento. Neste traballo desenvólvense tres modelos diferentes de ANN, abordando tres grandes problemas na predición de series de datos con esta técnica: garantía de calidade de datos e imputación de datos non válidos, asignación de hiperparámetros e selección de funcións. Os modelos desenvolvidos baséanse en técnicas de agrupación, optimización e procesamento de sinais para proporcionar predicións de velocidade e potencia do vento a curto e medio prazo (de minutos a horas)

spateGAN: spatio‐temporal downscaling of rainfall fields using a cGAN approach

Climate models face limitations in their ability to accurately represent highly variable atmospheric phenomena. To resolve fine-scale physical processes, allowing for local impact assessments, downscaling techniques are essential. We propose spateGAN, a novel approach for spatio-temporal downscaling of precipitation data using conditional generative adversarial networks. Our method is based on a video super-resolution approach and trained on 10 years of country-wide radar observations for Germany. It simultaneously increases the spatial and temporal resolution of coarsened precipitation observations from 32 to 2 km and from 1 hr to 10 min. Our experiments indicate that the ensembles of generated temporally consistent rainfall fields are in high agreement with the observational data. Spatial structures with plausible advection were accurately generated. Compared to trilinear interpolation and a classical convolutional neural network, the generative model reconstructs the resolution-dependent extreme value distribution with high skill. It showed a high fractions skill score of 0.6 (spatio-temporal scale: 32 km and 1 hr) for rainfall intensities over 15 mm h−1 and a low relative bias of 3.35%. A power spectrum analysis confirmed that the probabilistic downscaling ability of our model further increased its skill. We observed that neural network predictions may be interspersed by recurrent structures not related to rainfall climatology, which should be a known issue for future studies. We were able to mitigate them by using an appropriate model architecture and model selection process. Our findings suggest that spateGAN offers the potential to complement and further advance the development of climate model downscaling techniques, due to its performance and computational efficiency

TempNet – Temporal Super-resolution Of Radar Rainfall Products With Residual CNNs

The temporal and spatial resolution of rainfall data is crucial for environmental modeling studies in which its variability in space and time is considered as a primary factor. Rainfall products from different remote sensing instruments (e.g., radar, satellite) have different space-time resolutions because of the differences in their sensing capabilities and post-processing methods. In this study, we developed a deep-learning approach that augments rainfall data with increased time resolutions to complement relatively lower-resolution products. We propose a neural network architecture based on Convolutional Neural Networks (CNNs), namely TempNet, to improve the temporal resolution of radar-based rainfall products and compare the proposed model with an optical flow-based interpolation method and CNN-baseline model. While TempNet achieves a mean absolute error of 0.332 mm/h, comparison methods achieve 0.35 and 0.341, respectively. The methodology presented in this study could be used for enhancing rainfall maps with better temporal resolution and imputation of missing frames in sequences of 2D rainfall maps to support hydrological and flood forecasting studies

Flood Forecasting Using Machine Learning Methods

This book is a printed edition of the Special Issue Flood Forecasting Using Machine Learning Methods that was published in Wate

TempNet -- Temporal Super Resolution of Radar Rainfall Products with Residual CNNs

The temporal and spatial resolution of rainfall data is crucial for environmental modeling studies in which its variability in space and time is considered as a primary factor. Rainfall products from different remote sensing instruments (e.g., radar, satellite) have different space-time resolutions because of the differences in their sensing capabilities and post-processing methods. In this study, we developed a deep learning approach that augments rainfall data with increased time resolutions to complement relatively lower resolution products. We propose a neural network architecture based on Convolutional Neural Networks (CNNs) to improve the temporal resolution of radar-based rainfall products and compare the proposed model with an optical flow-based interpolation method and CNN-baseline model. The methodology presented in this study could be used for enhancing rainfall maps with better temporal resolution and imputation of missing frames in sequences of 2D rainfall maps to support hydrological and flood forecasting studies

Comparative Studies of Different Imputation Methods for Recovering Streamflow Observation

Faulty field sensors cause unreliability in the observed data that needed to calibrate and assess hydrology models. However, it is illogical to ignore abnormal or missing values if there are limited data available. This study addressed this problem by applying data imputation to replace incorrect values and recover missing streamflow information in the dataset of the Samho gauging station at Taehwa River (TR), Korea from 2004 to 2006. Soil and Water Assessment Tool (SWAT) and two machine learning techniques, Artificial Neural Network (ANN) and Self Organizing Map (SOM), were employed to estimate streamflow using reasonable flow datasets of Samho station from 2004 to 2009. The machine learning models were generally better at capturing high flows, while SWAT was better at simulating low flows.open

Modeling and analysis of actual evapotranspiration using data driven and wavelet techniques

Large-scale mining practices have disturbed many natural watersheds in northern Alberta, Canada. To restore disturbed landscapes and ecosystems’ functions, reconstruction strategies have been adopted with the aim of establishing sustainable reclaimed lands. The success of the reconstruction process depends on the design of reconstruction strategies, which can be optimized by improving the understanding of the controlling hydrological processes in the reconstructed watersheds. Evapotranspiration is one of the important components of the hydrological cycle; its estimation and analysis are crucial for better assessment of the reconstructed landscape hydrology, and for more efficient design. The complexity of the evapotranspiration process and its variability in time and space has imposed some limitations on previously developed evapotranspiration estimation models. The vast majority of the available models estimate the rate of potential evapotranspiration, which occurs under unlimited water supply condition. However, the rate of actual evapotranspiration (AET) depends on the available soil moisture, which makes its physical modeling more complicated than the potential evapotranspiration. The main objective of this study is to estimate and analyze the AET process in a reconstructed landscape. Data driven techniques can model the process without having a complete understanding of its physics. In this study, three data driven models; genetic programming (GP), artificial neural networks (ANNs), and multilinear regression (MLR), were developed and compared for estimating the hourly eddy covariance (EC)-measured AET using meteorological variables. The AET was modeled as a function of five meteorological variables: net radiation (Rn), ground temperature (Tg), air temperature (Ta), relative humidity (RH), and wind speed (Ws) in a reconstructed landscape located in northern Alberta, Canada. Several ANN models were evaluated using two training algorithms of Levenberg-Marquardt and Bayesian regularization. The GP technique was employed to generate mathematical equations correlating AET to the five meteorological variables. Furthermore, the available data were statistically analyzed to obtain MLR models and to identify the meteorological variables that have significant effect on the evapotranspiration process. The utility of the investigated data driven models was also compared with that of HYDRUS-1D model, which is a physically based model that makes use of conventional Penman-Monteith (PM) method for the prediction of AET. HYDRUS-1D model was examined for estimating AET using meteorological variables, leaf area index, and soil moisture information. Furthermore, Wavelet analysis (WA), as a multiresolution signal processing tool, was examined to improve the understanding of the available time series temporal variations, through identifying the significant cyclic features, and to explore the possible correlation between AET and the meteorological signals. WA was used with the purpose of input determination of AET models, a priori. The results of this study indicated that all three proposed data driven models were able to approximate the AET reasonably well; however, GP and MLR models had better generalization ability than the ANN model. GP models demonstrated that the complex process of hourly AET can be efficiently modeled as simple semi-linear functions of few meteorological variables. The results of HYDRUS-1D model exhibited that a physically based model, such as HYDRUS-1D, might perform on par or even inferior to the data driven models in terms of the overall prediction accuracy. The developed equation-based models; GP and MLR, revealed the larger contribution of net radiation and ground temperature, compared to other variables, to the estimation of AET. It was also found that the interaction effects of meteorological variables are important for the AET modeling. The results of wavelet analysis demonstrated the presence of both small-scale (2 to 8 hours) and larger-scale (e.g. diurnal) cyclic features in most of the investigated time series. Larger-scale cyclic features were found to be the dominant source of temporal variations in the AET and most of the meteorological variables. The results of cross wavelet analysis indicated that the cause and effect relationship between AET and the meteorological variables might vary based on the time-scale of variation under consideration. At small time-scales, significant linear correlations were observed between AET and Rn, RH, and Ws time series, while at larger time-scales significant linear correlations were observed between AET and Rn, RH, Tg, and Ta time series

Assessing the ability of MODIS EVI to estimate terrestrial ecosystem gross primary production of multiple land cover types

© 2016 Elsevier Ltd Terrestrial ecosystem gross primary production (GPP) is the largest component in the global carbon cycle. The enhanced vegetation index (EVI) has been proven to be strongly correlated with annual GPP within several biomes. However, the annual GPP-EVI relationship and associated environmental regulations have not yet been comprehensively investigated across biomes at the global scale. Here we explored relationships between annual integrated EVI (iEVI) and annual GPP observed at 155 flux sites, where GPP was predicted with a log-log model: ln(GPP)=a×ln(iEVI)+b. iEVI was computed from MODIS monthly EVI products following removal of values affected by snow or cold temperature and without calculating growing season duration. Through categorisation of flux sites into 12 land cover types, the ability of iEVI to estimate GPP was considerably improved (R2 from 0.62 to 0.74, RMSE from 454.7 to 368.2 g C m−2 yr−1). The biome-specific GPP-iEVI formulae generally showed a consistent performance in comparison to a global benchmarking dataset (R2 = 0.79, RMSE = 387.8 g C m−2 yr−1). Specifically, iEVI performed better in cropland regions with high productivity but poorer in forests. The ability of iEVI in estimating GPP was better in deciduous biomes (except deciduous broadleaf forest) than in evergreen due to the large seasonal signal in iEVI in deciduous biomes. Likewise, GPP estimated from iEVI was in a closer agreement to global benchmarks at mid and high-latitudes, where deciduous biomes are more common and cloud cover has a smaller effect on remote sensing retrievals. Across biomes, a significant and negative correlation (R2 = 0.37, p < 0.05) was observed between the strength (R2) of GPP-iEVI relationships and mean annual maximum leaf area index (LAImax), and the relationship between the strength and mean annual precipitation followed a similar trend. LAImax also revealed a scaling effect on GPP-iEVI relationships. Our results suggest that iEVI provides a very simple but robust approach to estimate spatial patterns of global annual GPP whereas its effect is comparable to various light-use-efficiency and data-driven models. The impact of vegetation structure on accuracy and sensitivity of EVI in estimating spatial GPP provides valuable clues to improve EVI-based models