A Machine Learning Approach for Improving Near-Real-Time Satellite-Based Rainfall Estimates by Integrating Soil Moisture

Near-real-time (NRT) satellite-based rainfall estimates (SREs) are a viable option for flood/drought monitoring. However, SREs have often been associated with complex and nonlinear errors. One way to enhance the quality of SREs is to use soil moisture information. Few studies have indicated that soil moisture information can be used to improve the quality of SREs. Nowadays, satellite-based soil moisture products are becoming available at desired spatial and temporal resolutions on an NRT basis. Hence, this study proposes an integrated approach to improve NRT SRE accuracy by combining it with NRT soil moisture through a nonlinear support vector machine-based regression (SVR) model. To test this novel approach, Ashti catchment, a sub-basin of Godavari river basin, India, is chosen. Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA)-based NRT SRE 3B42RT and Advanced Scatterometer-derived NRT soil moisture are considered in the present study. The performance of the 3B42RT and the corrected product are assessed using different statistical measures such as correlation coeffcient (CC), bias, and root mean square error (RMSE), for the monsoon seasons of 2012–2015. A detailed spatial analysis of these measures and their variability across different rainfall intensity classes are also presented. Overall, the results revealed significant improvement in the corrected product compared to 3B42RT (except CC) across the catchment. Particularly, for light and moderate rainfall classes, the corrected product showed the highest improvement (except CC). On the other hand, the corrected product showed limited performance for the heavy rainfall class. These results demonstrate that the proposed approach has potential to enhance the quality of NRT SRE through the use of NRT satellite-based soil moisture estimates

Uncertainty Management of Intelligent Feature Selection in Wireless Sensor Networks

Wireless sensor networks (WSN) are envisioned to revolutionize the paradigm of monitoring complex real-world systems at a very high resolution. However, the deployment of a large number of unattended sensor nodes in hostile environments, frequent changes of environment dynamics, and severe resource constraints pose uncertainties and limit the potential use of WSN in complex real-world applications. Although uncertainty management in Artificial Intelligence (AI) is well developed and well investigated, its implications in wireless sensor environments are inadequately addressed. This dissertation addresses uncertainty management issues of spatio-temporal patterns generated from sensor data. It provides a framework for characterizing spatio-temporal pattern in WSN. Using rough set theory and temporal reasoning a novel formalism has been developed to characterize and quantify the uncertainties in predicting spatio-temporal patterns from sensor data. This research also uncovers the trade-off among the uncertainty measures, which can be used to develop a multi-objective optimization model for real-time decision making in sensor data aggregation and samplin

Data Reduction and Deep-Learning Based Recovery for Geospatial Visualization and Satellite Imagery

The storage, retrieval and distribution of data are some critical aspects of big data management. Data scientists and decision-makers often need to share large datasets and make decisions on archiving or deleting historical data to cope with resource constraints. As a consequence, there is an urgency of reducing the storage and transmission requirement. A potential approach to mitigate such problems is to reduce big datasets into smaller ones, which will not only lower storage requirements but also allow light load transfer over the network. The high dimensional data often exhibit high repetitiveness and paradigm across different dimensions. Carefully prepared data by removing redundancies, along with a machine learning model capable of reconstructing the whole dataset from its reduced version, can improve the storage scalability, data transfer, and speed up the overall data management pipeline. In this thesis, we explore some data reduction strategies for big datasets, while ensuring that the data can be transferred and used ubiquitously by all stakeholders, i.e., the entire dataset can be reconstructed with high quality whenever necessary. One of our data reduction strategies follows a straightforward uniform pattern, which guarantees a minimum of 75% data size reduction. We also propose a novel variance based reduction technique, which focuses on removing only redundant data and offers additional 1% to 2% deletion rate. We have adopted various traditional machine learning and deep learning approaches for high-quality reconstruction. We evaluated our pipelines with big geospatial data and satellite imageries. Among them, our deep learning approaches have performed very well both quantitatively and qualitatively with the capability of reconstructing high quality features. We also show how to leverage temporal data for better reconstruction. For uniform deletion, the reconstruction accuracy observed is as high as 98.75% on an average for spatial meteorological data (e.g., soil moisture and albedo), and 99.09% for satellite imagery. Pushing the deletion rate further by following variance based deletion method, the decrease in accuracy remains within 1% for spatial meteorological data and 7% for satellite imagery

Modeling Soil Water Content and Reference Evapotranspiration from Climate Data Using Deep Learning Method

In recent years, deep learning algorithms have been successfully applied in the development of decision support systems in various aspects of agriculture, such as yield estimation, crop diseases, weed detection, etc. Agriculture is the largest consumer of freshwater. Due to challenges such as lack of natural resources and climate change, an efficient decision support system for irrigation is crucial. Evapotranspiration and soil water content are the most critical factors in irrigation scheduling. In this paper, the ability of Long Short-Term Memory (LSTM) and Bidirectional LSTM (BLSTM) to model daily reference evapotranspiration and soil water content is investigated. The application of these techniques to predict these parameters was tested for three sites in Portugal. A single-layer BLSTM with 512 nodes was selected. Bayesian optimization was used to determine the hyperparameters, such as learning rate, decay, batch size, and dropout size.The model achieved the values of mean square error values within the range of 0.014 to 0.056 and R2 ranging from 0.96 to 0.98. A Convolutional Neural Network (CNN) model was added to the LSTM to investigate potential performance improvement. Performance dropped in all datasets due to the complexity of the model. The performance of the models was also compared with CNN, traditional machine learning algorithms Support Vector Regression, and Random Forest. LSTM achieved the best performance. Finally, the impact of the loss function on the performance of the proposed models was investigated. The model with the mean square error as loss function performed better than the model with other loss functions.Project Centro-01-0145-FEDER000017-EMaDeS-Energy, Materials, and Sustainable Development, co-funded by the Portugal 2020 Program (PT 2020), within the Regional Operational Program of the Center (CENTRO 2020) and the EU through the European Regional Development Fund (ERDF). Fundação para a Ciência e a Tecnologia (FCT—MCTES) also provided financial support via project UIDB/00151/2020 (C-MAST).info:eu-repo/semantics/publishedVersio

Drought Forecasting: A Review and Assessment of the Hybrid Techniques and Data Pre-processing

Drought is a prolonged period of low precipitation that negatively impacts agriculture, animals, and people. Over the last decades, gradual changes in drought indices have been observed. Therefore, understanding and forecasting drought is essential to avoid its economic impacts and appropriate water resource planning and management. This paper presents a recent literature review, including a brief description of data pre-processing, data-driven modelling strategies (i.e., univariate or multivariate), machine learning algorithms (i.e., advantages and disadvantages), hybrid models, and performance metrics. Combining various prediction methods to create efficient hybrid models has become the most popular use in recent years. Accordingly, hybrid models have been increasingly used for predicting drought. As such, these models will be extensively reviewed, including preprocessing-based hybrid models, parameter optimisation-based hybrid models, and hybridisation of components combination-based with preprocessing-based hybrid models. In addition, using statistical criteria, such as RMSE, MAE, NSE, MPE, SI, BIC, AIC, and AAD, is essential to evaluate the performance of the models

Soil Moisture Measuring Techniques and Factors Affecting the Moisture Dynamics: A Comprehensive Review

The amount of surface soil moisture (SSM) is a crucial ecohydrological natural resource that regulates important land surface processes. It affects critical land–atmospheric phenomena, including the division of energy and water (infiltration, runoff, and evaporation), that impacts the effectiveness of agricultural output (sensible and latent heat fluxes and surface air temperature). Despite its significance, there are several difficulties in making precise measurements, monitoring, and interpreting SSM at high spatial and temporal resolutions. The current study critically reviews the methods and procedures for calculating SSM and the variables influencing measurement accuracy and applicability under different fields, climates, and operational conditions. For laboratory and field measurements, this study divides SSM estimate strategies into (i) direct and (ii) indirect procedures. The accuracy and applicability of a technique depends on the environment and the resources at hand. Comparative research is geographically restricted, although precise and economical—direct measuring techniques like the gravimetric method are time-consuming and destructive. In contrast, indirect methods are more expensive and do not produce measurements at the spatial scale but produce precise data on a temporal scale. While measuring SSM across more significant regions, ground-penetrating radar and remote sensing methods are susceptible to errors caused by overlapping data and atmospheric factors. On the other hand, soft computing techniques like machine/deep learning are quite handy for estimating SSM without any technical or laborious procedures. We determine that factors, e.g., topography, soil type, vegetation, climate change, groundwater level, depth of soil, etc., primarily influence the SSM measurements. Different techniques have been put into practice for various practical situations, although comparisons between them are not available frequently in publications. Each method offers a unique set of potential advantages and disadvantages. The most accurate way of identifying the best soil moisture technique is the value selection method (VSM). The neutron probe is preferable to the FDR or TDR sensor for measuring soil moisture. Remote sensing techniques have filled the need for large-scale, highly spatiotemporal soil moisture monitoring. Through self-learning capabilities in data-scarce areas, machine/deep learning approaches facilitate soil moisture measurement and prediction

Optimized Preprocessing using Time Variant Particle Swarm Optimization (TVPSO) and Deep Learning on Rainfall Data

In the recent past, rainfall prediction has played a significant role in the meteorology department. Changes in rainfall might affect the world's manufacturing and service sectors. Rainfall prediction is a substantial progression in giving input data for weather information and hydrological development applications. In machine learning, accurate and efficient rainfall predictionis used to support strategy for watershed management. The prediction of rain is a problematic occurrence and endures to be a challenging task. This paper implements a novel algorithm for preprocessing and optimization using historical weather from a collection of various weather parameters. The Moving Average-Probabilistic Regression Filtering (MV-PRF) method eliminates unwanted samples with less amplitude from the database. The Time Variant Particle Swarm Optimization (TVPSO) model optimizes the preprocessing rainfall data. Then this optimized data is used for the different classification processes. The preprocessing methods emphasize the recent rainfall data of the time series to improve the rainfall forecast using classification methods. Machine Learning (ML) technique classifies the weather parameters to predict rainfall daily or monthly. These experimental results show that the proposed methods are efficient and accurate for rainfall analysis

ANALYZING THE RELATIONSHIP BETWEEN LARGE SCALE CLIMATE VARIABILITY AND STREAMFLOW OF THE CONTINENTAL UNITED STATES

Over the years there is an increasing evidence of climate change on the available water resources. The interaction of hydrological cycle with climate variability and change may provide information related with several water management issues. The current study analyzes streamflow variability of the United States due to large-scale ocean-atmospheric climate variability. In addition, forecast lead-time is also improved by coupling climate information in a data driven modeling framework. The spatial-temporal correlation between streamflow and oceanic-atmospheric variability represented by sea surface temperature (SST), 500-mbar geopotential height (Z500), 500-mbar specific humidity (SH500), and 500-mbar east-west wind (U500) of the Pacific and the Atlantic Ocean is obtained through singular value decomposition (SVD). For forecasting of streamflow, SVD significant regions are weighted using a non-parametric method and utilized as input in a support vector machine (SVM) framework. The Upper Rio Grande River Basin (URGRB) is selected to test the applicability of the proposed forecasting model for the period of 1965-2014. The April-August streamflow volume is forecasted using previous year climate variability, creating a lagged relationship of 1-13 months. To understand the effect of predefined indices such as El Nino Southern Oscillation (ENSO), Pacific Decadal Oscillation (PDO), and Atlantic Multidecadal Oscillation (AMO) on the regional streamflow, a wavelet analysis is also performed for regions developed by from 2014 National Climate Assessment (NCA). Moreover, different SVD approach is performed for streamflow of each of the six NCA regions named as Great Plains, Midwest, Northeast, Northwest, Southeast, and Southwest. In regional case, SVD is applied initially with streamflow and SST; and that spatial-temporal correlation is later correlated with Z500, SH500, and U500 separately to evaluate the interconnections between climate variables. SVD result showed that the streamflow variability of the URGRB was better explained by SST and U500 as compared to Z500 and SH500. The SVM model showed satisfactory forecasting ability as the observed and forecasted streamflow volume for different selected sites were well correlated. The best results were achieved using a 1-month lead to forecast the following 4-month period. Overall, the SVM results showed excellent predictive ability with average linear correlation coefficient of 0.89 and Nash-Sutcliffe efficiency of 0.79. Whereas regional SVD analysis showed that streamflow variability in the Great Plains, Midwest, and Southwest region is strongly associated with SST of ENSO-like region. However, for Northeast and Southeast region, U500 and SH500 were strongly correlated with streamflow as compared to the SST of the Pacific Ocean. The continuous wavelet analysis of ENSO/PDO/AMO and the regional streamflow patterns revealed different significant timescale bands that affected their variation over the study period. Identification of several teleconnected regions of the climate variables and the association with the streamflow can be helpful to improve long-term prediction of streamflow resulting in better management of water resources in the regional scale