Global soil moisture data derived through machine learning trained with in-situ measurements

Measurement(s) wetness of soil Technology Type(s) machine learning Factor Type(s) soil layer • temporal interval • geographic location Sample Characteristic - Environment soil Machine-accessible metadata file describing the reported data: https://doi.org/10.6084/m9.figshare.1479051

Enhancing Surface Soil Moisture Estimation through Integration of Artificial Neural Networks Machine Learning and Fusion of Meteorological, Sentinel-1A and Sentinel-2A Satellite Data

For many environmental and agricultural applications, an accurate estimation of surface soil moisture is essential. This study sought to determine whether combining Sentinel-1A, Sentinel-2A, and meteorological data with artificial neural networks (ANN) could improve soil moisture estimation in various land cover types. To train and evaluate the model’s performance, we used field data (provided by La Tuscia University) on the study area collected during time periods between October 2022, and December 2022. Surface soil moisture was measured at 29 locations. The performance of the model was trained, validated, and tested using input features in a 60:10:30 ratio, using the feed-forward ANN model. It was found that the ANN model exhibited high precision in predicting soil moisture. The model achieved a coefficient of determination (R2) of 0.71 and correlation coefficient (R) of 0.84. Furthermore, the incorporation of Random Forest (RF) algorithms for soil moisture prediction resulted in an improved R2 of 0.89. The unique combination of active microwave, meteorological data and multispectral data provides an opportunity to exploit the complementary nature of the datasets. Through preprocessing, fusion, and ANN modeling, this research contributes to advancing soil moisture estimation techniques and providing valuable insights for water resource management and agricultural planning in the study area

The future of Earth observation in hydrology

In just the past 5 years, the field of Earth observation has progressed beyond the offerings of conventional space-agency-based platforms to include a plethora of sensing opportunities afforded by CubeSats, unmanned aerial vehicles (UAVs), and smartphone technologies that are being embraced by both for-profit companies and individual researchers. Over the previous decades, space agency efforts have brought forth well-known and immensely useful satellites such as the Landsat series and the Gravity Research and Climate Experiment (GRACE) system, with costs typically of the order of 1 billion dollars per satellite and with concept-to-launch timelines of the order of 2 decades (for new missions). More recently, the proliferation of smart-phones has helped to miniaturize sensors and energy requirements, facilitating advances in the use of CubeSats that can be launched by the dozens, while providing ultra-high (3-5 m) resolution sensing of the Earth on a daily basis. Start-up companies that did not exist a decade ago now operate more satellites in orbit than any space agency, and at costs that are a mere fraction of traditional satellite missions. With these advances come new space-borne measurements, such as real-time high-definition video for tracking air pollution, storm-cell development, flood propagation, precipitation monitoring, or even for constructing digital surfaces using structure-from-motion techniques. Closer to the surface, measurements from small unmanned drones and tethered balloons have mapped snow depths, floods, and estimated evaporation at sub-metre resolutions, pushing back on spatio-temporal constraints and delivering new process insights. At ground level, precipitation has been measured using signal attenuation between antennae mounted on cell phone towers, while the proliferation of mobile devices has enabled citizen scientists to catalogue photos of environmental conditions, estimate daily average temperatures from battery state, and sense other hydrologically important variables such as channel depths using commercially available wireless devices. Global internet access is being pursued via high-altitude balloons, solar planes, and hundreds of planned satellite launches, providing a means to exploit the "internet of things" as an entirely new measurement domain. Such global access will enable real-time collection of data from billions of smartphones or from remote research platforms. This future will produce petabytes of data that can only be accessed via cloud storage and will require new analytical approaches to interpret. The extent to which today's hydrologic models can usefully ingest such massive data volumes is unclear. Nor is it clear whether this deluge of data will be usefully exploited, either because the measurements are superfluous, inconsistent, not accurate enough, or simply because we lack the capacity to process and analyse them. What is apparent is that the tools and techniques afforded by this array of novel and game-changing sensing platforms present our community with a unique opportunity to develop new insights that advance fundamental aspects of the hydrological sciences. To accomplish this will require more than just an application of the technology: in some cases, it will demand a radical rethink on how we utilize and exploit these new observing systems

Soil moisture deficit estimation using satellite multi-angle brightness temperature

Accurate soil moisture information is critically important for hydrological modelling. Although remote sensing soil moisture measurement has become an important data source, it cannot be used directly in hydrological modelling. A novel study based on nonlinear techniques (a local linear regression (LLR) and two feedforward artificial neural networks (ANNs)) is carried out to estimate soil moisture deficit (SMD), using the Soil Moisture and Ocean Salinity (SMOS) multi-angle brightness temperatures (Tbs) with both horizontal (H) and vertical (V) polarisations. The gamma test is used for the first time to determine the optimum number of Tbs required to construct a reliable smooth model for SMD estimation, and the relationship between model input and output is achieved through error variance estimation. The simulated SMD time series in the study area is from the Xinanjiang hydrological model. The results have shown that LLR model is better at capturing the interrelations between SMD and Tbs than ANNs, with outstanding statistical performances obtained during both training (NSE = 0.88, r = 0.94, RMSE = 0.008 m) and testing phases (NSE = 0.85, r = 0.93, RMSE = 0.009 m). Nevertheless, both ANN training algorithms (radial BFGS and conjugate gradient) have performed well in estimating the SMD data and showed excellent performances compared with those derived directly from the SMOS soil moisture products. This study has also demonstrated the informative capability of the gamma test in the input data selection for model development. These results provide interesting perspectives for data-assimilation in flood-forecasting

Delineation of high resolution climate regions over the Korean Peninsula using machine learning approaches

In this research, climate classification maps over the Korean Peninsula at 1 km resolution were generated using the satellite-based climatic variables of monthly temperature and precipitation based on machine learning approaches. Random forest (RF), artificial neural networks (ANN), k-nearest neighbor (KNN), logistic regression (LR), and support vector machines (SVM) were used to develop models. Training and validation of these models were conducted using in-situ observations from the Korea Meteorological Administration (KMA) from 2001 to 2016. The rule of the traditional Koppen-Geiger (K-G) climate classification was used to classify climate regions. The input variables were land surface temperature (LST) of the Moderate Resolution Imaging Spectroradiometer (MODIS), monthly precipitation data from the Tropical Rainfall Measuring Mission (TRMM) 3B43 product, and the Digital Elevation Map (DEM) from the Shuttle Radar Topography Mission (SRTM). The overall accuracy (OA) based on validation data from 2001 to 2016 for all models was high over 95%. DEM and minimum winter temperature were two distinct variables over the study area with particularly high relative importance. ANN produced more realistic spatial distribution of the classified climates despite having a slightly lower OA than the others. The accuracy of the models using high altitudinal in-situ data of the Mountain Meteorology Observation System (MMOS) was also assessed. Although the data length of the MMOS data was relatively short (2013 to 2017), it proved that the snowy, dry and cold winter and cool summer class (Dwc) is widely located in the eastern coastal region of South Korea. Temporal shifting of climate was examined through a comparison of climate maps produced by period: from 1950 to 2000, from 1983 to 2000, and from 2001 to 2013. A shrinking trend of snow classes (D) over the Korean Peninsula was clearly observed from the ANN-based climate classification results. Shifting trends of climate with the decrease/increase of snow (D)/temperate (C) classes were clearly shown in the maps produced using the proposed approaches, consistent with the results from the reanalysis data of the Climatic Research Unit (CRU) and Global Precipitation Climatology Centre (GPCC)

The International Soil Moisture Network:Serving Earth system science for over a decade

In 2009, the International Soil Moisture Network (ISMN) was initiated as a community effort, funded by the European Space Agency, to serve as a centralised data hosting facility for globally available in situ soil moisture measurements (Dorigo et al., 2011b, a). The ISMN brings together in situ soil moisture measurements collected and freely shared by a multitude of organisations, harmonises them in terms of units and sampling rates, applies advanced quality control, and stores them in a database. Users can freely retrieve the data from this database through an online web portal (https://ismn.earth/en/, last access: 28 October 2021). Meanwhile, the ISMN has evolved into the primary in situ soil moisture reference database worldwide, as evidenced by more than 3000 active users and over 1000 scientific publications referencing the data sets provided by the network. As of July 2021, the ISMN now contains the data of 71 networks and 2842 stations located all over the globe, with a time period spanning from 1952 to the present. The number of networks and stations covered by the ISMN is still growing, and approximately 70 % of the data sets contained in the database continue to be updated on a regular or irregular basis. The main scope of this paper is to inform readers about the evolution of the ISMN over the past decade, including a description of network and data set updates and quality control procedures. A comprehensive review of the existing literature making use of ISMN data is also provided in order to identify current limitations in functionality and data usage and to shape priorities for the next decade of operations of this unique community-based data repository

Soil moisture estimation synergy using GNSS-R and L-Band microwave radiometry data from FSSCat/FMPL-2

The Federated Satellite System mission (FSSCat) was the winner of the 2017 Copernicus Masters Competition and the first Copernicus third-party mission based on CubeSats. One of FSSCat’s objectives is to provide coarse Soil Moisture (SM) estimations by means of passive microwave measurements collected by Flexible Microwave Payload-2 (FMPL-2). This payload is a novel CubeSat based instrument combining an L1/E1 Global Navigation Satellite Systems-Reflectometer (GNSS-R) and an L-band Microwave Radiometer (MWR) using software-defined radio. This work presents the first results over land of the first two months of operations after the commissioning phase, from 1 October to 4 December 2020. Four neural network algorithms are implemented and analyzed in terms of different sets of input features to yield maps of SM content over the Northern Hemisphere (latitudes above 45° N). The first algorithm uses the surface skin temperature from the European Centre of Medium-Range Weather Forecast (ECMWF) in conjunction with the 16 day averaged Normalized Difference Vegetation Index (NDVI) from the Moderate Resolution Imaging Spectroradiometer (MODIS) to estimate SM and to use it as a comparison dataset for evaluating the additional models. A second approach is implemented to retrieve SM, which complements the first model using FMPL-2 L-band MWR antenna temperature measurements, showing a better performance than in the first case. The error standard deviation of this model referred to the Soil Moisture and Ocean Salinity (SMOS) SM product gridded at 36 km is 0.074 m3/m3. The third algorithm proposes a new approach to retrieve SM using FMPL-2 GNSS-R data. The mean and standard deviation of the GNSS-R reflectivity are obtained by averaging consecutive observations based on a sliding window and are further included as additional input features to the network. The model output shows an accurate SM estimation compared to a 9 km SMOS SM product, with an error of 0.087 m3/m3. Finally, a fourth model combines MWR and GNSS-R data and outperforms the previous approaches, with an error of just 0.063 m3/m3. These results demonstrate the capabilities of FMPL-2 to provide SM estimates over land with a good agreement with respect to SMOS SM.This work was supported by the 2017 ESA S3 challenge and Copernicus Masters overall winner award (“FSSCat” project). This work was (partially) sponsored by project SPOT: Sensing with Pioneering Opportunistic Techniques grant RTI2018-099008-B-C21 / AEI / 10.13039/501100011033, and by the Unidad de Excelencia Maria de Maeztu MDM-2016-0600. This work was also (partially) sponsored by the Spanish Ministry of Science and Innovation through the project ESP2017-89463-C3, by the Centro de Excelencia Severo Ochoa (CEX2019-000928-S), and by the CSIC Plataforma Temática Interdisciplinar de Teledetección (PTI-Teledetect). Joan Francesc Munoz-Martin received support from the grant for the recruitment of early-stage research staff FI-DGR 2018 of the AGAUR - Generalitat de Catalunya (FEDER), Spain; Christoph Herbert received the support of a fellowship from “la Caixa” Foundation (ID 100010434) with the fellowship code LCF/BQ/DI18/11660050 and funding from the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie Grant Agreement No. 713673; David Llavería received support from an FPU fellowship from the Spanish Ministry of Education FPU18/06107.Peer ReviewedPostprint (published version

Désagrégation de l'humidité du sol issue des produits satellitaires micro-ondes passives et exploration de son utilisation pour l'amélioration de la modélisation et la prévision hydrologique

De plus en plus de produits satellitaires en micro-ondes passives sont disponibles. Cependant, leur large résolution spatiale (25-50 km) n’en font pas un outil adéquat pour des applications hydrologiques à une échelle locale telles que la modélisation et la prévision hydrologiques. Dans de nombreuses études, une désagrégation d’échelle de l’humidité du sol des produits satellites micro-ondes est faite puis validée avec des mesures in-situ. Toutefois, l’utilisation de ces données issues d’une désagrégation d’échelle n’a pas encore été pleinement étudiée pour des applications en hydrologie. Ainsi, l’objectif de cette thèse est de proposer une méthode de désagrégation d’échelle de l’humidité du sol issue de données satellitaires en micro-ondes passives (Satellite Passive Microwave Active and Passive - SMAP) à différentes résolutions spatiales afin d’évaluer leur apport sur l’amélioration potentielle des modélisations et prévisions hydrologiques. À partir d’un modèle de forêt aléatoire, une désagrégation d’échelle de l’humidité du sol de SMAP l’amène de 36-km de résolution initialement à des produits finaux à 9-, 3- et 1-km de résolution. Les prédicteurs utilisés sont à haute résolution spatiale et de sources différentes telles que Sentinel-1A, MODIS et SRTM. L'humidité du sol issue de cette désagrégation d’échelle est ensuite assimilée dans un modèle hydrologique distribué à base physique pour tenter d’améliorer les sorties de débit. Ces expériences sont menées sur les bassins versants des rivières Susquehanna (de grande taille) et Upper-Susquehanna (en comparaison de petite taille), tous deux situés aux États-Unis. De plus, le modèle assimile aussi des données d’humidité du sol en profondeur issue d’une extrapolation verticale des données SMAP. Par ailleurs, les données d’humidité du sol SMAP et les mesures in-situ sont combinées par la technique de fusion conditionnelle. Ce produit de fusion SMAP/in-situ est assimilé dans le modèle hydrologique pour tenter d’améliorer la prévision hydrologique sur le bassin versant Au Saumon situé au Québec. Les résultats montrent que l'utilisation de l’humidité du sol à fine résolution spatiale issue de la désagrégation d’échelle améliore la représentation de la variabilité spatiale de l’humidité du sol. En effet, le produit à 1- km de résolution fournit plus de détails que les produits à 3- et 9-km ou que le produit SMAP de base à 36-km de résolution. De même, l’utilisation du produit de fusion SMAP/ in-situ améliore la qualité et la représentation spatiale de l’humidité du sol. Sur le bassin versant Susquehanna, la modélisation hydrologique s’améliore avec l’assimilation du produit de désagrégation d’échelle à 9-km, sans avoir recours à des résolutions plus fines. En revanche, sur le bassin versant Upper-Susquehanna, c’est le produit avec la résolution spatiale la plus fine à 1- km qui offre les meilleurs résultats de modélisation hydrologique. L’assimilation de l’humidité du sol en profondeur issue de l’extrapolation verticale des données SMAP n’améliore que peu la qualité du modèle hydrologique. Par contre, l’assimilation du produit de fusion SMAP/in-situ sur le bassin versant Au Saumon améliore la qualité de la prévision du débit, même si celle-ci n’est pas très significative.Abstract: The availability of satellite passive microwave soil moisture is increasing, yet its spatial resolution (i.e., 25-50 km) is too coarse to use for local scale hydrological applications such as streamflow simulation and forecasting. Many studies have attempted to downscale satellite passive microwave soil moisture products for their validation with in-situ soil moisture measurements. However, their use for hydrological applications has not yet been fully explored. Thus, the objective of this thesis is to downscale the satellite passive microwave soil moisture (i.e., Satellite Microwave Active and Passive - SMAP) to a range of spatial resolutions and explore its value in improving streamflow simulation and forecasting. The random forest machine learning technique was used to downscale the SMAP soil moisture from 36-km to 9-, 3- and 1-km spatial resolutions. A combination of host of high-resolution predictors derived from different sources including Sentinel-1A, MODIS and SRTM were used for downscaling. The downscaled SMAP soil moisture was then assimilated into a physically-based distributed hydrological model for improving streamflow simulation for Susquehanna (larger in size) and Upper Susquehanna (relatively smaller in size) watersheds, located in the United States. In addition, the vertically extrapolated SMAP soil moisture was assimilated into the model. On the other hand, the SMAP and in-situ soil moisture were merged using the conditional merging technique and the merged SMAP/in-situ soil moisture was then assimilated into the model to improve streamflow forecast over the au Saumon watershed. The results show that the downscaling improved the spatial variability of soil moisture. Indeed, the 1-km downscaled SMAP soil moisture presented a higher spatial detail of soil moisture than the 3-, 9- or original resolution (36-km) SMAP product. Similarly, the merging of SMAP and in-situ soil moisture improved the accuracy as well as spatial representation soil moisture. Interestingly, the assimilation of the 9-km downscaled SMAP soil moisture significantly improved the accuracy of streamflow simulation for the Susquehanna watershed without the need of going to higher spatial resolution, whereas for the Upper Susquehanna watershed the 1-km downscaled SMAP showed better results than the coarser resolutions. The assimilation of vertically extrapolated SMAP soil moisture only slightly further improved the accuracy of the streamflow simulation. On the other hand, the assimilation of merged SMAP/in-situ soil moisture for the au Saumon watershed improved the accuracy of streamflow forecast, yet the improvement was not that significant. Overall, this study demonstrated the potential of satellite passive microwave soil moisture for streamflow simulation and forecasting

Application of machine learning techniques to simulate the evaporative fraction and its relationship with environmental variables in corn crops

Abstract Background The evaporative fraction (EF) represents an important biophysical parameter reflecting the distribution of surface available energy. In this study, we investigated the daily and seasonal patterns of EF in a multi-year corn cultivation located in southern Italy and evaluated the performance of five machine learning (ML) classes of algorithms: the linear regression (LR), regression tree (RT), support vector machine (SVM), ensembles of tree (ETs) and Gaussian process regression (GPR) to predict the EF at daily time step. The adopted methodology consisted of three main steps that include: (i) selection of the EF predictors; (ii) comparison of the different classes of ML; (iii) application, cross-validation of the selected ML algorithms and comparison with the observed data. Results Our results indicate that SVM and GPR were the best classes of ML at predicting the EF, with a total of four different algorithms: cubic SVM, medium Gaussian SVM, the Matern 5/2 GPR, and the rational quadratic GPR. The comparison between observed and predicted EF in all four algorithms, during the training phase, were within the 95% confidence interval: the R2 value between observed and predicted EF was 0.76 (RMSE 0.05) for the medium Gaussian SVM, 0.99 (RMSE 0.01) for the rational quadratic GPR, 0.94 (RMSE 0.02) for the Matern 5/2 GPR, and 0.83 (RMSE 0.05) for the cubic SVM algorithms. Similar results were obtained during the testing phase. The results of the cross-validation analysis indicate that the R2 values obtained between all iterations for each of the four adopted ML algorithms were basically constant, confirming the ability of ML as a tool to predict EF. Conclusion ML algorithms represent a valid alternative able to predict the EF especially when remote sensing data are not available, or the sky conditions are not suitable. The application to different geographical areas, or crops, requires further development of the model based on different data sources of soils, climate, and cropping systems