Multi-sensor large scale land surface data assimilation using ensemble approaches

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Civil and Environmental Engineering, 2006.Includes bibliographical references (p. 223-234).One of the ensemble Kalman filter's (EnKF) attractive features in land surface applications is its ability to provide distributional information. The EnKF relies on normality approximations that improve its efficiency but can also compromise the accuracy of its distributional estimates. The effects of these approximations are evaluated by comparing the conditional marginal distributions and moments estimated by the EnKF to those obtained from an SIR particle filter, which gives exact solutions for large ensemble sizes. The results show that overall the EnKF appears to provide a good approximation for nonlinear, non-normal land surface problems. A difficulty in land data assimilation problems results from the high dimensionality of states created by spatial discretization over large computational grids. The high dimensionality can be reduced by exploiting the fact that soil moisture field may have significant spatial correlation structure especially after extensive rainfall while it may have local structure determined by soil and vegetation variability after prolonged drydown. This is confirmed by SVD of the replicate matrix produced in an ensemble forecasting experiment. Local EnKF's are suitable for problems during dry periods but give less accurate results after rainfall.(cont.) The most promising option is to develop a generalized method that reflects structural changes in the ensemble. A highly efficient ensemble multiscale filter (EnMSF) is then proposed to solve large scale nonlinear estimation problems with arbitrary uncertainties. At each prediction step realizations of the state variables are propagated. At update times, joint Gaussian distribution of states and measurements are assumed and the Predictive Efficiency method is used to identify a multiscale tree to approximate statistics of the propagated ensemble. Then a two-sweep update is performed to estimate the state variables using all the data. By controlling the tree parameters, the EnMSF can reduce sampling error while keep long range correlation in the ensemble. Applications of the EnMSF to Navier-Stokes equation and a nonlinear diffusion problem are demonstrated. Finally, the EnMSF is successfully applied to soil moisture and surface fluxes estimation over the Great Plains using synthetic multiresolution L-band passive and active microwave soil moisture measurements following HYDROS specifications.by Yuhua Zhou.Ph.D

Hillslope-scale soil moisture estimation with a physically-based ecohydrology model and L-band microwave remote sensing observations from space

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Civil and Environmental Engineering, 2009.Includes bibliographical references (p. 469-488).Soil moisture is a critical hydrosphere state variable that links the global water, energy, and carbon cycles. Knowledge of soil moisture at scales of individual hillslopes (10's to 100's of meters) is critical to advancing applications such as landslide prediction, rainfall-runoff modeling, and wildland fire fuel load assessment. This thesis develops a data assimilation framework that employs the ensemble Kalman Filter (EnKF) to estimate the spatial distribution of soil moisture at hillslope scales by combining uncertain model estimates with noisy active and passive L-band microwave observations. Uncertainty in the modeled soil moisture state is estimated through Monte Carlo simulations with an existing spatially distributed ecohydrology model. Application of the EnKF to estimate hillslope-scale soil moisture in a watershed critically depends on: (1) identification of factors contributing to uncertainty in soil moisture, (2) adequate representation of the sources of uncertainty in soil moisture, and (3) formulation of an observing system to estimate the geophysically observable quantities based on the modeled soil moisture. Uncertainty in the modeled soil moisture distribution arises principally from uncertainty in the hydrometeorological forcings and imperfect knowledge of the soil parameters required as input to the model. Three stochastic models are used in combination to simulate uncertain hourly hydrometeorological forcings for the model. Soil parameter sets are generated using a stochastic approach that samples low probability but potentially high consequence parameter values and preserves correlation among the parameters. The observing system recognizes the role of the model in organizing the factors effecting emission and reflection of L-band microwave energy and emphasizes the role of topography in determining the satellite viewing geometry at hillslope scales.(cont.) Experiments in which true soil moisture conditions were simulated by the model and used to produce synthetic observations at spatial scales significantly coarser than the model resolution reveal that sequential assimilation of observations improves the hillslope-scale near-surface moisture estimate. Results suggest that the data assimilation framework is an effective means of disaggregating coarse-scale observations according to the model physics represented by the ecohydrology model. The thesis concludes with a discussion of contributions, implications, and future directions of this work.by Alejandro Nicolas Flores.Ph.D

Estimation de l'humidité du sol à haute résolution spatio-temporelle : une nouvelle approche basée sur la synergie des observations micro-ondes actives/passives et optiques/thermiques

Les capteurs micro-ondes passifs SMOS et SMAP fournissent des données d'humidité du sol (SM) à une résolution d'environ 40 km avec un intervalle de 2 à 3 jours à l' échelle mondiale et une profondeur de détection de 0 à 5 cm. Ces données sont très pertinentes pour les applications cli- matiques et météorologiques. Cependant, pour les applications à échelle régionales (l'hydrologie) ou locales (l'agriculture), des données de SM à une haute résolution spatiale (typiquement 100 m ou plus fine) seraient nécessaires. Les données collectées par les capteurs optiques/thermiques et les radars peuvent fournir des indicateurs de SM à haute résolution spatiale, mais ces deux approches alternatives ont des limites. En particulier, les données optiques/thermiques ne sont pas disponibles sous les nuages et sous les couverts végétaux. Quant aux données radar, elles sont sensibles à la rugosité du sol et à la structure de la végétation, qui sont tous deux difficiles à caractériser depuis l'espace. De plus, la résolution temporelle de ces données est d'environ 6 jours. Dans ce contexte, la ligne directrice de la thèse est de proposer une nouvelle approche qui combine pour la première fois des capteurs passifs micro-ondes, optiques/thermiques et actifs micro-ondes (radar) pour estimer SM sur de grandes étendues à une résolution de 100 m chaque jour. Notre hypothèse est d'abord de nous appuyer sur une méthode de désagrégation existante (DISPATCH) des données SMOS/SMAP pour atteindre la résolution cible obtenue par les radars. A l'origine, DISPATCH est basé sur l'efficacité d' évaporation du sol (SEE) estimée sur des pixels partiellement végétalisés à partir de données optiques/thermiques (généralement MODIS) de température de surface et de couverture végétale à résolution de 1 km. Les données désagrégées de SM sont ensuite combinées avec une méthode d'inversion de SM basée sur les données radar afin d'exploiter les capacités de détection des radars Sentinel-1. Enfin, les capacités de l'assimilation des donnés satellitaires de SM dans un modèle de bilan hydrique du sol sont évaluées en termes de prédiction de SM à une résolution de 100 m et à une échelle temporelle quotidienne.Dans une première étape, l'algorithme DISPATCH est amélioré par rapport à sa version actuelle, principalement 1) en étendant son applicabilité aux pixels optiques entièrement végétalisés en utilisant l'indice de sécheresse de la végétation basé sur la température et un produit de couverture végétale amélioré, et 2) en augmentant la résolution de désagrégation de 1 km à 100 m en utilisant les données optiques/thermiques de Landsat (en plus de MODIS). Le produit de SM désagrégé à la résolution de 100 m est validé avec des mesures in situ collectées sur des zones irriguées au Maroc, indiquant une corrélation spatiale quotidienne variant de 0,5 à 0,9. Dans un deuxième étape, un nouvel algorithme est construit en développant une synergie entre les données DISPATCH et radar à 100 m de résolution. En pratique, le produit SM issu de DISPATCH les jours de ciel clair est d'abord utilisé pour calibrer un modèle de transfert radiatif radar en mode direct. Ensuite, le modèle de transfert radiatif radar ainsi calibré est utilisé en mode inverse pour estimer SM à la résolution spatio-temporelle de Sentinel-1. Sur les sites de validation, les résultats indiquent une corrélation entre les mesures satellitaires et in situ, de l'ordre de 0,66 à 0,81 pour un indice de végétation inférieur à 0,6. Dans une troisième et dernière étape, une méthode d'assimilation optimale est utilisée pour interpoler dans le temps les données de SM à la résolution de 100 m. La dynamique du produit SM dérivé de l'assimilation de SM DISPATCH à 100 m de résolution est cohérente avec les événements d'irrigation. Cette approche peut être facilement appliquée sur de grandes zones, en considérant que toutes les données (télédétection et météorologique) requises en entrée sont disponibles à l' échelle globale.SMOS and SMAP passive microwave sensors provide soil moisture (SM) data at 40 km resolution every 2-3 days globally, with a 0-5 cm sensing depth relevant for climatic and meteorological applications. However, SM data would be required at a higher (typically 100 m or finer) spatial resolution for many other regional (hydrology) or local (agriculture) applications. Optical/thermal and radar sensors can be used for retrieving SM proxies at such high spatial resolution, but both techniques have limitations. In particular, optical/thermal data are not available under clouds and under plant canopies. Moreover, radar data are sensitive to soil roughness and vegetation structure, which are challenging to characterize from outer space, and have a repeat cycle of at least six days, limiting the observations' temporal frequency. In this context, the leading principle of the thesis is to propose a new approach that combines passive microwave, optical/thermal, and active microwave (radar) sensors for the first time to retrieve SM data at 100 m resolution on a daily temporal scale. Our assumption is first to rely on an existing disaggregation method (DISPATCH) of SMOS/SMAP SM data to meet the target resolution achieved by radars. DISPATCH is originally based on the soil evaporative efficiency (SEE) retrieved over partially vegetated pixels from 1 km resolution optical/thermal (typically MODIS) surface temperature and vegetation cover data. The disaggregated SM data is then combined with a radar-based SM retrieval method to exploit the sensing capabilities of the Sentinel-1 radars. Finally, the efficacy of the assimilation of satellite-based SM data in a soil water balance model is assessed in terms of SM predictions at the 100 m resolution and daily temporal scale. As a first step, the DISPATCH algorithm is improved from its current version by mainly 1) extending its applicability to fully vegetated optical pixels using the temperature vegetation dryness index and an enhanced vegetation cover product, and 2) increasing the targeted downscaling resolution from 1 km to 100 m using Landsat (in addition to MODIS) optical/thermal data. The 100 m resolution disaggregated SM product is validated with in situ measurements collected over irrigated areas in Morocco, showing a daily spatial correlation in the range of 0.5-0.9. As a second step, a new algorithm is built on a synergy between DISPATCH and radar 100 m resolution data. In practice, the DISPATCH SM product available on clear sky days is first used to calibrate a radar radiative transfer model in the direct mode. Then the calibrated radar radia- tive transfer model is used in the inverse mode to estimate SM at the spatio-temporal resolution of Sentinel-1. Results indicate a positive correlation between satellite and in situ measurements in the range of 0.66 to 0.81 for a vegetation index lower than 0.6. As a third and final step, an optimal assimilation method is used to interpolate 100 m resolution SM data in time. The assimilation exercise is undertaken over irrigated crop fields in Spain. The analyzed SM product derived from the assimilation of 100 m resolution DISPATCH SM is consistent with irrigation events. This approach can be readily applied over large areas, given that all the required input (remote sensing and meteorological) data are available globally

Data-driven Regularization and Uncertainty Estimation to Improve Sea Ice Data Assimilation

Accurate estimates of sea ice conditions such as ice thickness and ice concentration in the ice-covered regions are critical for shipping activities, ice operations and weather forecasting. The need for this information has increased due to the recent record of decline in Arctic ice extent and thinning of the ice cover, which has resulted in more shipping activities and climate studies. Despite the extensive studies and progress to improve the quality of sea ice forecasts from prognostic models, there is still significant room for improvement. For example, ice-ocean models have difficulty estimating the ice thickness distribution accurately. To help improve model forecasts, data assimilation is used to combine observational data with model forecasts and produce more accurate estimates. The assimilation of ice thickness observations, compared to other ice parameters such as ice concentration, is still relatively unexplored since the satellite-based ice thickness observations have only recently become common. Also, preserving sharp features of ice cover, such as leads and ridges, can be difficult, due to the spatial correlations in the background error covariance matrices. At the same time, the current ice concentration assimilation systems do not directly assimilate high resolution sea ice information from synthetic aperture radar (SAR), even though they are the main source of information for operational production of ice chart products at the Canadian Ice Service. The key challenge in SAR data assimilation is automating the interpretation of SAR images. To address the problem of assimilating ice thickness observations while preserving sharp features, two different objective functions are studied. One with a conventional l2-norm and one imposing an additional l1-norm on the derivative of the ice thickness state estimate as a sparse regularization. The latter is motivated by analysis of high resolution ice thickness observations derived from an airborne electromagnetic sensor demonstrating the sparsity of the ice thickness in the derivative domain. The data fusion and data assimilation experiments are performed over a wide range of background and observation error correlation length scales. Results demonstrate the superiority of using a combined l1-l2 regularization framework especially when the background error correlation length scale was relatively short (approximately five times the analysis grid spacing). The problem of automated information retrieval from SAR images has been explored in a problem of ice/water classification. The selected classification approach takes advantage of neural networks to produce results comparable to a previous study using logistic regression. The employed dataset in both studies is a comprehensive dataset consisting of 15405 SAR images over a seven year period, covering all months and different locations. In addition, recent neural network uncertainty estimation approaches are employed to estimate the uncertainty associated with the classification of ice/water labels, which was not explored in this problem domain previously. These predicted uncertainties can improve the automated classification process by identifying regions in the predictions that should be checked manually by an analyst