Bayesian atmospheric correction over land: Sentinel-2/MSI and Landsat 8/OLI

Mitigating the impact of atmospheric effects on optical remote sensing data is critical for monitoring intrinsic land processes and developing Analysis Ready Data (ARD). This work develops an approach to this for the NERC NCEO medium resolution ARD Landsat 8 (L8) and Sentinel 2 (S2) products, called Sensor Invariant Atmospheric Correction (SIAC). The contribution of the work is to phrase and solve that problem within a probabilistic (Bayesian) framework for medium resolution multispectral sensors S2/MSI and L8/OLI and to provide per-pixel uncertainty estimates traceable from assumed top-of-atmosphere (TOA) measurement uncertainty, making progress towards an important aspect of CEOS ARD target requirements. A set of observational and a priori constraints are developed in SIAC to constrain an estimate of coarse resolution (500 m) aerosol optical thickness (AOT) and total column water vapour (TCWV), along with associated uncertainty. This is then used to estimate the medium resolution (10–60 m) surface reflectance and uncertainty, given an assumed uncertainty of 5 % in TOA reflectance. The coarse resolution a priori constraints used are the MODIS MCD43 BRDF/Albedo product, giving a constraint on 500 m surface reflectance, and the Copernicus Atmosphere Monitoring Service (CAMS) operational forecasts of AOT and TCWV, providing estimates of atmospheric state at core 40 km spatial resolution, with an associated 500 m resolution spatial correlation model. The mapping in spatial scale between medium resolution observations and the coarser resolution constraints is achieved using a calibrated effective point spread function for MCD43. Efficient approximations (emulators) to the outputs of the 6S atmospheric radiative transfer code are used to estimate the state parameters in the atmospheric correction stage. SIAC is demonstrated for a set of global S2 and L8 images covering AERONET and RadCalNet sites. AOT retrievals show a very high correlation to AERONET estimates (correlation coefficient around 0.86, RMSE of 0.07 for both sensors), although with a small bias in AOT. TCWV is accurately retrieved from both sensors (correlation coefficient over 0.96, RMSE <0.32 g cm−2). Comparisons with in situ surface reflectance measurements from the RadCalNet network show that SIAC provides accurate estimates of surface reflectance across the entire spectrum, with RMSE mismatches with the reference data between 0.01 and 0.02 in units of reflectance for both S2 and L8. For near-simultaneous S2 and L8 acquisitions, there is a very tight relationship (correlation coefficient over 0.95 for all common bands) between surface reflectance from both sensors, with negligible biases. Uncertainty estimates are assessed through discrepancy analysis and are found to provide viable estimates for AOT and TCWV. For surface reflectance, they give conservative estimates of uncertainty, suggesting that a lower estimate of TOA reflectance uncertainty might be appropriate

Linking Remote Sensing with APSIM through Emulation and Bayesian Optimization to Improve Yield Prediction

The enormous increase in the volume of Earth Observations (EOs) has provided the scientific community with unprecedented temporal, spatial, and spectral information. However, this increase in the volume of EOs has not yet resulted in proportional progress with our ability to forecast agricultural systems. This study examines the applicability of EOs obtained from Sentinel-2 and Landsat-8 for constraining the APSIM-Maize model parameters. We leveraged leaf area index (LAI) retrieved from Sentinel-2 and Landsat-8 NDVI (Normalized Difference Vegetation Index) to constrain a series of APSIM-Maize model parameters in three different Bayesian multi-criteria optimization frameworks across 13 different calibration sites in the U.S. Midwest. The novelty of the current study lies in its approach in providing a mathematical framework to directly integrate EOs into process-based models for improved parameter estimation and system representation. Thus, a time variant sensitivity analysis was performed to identify the most influential parameters driving the LAI (Leaf Area Index) estimates in APSIM-Maize model. Then surrogate models were developed using random samples taken from the parameter space using Latin hypercube sampling to emulate APSIM’s behavior in simulating NDVI and LAI at all sites. Site-level, global and hierarchical Bayesian optimization models were then developed using the site-level emulators to simultaneously constrain all parameters and estimate the site to site variability in crop parameters. For within sample predictions, site-level optimization showed the largest predictive uncertainty around LAI and crop yield, whereas the global optimization showed the most constraint predictions for these variables. The lowest RMSE within sample yield prediction was found for hierarchical optimization scheme (1423 Kg ha−1) while the largest RMSE was found for site-level (1494 Kg ha−1). In out-of-sample predictions for within the spatio-temporal extent of the training sites, global optimization showed lower RMSE (1627 Kg ha−1) compared to the hierarchical approach (1822 Kg ha−1) across 90 independent sites in the U.S. Midwest. On comparison between these two optimization schemes across another 242 independent sites outside the spatio-temporal extent of the training sites, global optimization also showed substantially lower RMSE (1554 Kg ha−1) as compared to the hierarchical approach (2532 Kg ha−1). Overall, EOs demonstrated their real use case for constraining process-based crop models and showed comparable results to model calibration exercises using only field measurements

Location, biophysical and agronomic parameters for croplands in northern Ghana

Smallholder agriculture is the bedrock of the food production system in sub-Saharan Africa. Yields in Africa are significantly below potentially attainable yields for a number of reasons, and they are particularly vulnerable to climate change impacts. Monitoring of these highly heterogeneous landscapes is needed to respond to farmer needs, develop an appropriate policy and ensure food security, and Earth observation (EO) must be part of these efforts, but there is a lack of ground data for developing and testing EO methods in western Africa, and in this paper, we present data on (i) crop locations, (ii) biophysical parameters and (iii) crop yield, and biomass was collected in 2020 and 2021 in Ghana and is reported in this paper. In 2020, crop type was surveyed in more than 1800 fields in three different agroecological zones across Ghana (the Guinea Savannah, Transition and Deciduous zones). In 2021, a smaller number of fields were surveyed in the Guinea Savannah zone, and additionally, repeated measurements of leaf area index (LAI) and leaf chlorophyll concentration were made on a set of 56 maize fields. Yield and biomass were also sampled at harvesting. LAI in the sampled fields ranged from 0.1 to 5.24 m2 m−2, whereas leaf chlorophyll concentration varied between 6.1 and 60.3 µg cm−2. Yield varied between 190 and 4580 kg ha−1, with an important within-field variability (average per-field standard deviation 381 kg ha−1). The data are used in this paper to (i) evaluate the Digital Earth Africa 2019 cropland masks, where 61 % of sampled 2020/21 cropland is flagged as cropland by the data set, (ii) develop and test an LAI retrieval method from Earth observation Planet surface reflectance data (validation correlation coefficient R=0.49, root mean square error (RMSE) 0.44 m2 m−2), (iii) create a maize classification data set for Ghana for 2021 (overall accuracy within the region tested: 0.84), and (iv) explore the relationship between maximum LAI and crop yield using a linear model (correlation coefficient R=0.66 and R=0.53 for in situ and Planet-derived LAI, respectively). The data set, made available here within the context of the Group on Earth Observations Global Agricultural Monitoring (GEOGLAM) initiative, is an important contribution to understanding crop evolution and distribution in smallholder farming systems and will be useful for researchers developing/validating methods to monitor these systems using Earth observation data. The data described in this paper are available from https://doi.org/10.5281/zenodo.6632083 (Gomez-Dans et al., 2022)

Quantifying Vegetation Biophysical Variables from Imaging Spectroscopy Data: A Review on Retrieval Methods

An unprecedented spectroscopic data stream will soon become available with forthcoming Earth-observing satellite missions equipped with imaging spectroradiometers. This data stream will open up a vast array of opportunities to quantify a diversity of biochemical and structural vegetation properties. The processing requirements for such large data streams require reliable retrieval techniques enabling the spatiotemporally explicit quantification of biophysical variables. With the aim of preparing for this new era of Earth observation, this review summarizes the state-of-the-art retrieval methods that have been applied in experimental imaging spectroscopy studies inferring all kinds of vegetation biophysical variables. Identified retrieval methods are categorized into: (1) parametric regression, including vegetation indices, shape indices and spectral transformations; (2) nonparametric regression, including linear and nonlinear machine learning regression algorithms; (3) physically based, including inversion of radiative transfer models (RTMs) using numerical optimization and look-up table approaches; and (4) hybrid regression methods, which combine RTM simulations with machine learning regression methods. For each of these categories, an overview of widely applied methods with application to mapping vegetation properties is given. In view of processing imaging spectroscopy data, a critical aspect involves the challenge of dealing with spectral multicollinearity. The ability to provide robust estimates, retrieval uncertainties and acceptable retrieval processing speed are other important aspects in view of operational processing. Recommendations towards new-generation spectroscopy-based processing chains for operational production of biophysical variables are given

A user friendly introduction to Data Assimilation

<p>A short guide about Data Assimilation methods. This is a live document, it gets updated whenever I can be bothered.</p

Effectively using EO data to monitor the land surface in the Sentinel era

A talk in the Wavelength 2017 conference on methods to infer the state of the land surface from heterogeneous sets of Earth Observation data

DA for land surface studies in the Sentinel era: how can we deliver?

<p>Towards an efficient approach to DA for optical remote senisng parameter inversion.</p

Efficient Data Assimilation for optical data using EO-LDAS

<p>The Earth is being continuously monitored by a large fleet of space-borne sensors, acquiring<br>information on the state of the land surface. Converting these observations into variables of<br>ecological interest is challenging, as the observations are not necessarily very sensitive to<br>everything that one wants to monitor. Further, the available sensors have very different<br>characteristics: different temporal sampling, different spatial resolutions and different spectral<br>characteristics. We envisage a system where all the information collected by this combination of<br>heterogenous sensors can be interpreted consistently using our understanding of the physical<br>processes thattake place in the land surface.In some circumstances, due to the low sensitivity<br>of the available observations, or just due to the lack of data, inferences on the state of the land<br>surface need are very uncertain, and extra prior knowledge needs to be brought in. Prior<br>knowledge in thiscontext refers to models that describe the evolution of the land surface, but<br>also typical behaviour patterns on the scene. The consistent combination of this a<br>prioriknowledge with the observations is done through Bayes' Rule. In this contribution, we<br>present a software library to do data assimilation (DA) experiments, using optical EO data to<br>infer the state of the vegetation in the landsurface. The library presents an implementation of a<br>variational data assimilationscheme that allows the use of observation operators that map from<br>the stateof the vegetation to observations of directional surface reflectance. The use of<br>aphysical model as an observation operator allows for the consistent treatment of different<br>spectral bandpass characteristics as well as illumination geometries.The state is thus the<br>parameters required to simulate the scene (e.g. LAI, chlorophyll a+b concentration, equivalent<br>water thickness, etc. for optical data). Since the temporal and spatial evolution of the state<br>parameters is not wellunderstood, we suggest the use of spatial and temporal smoothness<br>constraints. Theseallow the estimation of the state if no observation is available, but also<br>allowconsistent fusion of data at different spatial resolutions. To relax the requirementof<br>obtaining partial derivatives of the observation operator using e.g. adjoints tominimise the<br>resulting cost function, and in order to speed up calculations, GaussianProcesses are used to<br>"emulate" the observation operator, resulting in an efficientoperation. </p> <p> </p

Linking Remote Sensing with APSIM through Emulation and Bayesian Optimization to Improve Yield Prediction

The enormous increase in the volume of Earth Observations (EOs) has provided the scientific community with unprecedented temporal, spatial, and spectral information. However, this increase in the volume of EOs has not yet resulted in proportional progress with our ability to forecast agricultural systems. This study examines the applicability of EOs obtained from Sentinel-2 and Landsat-8 for constraining the APSIM-Maize model parameters. We leveraged leaf area index (LAI) retrieved from Sentinel-2 and Landsat-8 NDVI (Normalized Difference Vegetation Index) to constrain a series of APSIM-Maize model parameters in three different Bayesian multi-criteria optimization frameworks across 13 different calibration sites in the U.S. Midwest. The novelty of the current study lies in its approach in providing a mathematical framework to directly integrate EOs into process-based models for improved parameter estimation and system representation. Thus, a time variant sensitivity analysis was performed to identify the most influential parameters driving the LAI (Leaf Area Index) estimates in APSIM-Maize model. Then surrogate models were developed using random samples taken from the parameter space using Latin hypercube sampling to emulate APSIM’s behavior in simulating NDVI and LAI at all sites. Site-level, global and hierarchical Bayesian optimization models were then developed using the site-level emulators to simultaneously constrain all parameters and estimate the site to site variability in crop parameters. For within sample predictions, site-level optimization showed the largest predictive uncertainty around LAI and crop yield, whereas the global optimization showed the most constraint predictions for these variables. The lowest RMSE within sample yield prediction was found for hierarchical optimization scheme (1423 Kg ha−1) while the largest RMSE was found for site-level (1494 Kg ha−1). In out-of-sample predictions for within the spatio-temporal extent of the training sites, global optimization showed lower RMSE (1627 Kg ha−1) compared to the hierarchical approach (1822 Kg ha−1) across 90 independent sites in the U.S. Midwest. On comparison between these two optimization schemes across another 242 independent sites outside the spatio-temporal extent of the training sites, global optimization also showed substantially lower RMSE (1554 Kg ha−1) as compared to the hierarchical approach (2532 Kg ha−1). Overall, EOs demonstrated their real use case for constraining process-based crop models and showed comparable results to model calibration exercises using only field measurements