Bidirectional recurrent imputation and abundance estimation of LULC classes with MODIS multispectral time series and geo-topographic and climatic data

Remotely sensed data are dominated by mixed Land Use and Land Cover (LULC) types. Spectral unmixing (SU) is a key technique that disentangles mixed pixels into constituent LULC types and their abundance fractions. While existing studies on Deep Learning (DL) for SU typically focus on single time-step hyperspectral (HS) or multispectral (MS) data, our work pioneers SU using MODIS MS time series, addressing missing data with end-to-end DL models. Our approach enhances a Long-Short Term Memory (LSTM)-based model by incorporating geographic, topographic (geo-topographic), and climatic ancillary information. Notably, our method eliminates the need for explicit endmember extraction, instead learning the input-output relationship between mixed spectra and LULC abundances through supervised learning. Experimental results demonstrate that integrating spectral-temporal input data with geo-topographic and climatic information significantly improves the estimation of LULC abundances in mixed pixels. To facilitate this study, we curated a novel labeled dataset for Andalusia (Spain) with monthly MODIS multispectral time series at 460m resolution for 2013. Named Andalusia MultiSpectral MultiTemporal Unmixing (Andalusia-MSMTU), this dataset provides pixel-level annotations of LULC abundances along with ancillary information. The dataset (https://zenodo.org/records/7752348) and code (https://github.com/jrodriguezortega/MSMTU) are available to the public

Manifold learning based spectral unmixing of hyperspectral remote sensing data

Nonlinear mixing effects inherent in hyperspectral data are not properly represented in linear spectral unmixing models. Although direct nonlinear unmixing models provide capability to capture nonlinear phenomena, they are difficult to formulate and the results are not always generalizable. Manifold learning based spectral unmixing accommodates nonlinearity in the data in the feature extraction stage followed by linear mixing, thereby incorporating some characteristics of nonlinearity while retaining advantages of linear unmixing approaches. Since endmember selection is critical to successful spectral unmixing, it is important to select proper endmembers from the manifold space. However, excessive computational burden hinders development of manifolds for large-scale remote sensing datasets. This dissertation addresses issues related to high computational overhead requirements of manifold learning for developing representative manifolds for the spectral unmixing task. Manifold approximations using landmarks are popular for mitigating the computational complexity of manifold learning. A new computationally effective landmark selection method that exploits spatial redundancy in the imagery is proposed. A robust, less costly landmark set with low spectral and spatial redundancy is successfully incorporated with a hybrid manifold which shares properties of both global and local manifolds. While landmark methods reduce computational demand, the resulting manifolds may not represent subtle features of the manifold adequately. Active learning heuristics are introduced to increase the number of landmarks, with the goal of developing more representative manifolds for spectral unmixing. By communicating between the landmark set and the query criteria relative to spectral unmixing, more representative and stable manifolds with less spectrally and spatially redundant landmarks are developed. A new ranking method based on the pixels with locally high spectral variability within image subsets and convex-geometry finds a solution more quickly and precisely. Experiments were conducted to evaluate the proposed methods using the AVIRIS Cuprite hyperspectral reference dataset. A case study of manifold learning based spectral unmixing in agricultural areas is included in the dissertation.Remotely sensed data collected by airborne or spaceborne sensors are utilized to quantify crop residue cover over an extensive area. Although remote sensing indices are popular for characterizing residue amounts, they are not effective with noisy Hyperion data because the effect of residual striping artifacts is amplified in ratios involving band differences. In this case study, spectral unmixing techniques are investigated for estimating crop residue as an alternative approach to empirical models developed using band based indices. The spectral unmixing techniques, and especially the manifold learning approaches, provide more robust, lower RMSE estimates for crop residue cover than the hyperspectral index based method for Hyperion data

Soil erosion in the Alps : causes and risk assessment

The issue of soil erosion in the Alps has long been neglected due to the low economic value of the agricultural land. However, soil stability is a key parameter which affects ecosystem services like slope stability, water budgets (drinking water reservoirs as well as flood prevention), vegetation productivity, ecosystem biodiversity and nutrient production. In alpine regions, spatial estimates on soil erosion are difficult to derive because the highly heterogeneous biogeophysical structure impedes measurement of soil erosion and the applicability of soil erosion models. However, remote sensing and geographic information system (GIS) methods allow for spatial estimation of soil erosion by direct detection of erosion features and supply of input data for soil erosion models. Thus, the main objective of this work is to address the problem of soil erosion risk assessment in the Alps on catchment scale with remote sensing and GIS tools. Regarding soil erosion processes the focus is on soil erosion by water (here sheet erosion) and gravity (here landslides). For these two processes we address i) the monitoring and mapping of the erosion features and related causal factors ii) soil erosion risk assessment with special emphasis on iii) the validation of existing models for alpine areas. All investigations were accomplished in the Urseren Valley (Central Swiss Alps) where the valley slopes are dramatically affected by sheet erosion and landslides. For landslides, a natural susceptibility of the catchment has been indicated by bivariate and multivariate statistical analysis. Geology, slope and stream density are the most significant static landslide causal factors. Static factors are here defined as factors that do not change their attributes during the considered time span of the study (45 years), e.g. geology, stream network. The occurrence of landslides might be significantly increased by the combined effects of global climate and land use change. Thus, our hypothesis is that more recent changes in land use and climate affected the spatial and temporal occurrence of landslides. The increase of the landslide area of 92% within 45 years in the study site confirmed our hypothesis. In order to identify the cause for the trend in landslide occurrence time-series of landslide causal factors were analysed. The analysis revealed increasing trends in the frequency and intensity of extreme rainfall events and stocking of pasture animals. These developments presumably enhanced landslide hazard. Moreover, changes in land-cover and land use were shown to have affected landslide occurrence. For instance, abandoned areas and areas with recently emerging shrub vegetation show very low landslide densities. Detailed spatial analysis of the land use with GIS and interviews with farmers confirmed the strong influence of the land use management practises on slope stability. The definite identification and quantification of the impact of these non-stationary landslide causal factors (dynamic factors) on the landslide trend was not possible due to the simultaneous change of several factors. The consideration of dynamic factors in statistical landslide susceptibility assessments is still unsolved. The latter may lead to erroneous model predictions, especially in times of dramatic environmental change. Thus, we evaluated the effect of dynamic landslide causal factors on the validity of landslide susceptibility maps for spatial and temporal predictions. For this purpose, a logistic regression model based on data of the year 2000 was set up. The resulting landslide susceptibility map was valid for spatial predictions. However, the model failed to predict the landslides that occurred in a subsequent event. In order to handle this weakness of statistic landslide modelling a multitemporal approach was developed. It is based on establishing logistic regression models for two points in time (here 1959 and 2000). Both models could correctly classify >70% of the independent spatial validation dataset. By subtracting the 1959 susceptibility map from the 2000 susceptibility map a deviation susceptibility map was obtained. Our interpretation was that these susceptibility deviations indicate the effect of dynamic causal factors on the landslide probability. The deviation map explained 85% of new independent landslides occurring after 2000. Thus, we believe it to be a suitable tool to add a time element to a susceptibility map pointing to areas with changing susceptibility due to recently changing environmental conditions or human interactions. In contrast to landslides that are a direct threat to buildings and infrastructure, sheet erosion attracts less attention because it is often an unseen process. Nonetheless, sheet erosion may account for a major proportion of soil loss. Soil loss by sheet erosion is related to high spatial variability, however, in contrast to arable fields for alpine grasslands erosion damages are long lasting and visible over longer time periods. A crucial erosion triggering parameter that can be derived from satellite imagery is fractional vegetation cover (FVC). Measurements of the radiogenic isotope Cs-137, which is a common tracer for soil erosion, confirm the importance of FVC for soil erosion yield in alpine areas. Linear spectral unmixing (LSU), mixture tuned matched filtering (MTMF) and the spectral index NDVI are applied for estimating fractional abundance of vegetation and bare soil. To account for the small scale heterogeneity of the alpine landscape very high resolved multispectral QuickBird imagery is used. The performance of LSU and MTMF for estimating percent vegetation cover is good (r²=0.85, r²=0.71 respectively). A poorer performance is achieved for bare soil (r²=0.28, r²=0.39 respectively) because compared to vegetation, bare soil has a less characteristic spectral signature in the wavelength domain detected by the QuickBird sensor. Apart from monitoring erosion controlling factors, quantification of soil erosion by applying soil erosion risk models is done. The performance of the two established models Universal Soil Loss Equation (USLE) and Pan-European Soil Erosion Risk Assessment (PESERA) for their suitability to model erosion for mountain environments is tested. Cs-137 is used to verify the resulting erosion rates from USLE and PESERA. PESERA yields no correlation to measured Cs-137 long term erosion rates and shows lower sensitivity to FVC. Thus, USLE is used to model the entire study site. The LSU-derived FVC map is used to adapt the C factor of the USLE. Compared to the low erosion rates computed with the former available low resolution dataset (1:25000) the satellite supported USLE map shows “hotspots” of soil erosion of up to 16 t ha-1 a-1. In general, Cs-137 in combination with the USLE is a very suitable method to assess soil erosion for larger areas, as both give estimates on long-term soil erosion. Especially for inaccessible alpine areas, GIS and remote sensing proved to be powerful tools that can be used for repetitive measurements of erosion features and causal factors. In times of global change it is of crucial importance to account for temporal developments. However, the evaluation of the applied soil erosion risk models revealed that the implementation of temporal aspects, such as varying climate, land use and vegetation cover is still insufficient. Thus, the proposed validation strategies (spatial, temporal and via Cs-137) are essential. Further case studies in alpine regions are needed to test the methods elaborated for the Urseren Valley. However, the presented approaches are promising with respect to improve the monitoring and identification of soil erosion risk areas in alpine regions

A novel fusion framework embedded with zero-shot super-resolution and multivariate autoregression for precipitable water vapor across the continental Europe

Precipitable water vapor (PWV), as the most abundant greenhouse gas, significantly impacts the evapotranspiration process and thus the global climate. However, the applicability of mainstream satellite PWV products is limited by the tradeoff between spatial and temporal resolutions, as well as some external factors such as cloud contamination. In this study, we proposed a novel PWV spatio-temporal fusion framework based on the zero-shot super-resolution and the multivariate autoregression models (ZSSR-ARF) to improve the accuracy and continuity of PWV. The framework is implemented in a way that the satellite-derived observations (MOD05) are fused with the reanalysis data (ERA5) to generate accurate and seamless PWV of high spatio-temporal resolution (0.01°, daily) across the European continent from 2001 to 2021. Firstly, the ZSSR approach is used to enhance the spatial resolution of ERA5 PWV based on the internal recurrence of image information. Secondly, the optimal ERA5-MOD05 image pairs are selected based on the image similarity as inputs to improve the fusion accuracy. Thirdly, the framework develops a multivariate autoregressive fusion approach to allocate weights adaptively for the high-resolution image prediction, which primely addresses the non-stationarity and autocorrelation of PWV. The results reveal that the accuracies of fused PWV are consistent with those of the GPS retrievals (r = 0.82–0.95 and RMSE = 2.21–4.01 mm), showing an enhancement in the accuracy and continuity compared to the original MODIS PWV. The ZSSR-ARF fusion framework outperforms the other methods with R$^2$ improved by over 24% and RMSE reduced by over 0.61 mm. Furthermore, the fused PWV exhibits similar temporal consistency (mean difference of 0.40 mm and DSTD of 3.22 mm) to the reliable ERA5 products, and substantial increasing trends (mean of 0.057 mm/year and over 0.1 mm/year near the southern and western coasts) are observed over the European continent. As the accuracy and continuity of PWV are improved, the outcome of this paper has potential for climatic analyses during the land-atmosphere cycle process

Visible and Near Infrared imaging spectroscopy and the exploration of small scale hydrothermally altered and hydrated environments on Earth and Mars

The use of Visible and Near Infrared (VNIR) imaging spectroscopy is a cornerstone of planetary exploration. This work shall present an investigation into the limitations of scale, both spectral and spatial, in the utility of VNIR images for identifying small scale hydrothermal and potential hydrated environments on Mars, and regions of the Earth that can serve as martian analogues. Such settings represent possible habitable environments; important locations for astrobiological research. The ESA/Roscosmos ExoMars rover PanCam captures spectrally coarse but spatially high resolution VNIR images. This instrument is still in development and the first field trial of an emulator fitted with the final set of geological filters is presented here. Efficient image analysis techniques are explored and the ability to accurately characterise a hydrothermally altered region using PanCam data products is established. The CRISM orbital instrument has been returning hyperspectral VNIR images with an 18 m2 pixel resolution since 2006. The extraction of sub-pixel information from CRISM pixels using Spectral Mixture Analysis (SMA) algorithms is explored. Using synthetic datasets a full SMA pipeline consisting of publically available Matlab algorithms and optimised for investigation of mineralogically complex hydrothermal suites is developed for the first time. This is validated using data from Námafjall in Iceland, the region used to field trial the PanCam prototype. The pipeline is applied to CRISM images covering four regions on Mars identified as having potentially undergone hydrothermal alteration in their past. A second novel use of SMA to extract a unique spectral signature for the potentially hydrated Recurring Slope Lineae features on Mars is presented. The specific methodology presented shows promise and future improvements are suggested. The importance of combining different scales of data and recognising their limitations is discussed based on the results presented and ways in which to take the results presented in this thesis forward are given

Linear Unmixing of Hyperspectral Signals via Wavelet Feature Extraction

A pixel in remotely sensed hyperspectral imagery is typically a mixture of multiple electromagnetic radiances from various ground cover materials. Spectral unmixing is a quantitative analysis procedure used to recognize constituent ground cover materials (or endmembers) and obtain their mixing proportions (or abundances) from a mixed pixel. The abundances are typically estimated using the least squares estimation (LSE) method based on the linear mixture model (LMM). This dissertation provides a complete investigation on how the use of appropriate features can improve the LSE of endmember abundances using remotely sensed hyperspectral signals. The dissertation shows how features based on signal classification approaches, such as discrete wavelet transform (DWT), outperform features based on conventional signal representation methods for dimensionality reduction, such as principal component analysis (PCA), for the LSE of endmember abundances. Both experimental and theoretical analyses are reported in the dissertation. A DWT-based linear unmixing system is designed specially for the abundance estimation. The system utilizes the DWT as a pre-processing step for the feature extraction. Based on DWT-based features, the system utilizes the constrained LSE for the abundance estimation. Experimental results show that the use of DWT-based features reduces the abundance estimation deviation by 30-50% on average, as compared to the use of original hyperspectral signals or conventional PCA-based features. Based on the LMM and the LSE method, a series of theoretical analyses are derived to reveal the fundamental reasons why the use of the appropriate features, such as DWT-based features, can improve the LSE of endmember abundances. Under reasonable assumptions, the dissertation derives a generalized mathematical relationship between the abundance estimation error and the endmember separabilty. It is proven that the abundance estimation error can be reduced through increasing the endmember separability. The use of DWT-based features provides a potential to increase the endmember separability, and consequently improves the LSE of endmember abundances. The stability of the LSE of endmember abundances is also analyzed using the concept of the condition number. Analysis results show that the use of DWT-based features not only improves the LSE of endmember abundances, but also improves the LSE stability

Hydrocarbon quantification using neural networks and deep learning based hyperspectral unmixing

Hydrocarbon (HC) spills are a global issue, which can seriously impact human life and the environment, therefore early identification and remedial measures taken at an early stage are important. Thus, current research efforts aim at remotely quantifying incipient quantities of HC mixed with soils. The increased spectral and spatial resolution of hyperspectral sensors has opened ground-breaking perspectives in many industries including remote inspection of large areas and the environment. The use of subpixel detection algorithms, and in particular the use of the mixture models, has been identified as a future advance that needs to be incorporated in remote sensing. However, there are some challenging tasks since the spectral signatures of the targets of interest may not be immediately available. Moreover, real time processing and analysis is required to support fast decision-making. Progressing in this direction, this thesis pioneers and researches novel methodologies for HC quantification capable of exceeding the limitations of existing systems in terms of reduced cost and processing time with improved accuracy. Therefore the goal of this research is to develop, implement and test different methods for improving HC detection and quantification using spectral unmixing and machine learning. An efficient hybrid switch method employing neural networks and hyperspectral is proposed and investigated. This robust method switches between state of the art hyperspectral unmixing linear and nonlinear models, respectively. This procedure is well suited for the quantification of small quantities of substances within a pixel with high accuracy as the most appropriate model is employed. Central to the proposed approach is a novel method for extracting parameters to characterise the non-linearity of the data. These parameters are fed into a feedforward neural network which decides in a pixel by pixel fashion which model is more suitable. The quantification process is fully automated by applying further classification techniques to the acquired hyperspectral images. A deep learning neural network model is designed for the quantification of HC quantities mixed with soils. A three-term backpropagation algorithm with dropout is proposed to avoid overfitting and reduce the computational complexity of the model. The above methods have been evaluated using classical repository datasets from the literature and a laboratory controlled dataset. For that, an experimental procedure has been designed to produce a labelled dataset. The data was obtained by mixing and homogenizing different soil types with HC substances, respectively and measuring the reflectance with a hyperspectral sensor. Findings from the research study reveal that the two proposed models have high performance, they are suitable for the detection and quantification of HC mixed with soils, and surpass existing methods. Improvements in sensitivity, accuracy, computational time are achieved. Thus, the proposed approaches can be used to detect HC spills at an early stage in order to mitigate significant pollution from the spill areas

Exoplanet Imaging Data Challenge, phase II: Characterization of exoplanet signals in high-contrast images

Today, there exists a wide variety of algorithms dedicated to high-contrast imaging, especially for the detection and characterisation of exoplanet signals. These algorithms are tailored to address the very high contrast between the exoplanet signal(s), which can be more than two orders of magnitude fainter than the bright starlight residuals in coronagraphic images. The starlight residuals are inhomogeneously distributed and follow various timescales that depend on the observing conditions and on the target star brightness. Disentangling the exoplanet signals within the starlight residuals is therefore challenging, and new post-processing algorithms are striving to achieve more accurate astrophysical results. The Exoplanet Imaging Data Challenge is a community-wide effort to develop, compare and evaluate algorithms using a set of benchmark high-contrast imaging datasets. After a first phase ran in 2020 and focused on the detection capabilities of existing algorithms, the focus of this ongoing second phase is to compare the characterisation capabilities of state-of-the-art techniques. The characterisation of planetary companions is two-fold: the astrometry (estimated position with respect to the host star) and spectrophotometry (estimated contrast with respect to the host star, as a function of wavelength). The goal of this second phase is to offer a platform for the community to benchmark techniques in a fair, homogeneous and robust way, and to foster collaborations.Comment: Submitted to SPIE Astronomical Telescopes + Instrumentation 2022, Adaptive Optics Systems VIII, Paper 12185-