A review of spatial enhancement of hyperspectral remote sensing imaging techniques

Remote sensing technology has undeniable importance in various industrial applications, such as mineral exploration, plant detection, defect detection in aerospace and shipbuilding, and optical gas imaging, to name a few. Remote sensing technology has been continuously evolving, offering a range of image modalities that can facilitate the aforementioned applications. One such modality is Hyperspectral Imaging (HSI). Unlike Multispectral Images (MSI) and natural images, HSI consist of hundreds of bands. Despite their high spectral resolution, HSI suffer from low spatial resolution in comparison to their MSI counterpart, which hinders the utilization of their full potential. Therefore, spatial enhancement, or Super Resolution (SR), of HSI is a classical problem that has been gaining rapid attention over the past two decades. The literature is rich with various SR algorithms that enhance the spatial resolution of HSI while preserving their spectral fidelity. This paper reviews and discusses the most important algorithms relevant to this area of research between 2002-2022, along with the most frequently used datasets, HSI sensors, and quality metrics. Meta-analysis are drawn based on the aforementioned information, which is used as a foundation that summarizes the state of the field in a way that bridges the past and the present, identifies the current gap in it, and recommends possible future directions

Gazing at the Solar System: Capturing the Evolution of Dunes, Faults, Volcanoes, and Ice from Space

Gazing imaging holds promise for improved understanding of surface characteristics and processes of Earth and solar system bodies. Evolution of earthquake fault zones, migration of sand dunes, and retreat of ice masses can be understood by observing changing features over time. To gaze or stare means to look steadily, intently, and with fixed attention, offering the ability to probe the characteristics of a target deeply, allowing retrieval of 3D structure and changes on fine and coarse scales. Observing surface reflectance and 3D structure from multiple perspectives allows for a more complete view of a surface than conventional remote imaging. A gaze from low Earth orbit (LEO) could last several minutes allowing for video capture of dynamic processes. Repeat passes enable monitoring time scales of days to years. Numerous vantage points are available during a gaze (Figure 1). Features in the scene are projected into each image frame enabling the recovery of dense 3D structure. The recovery is robust to errors in the spacecraft position and attitude knowledge, because features are from different perspectives. The combination of a varying look angle and the solar illumination allows recovering texture and reflectance properties and permits the separation of atmospheric effects. Applications are numerous and diverse, including, for example, glacier and ice sheet flux, sand dune migration, geohazards from earthquakes, volcanoes, landslides, rivers and floods, animal migrations, ecosystem changes, geysers on Enceladus, or ice structure on Europa. The Keck Institute for Space Studies (KISS) hosted a workshop in June of 2014 to explore opportunities and challenges of gazing imaging. The goals of the workshop were to develop and discuss the broad scientific questions that can be addressed using spaceborne gazing, specific types of targets and applications, the resolution and spectral bands needed to achieve the science objectives, and possible instrument configurations for future missions. The workshop participants found that gazing imaging offers the ability to measure morphology, composition, and reflectance simultaneously and to measure their variability over time. Gazing imaging can be applied to better understand the consequences of climate change and natural hazards processes, through the study of continuous and episodic processes in both domains

Deep neural network for complex open-water wetland mapping using high-resolution WorldView-3 and airborne LiDAR data

Wetland inventory maps are essential information for the conservation and management of natural wetland areas. The classification framework is crucial for successful mapping of complex wetlands, including the model selection, input variables and training procedures. In this context, deep neural network (DNN) is a powerful technique for remote sensing image classification, but this model application for wetland mapping has not been discussed in the previous literature, especially using commercial WorldView-3 data. This study developed a new framework for wetland mapping using DNN algorithm and WorldView-3 image in the Millrace Flats Wildlife Management Area, Iowa, USA. The study area has several wetlands with a variety of shapes and sizes, and the minimum mapping unit was defined as 20 m2 (0.002 ha). A set of potential variables was derived from WorldView-3 and auxiliary LiDAR data, and a feature selection procedure using principal components analysis (PCA) was used to identify the most important variables for wetland classification. Furthermore, traditional machine learning methods (support vector machine, random forest and k-nearest neighbor) were also implemented for the comparison of results. In general, the results show that DNN achieved satisfactory results in the study area (overall accuracy = 93.33 %), and we observed a high spatial overlap between reference and classified wetland polygons (Jaccard index ∼0.8). Our results confirm that PCA-based feature selection was effective in the optimization of DNN performance, and vegetation and textural indices were the most informative variables. In addition, the comparison of results indicated that DNN classification achieved relatively similar accuracies to other methods. The total classification errors vary from 0.104 to 0.111 among the methods, and the overlapped areas between reference and classified polygons range between 87.93 and 93.33 %. Finally, the findings of this study have three main implications. First, the integration of DNN model and WorldView-3 image is useful for wetland mapping at 1.2-m, but DNN results did not outperform other methods in this study area. Second, the feature selection was important for model performance, and the combination of most relevant input parameters contributes to the success of all tested models. Third, the spatial resolution of WorldView-3 is appropriate to preserve the shape and extent of small wetlands, while the application of medium resolution image (30-m) has a negative impact on the accurate delineation of these areas. Since commercial satellite data are becoming more affordable for remote sensing users, this study provides a framework that can be utilized to integrate very high-resolution imagery and deep learning in the classification of complex wetland areas

Remote Sensing of the Aquatic Environments

The book highlights recent research efforts in the monitoring of aquatic districts with remote sensing observations and proximal sensing technology integrated with laboratory measurements. Optical satellite imagery gathered at spatial resolutions down to few meters has been used for quantitative estimations of harmful algal bloom extent and Chl-a mapping, as well as winds and currents from SAR acquisitions. The knowledge and understanding gained from this book can be used for the sustainable management of bodies of water across our planet

THE UNCERTAINTY OF SPACEBORNE OBSERVATION OF VEGETATION STRUCTURE IN THE TAIGA-TUNDRA ECOTONE: A CASE STUDY IN NORTHERN SIBERIA

The ability to characterize vegetation structure in the taiga-tundra ecotone (TTE) at fine spatial scales is critical given its heterogeneity and the central role of its patterns on ecological processes in the high northern latitudes and global change scenarios. This research focuses on quantifying the uncertainty of TTE forest structure observations from remote sensing at fine spatial scales. I first quantify the uncertainty of forest biomass estimates from current airborne and spaceborne active remote sensing systems and a planned spaceborne LiDAR (ICESat-2) across sparse forest gradients. At plot-scales, current spaceborne models of biomass either explain less than a third of model variation or have biomass estimate uncertainties ranging from 50-100%. Simulations of returns from the planned ICESat-2 for a similar gradient show the uncertainty of near-term estimates vary according to the ground length along which returns are collected. The 50m length optimized the resolution of forest structure, for which there is a trade-off between horizontal precision of the measurement and vertical structure detail. At this scale biomass error ranges from 20-50%, which precludes identifying actual differences in aboveground live biomass density at 10 Mg•ha-1 intervals. These broad plot-scale uncertainties in structure from current and planned sensors provided the basis for examining a data integration technique with multiple sensors to measure the structure of sparse TTE forests. Spaceborne estimates of canopy height used complementary surface elevation measurements from passive optical and LiDAR to provide a means for directly measuring TTE forest height from spaceborne sensors. This spaceborne approach to estimating forest height was deployed to assess the spaceborne potential for examining the patterns of TTE forest structure explained with a conceptual biogeographic model linking TTE patterns and its dynamics. A patch-based analysis was used to scale estimates of TTE forest structure from multiple sensors and provided a means to simultaneously examine the horizontal and vertical structure of groups of TTE trees. The uncertainty of forest patch height estimates provides focus for improving spaceborne depictions of TTE structure patterns associated with recent change that may explain the variability of this change and the vulnerability of TTE forest structure

A Deep Learning Framework in Selected Remote Sensing Applications

The main research topic is designing and implementing a deep learning framework applied to remote sensing. Remote sensing techniques and applications play a crucial role in observing the Earth evolution, especially nowadays, where the effects of climate change on our life is more and more evident. A considerable amount of data are daily acquired all over the Earth. Effective exploitation of this information requires the robustness, velocity and accuracy of deep learning. This emerging need inspired the choice of this topic. The conducted studies mainly focus on two European Space Agency (ESA) missions: Sentinel 1 and Sentinel 2. Images provided by the ESA Sentinel-2 mission are rapidly becoming the main source of information for the entire remote sensing community, thanks to their unprecedented combination of spatial, spectral and temporal resolution, as well as their open access policy. The increasing interest gained by these satellites in the research laboratory and applicative scenarios pushed us to utilize them in the considered framework. The combined use of Sentinel 1 and Sentinel 2 is crucial and very prominent in different contexts and different kinds of monitoring when the growing (or changing) dynamics are very rapid. Starting from this general framework, two specific research activities were identified and investigated, leading to the results presented in this dissertation. Both these studies can be placed in the context of data fusion. The first activity deals with a super-resolution framework to improve Sentinel 2 bands supplied at 20 meters up to 10 meters. Increasing the spatial resolution of these bands is of great interest in many remote sensing applications, particularly in monitoring vegetation, rivers, forests, and so on. The second topic of the deep learning framework has been applied to the multispectral Normalized Difference Vegetation Index (NDVI) extraction, and the semantic segmentation obtained fusing Sentinel 1 and S2 data. The S1 SAR data is of great importance for the quantity of information extracted in the context of monitoring wetlands, rivers and forests, and many other contexts. In both cases, the problem was addressed with deep learning techniques, and in both cases, very lean architectures were used, demonstrating that even without the availability of computing power, it is possible to obtain high-level results. The core of this framework is a Convolutional Neural Network (CNN). {CNNs have been successfully applied to many image processing problems, like super-resolution, pansharpening, classification, and others, because of several advantages such as (i) the capability to approximate complex non-linear functions, (ii) the ease of training that allows to avoid time-consuming handcraft filter design, (iii) the parallel computational architecture. Even if a large amount of "labelled" data is required for training, the CNN performances pushed me to this architectural choice.} In our S1 and S2 integration task, we have faced and overcome the problem of manually labelled data with an approach based on integrating these two different sensors. Therefore, apart from the investigation in Sentinel-1 and Sentinel-2 integration, the main contribution in both cases of these works is, in particular, the possibility of designing a CNN-based solution that can be distinguished by its lightness from a computational point of view and consequent substantial saving of time compared to more complex deep learning state-of-the-art solutions

Ground Water Surface Water Interactions in Texas

Groundwater-surface water interactions need to be evaluated to optimize water management in Texas. This study provides an overview of the impacts of groundwater-surface water (gw-sw) interactions on water quality and water quantity using available data. A literature review was conducted to assess the status of knowledge of gw-sw interactions. A total of 300 references were compiled. References were subdivided into those related to water quantity issues, water quality including point and nonpoint sources of contamination, and methods such as seepage and temperature approaches and modeling analyses. Additional topics covered by the reference list include impacts of climate variability and land use/land cover changes on gw-sw interactions and ecological issues. A compilation of data and information sources for assessing gw-sw interactions in Texas is also provided. This section includes web links to online report catalogs and databases and a listing of offline reports and catalogs provided by state and federal agencies.Bureau of Economic Geolog

Single image super resolution for spatial enhancement of hyperspectral remote sensing imagery

Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations.Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations

Long-Term Monitoring of the Flooding Regime and Hydroperiod of Doñana Marshes with Landsat Time Series (1974–2014)

This paper presents a semi-automatic procedure to discriminate seasonally flooded areas in the shallow temporary marshes of Doñana National Park (SW Spain) by using a radiommetrically normalized long time series of Landsat MSS, TM, and ETM+ images (1974–2014). Extensive field campaigns for ground truth data retrieval were carried out simultaneous to Landsat overpasses. Ground truth was used as training and testing areas to check the performance of the method. Simple thresholds on TM and ETM band 5 (1.55–1.75 μm) worked significantly better than other empirical modeling techniques and supervised classification methods to delineate flooded areas at Doñana marshes. A classification tree was applied to band 5 reflectance values to classify flooded versus non-flooded pixels for every scene. Inter-scene cross-validation identified the most accurate threshold on band 5 reflectance (ρ < 0.186) to classify flooded areas (Kappa = 0.65). A joint TM-MSS acquisition was used to find the MSS band 4 (0.8 a 1.1 μm) threshold. The TM flooded area was identical to the results from MSS 4 band threshold ρ < 0.10 despite spectral and spatial resolution differences. Band slicing was retrospectively applied to the complete time series of MSS and TM images. About 391 flood masks were used to reconstruct historical spatial and temporal patterns of Doñana marshes flooding, including hydroperiod. Hydroperiod historical trends were used as a baseline to understand Doñana’s flooding regime, test hydrodynamic models, and give an assessment of relevant management and restoration decisions. The historical trends in the hydroperiod of Doñana marshes show two opposite spatial patterns. While the north-western part of the marsh is increasing its hydroperiod, the southwestern part shows a steady decline. Anomalies in each flooding cycle allowed us to assess recent management decisions and monitor their hydrological effects.: This study was funded by the Spanish Ministry of Science and Innovation through the research projects HYDRA (#CGL2006-02247/BOS) and HYDRA2 (CGL2009-09801/BOS), and by funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 641762 to ECOPOTENTIAL project. The Espacio Natural de Doñana provided permits for fieldwork in protected areas with restricted access and historical data from water column readings. We are grateful to many MSc students who helped in image processing and field sampling. We acknowledge support by the CSIC Open Access Publication Initiative through its Unit of Information Resources for Research (URICI).Peer reviewe