Unmixing-based Spatiotemporal Image Fusion Based on the Self-trained Random Forest Regression and Residual Compensation

Spatiotemporal satellite image fusion (STIF) has been widely applied in land surface monitoring to generate high spatial and high temporal reflectance images from satellite sensors. This paper proposed a new unmixing-based spatiotemporal fusion method that is composed of a self-trained random forest machine learning regression (R), low resolution (LR) endmember estimation (E), high resolution (HR) surface reflectance image reconstruction (R), and residual compensation (C), that is, RERC. RERC uses a self-trained random forest to train and predict the relationship between spectra and the corresponding class fractions. This process is flexible without any ancillary training dataset, and does not possess the limitations of linear spectral unmixing, which requires the number of endmembers to be no more than the number of spectral bands. The running time of the random forest regression is about ~1% of the running time of the linear mixture model. In addition, RERC adopts a spectral reflectance residual compensation approach to refine the fused image to make full use of the information from the LR image. RERC was assessed in the fusion of a prediction time MODIS with a Landsat image using two benchmark datasets, and was assessed in fusing images with different numbers of spectral bands by fusing a known time Landsat image (seven bands used) with a known time very-high-resolution PlanetScope image (four spectral bands). RERC was assessed in the fusion of MODIS-Landsat imagery in large areas at the national scale for the Republic of Ireland and France. The code is available at https://www.researchgate.net/proiile/Xiao_Li52

Shadow Detection and Restoration for Hyperspectral Images Based on Nonlinear Spectral Unmixing

Shadows are frequently observable in high-resolution images, raising challenges in image interpretation, such as classification and object detection. In this paper, we propose a novel framework for shadow detection and restoration of atmospherically corrected hyperspectral images based on nonlinear spectral unmixing. The mixture model is applied pixel-wise as a nonlinear combination of endmembers related to both pure sunlit and shadowed spectra, where the former are manually selected from scenes and the latter are derived from sunlit spectra following physical assumptions. Shadowed pixels are restored by simulating their exposure to sunlight through a combination of sunlit endmembers spectra, weighted by abundance values. The proposed framework is demonstrated on real airborne hyperspectral images. A comprehensive assessment of the restored images is carried out both visually and quantitatively. With respect to binary shadow masks, our framework can produce soft shadow detection results, keeping the natural transition of illumination conditions on shadow boundaries. Our results show that the framework can effectively detect shadows and restore information in shadowed regions

Very High Resolution (VHR) Satellite Imagery: Processing and Applications

Recently, growing interest in the use of remote sensing imagery has appeared to provide synoptic maps of water quality parameters in coastal and inner water ecosystems;, monitoring of complex land ecosystems for biodiversity conservation; precision agriculture for the management of soils, crops, and pests; urban planning; disaster monitoring, etc. However, for these maps to achieve their full potential, it is important to engage in periodic monitoring and analysis of multi-temporal changes. In this context, very high resolution (VHR) satellite-based optical, infrared, and radar imaging instruments provide reliable information to implement spatially-based conservation actions. Moreover, they enable observations of parameters of our environment at greater broader spatial and finer temporal scales than those allowed through field observation alone. In this sense, recent very high resolution satellite technologies and image processing algorithms present the opportunity to develop quantitative techniques that have the potential to improve upon traditional techniques in terms of cost, mapping fidelity, and objectivity. Typical applications include multi-temporal classification, recognition and tracking of specific patterns, multisensor data fusion, analysis of land/marine ecosystem processes and environment monitoring, etc. This book aims to collect new developments, methodologies, and applications of very high resolution satellite data for remote sensing. The works selected provide to the research community the most recent advances on all aspects of VHR satellite remote sensing

Detection of Marine Plastic Debris in the North Pacific Ocean using Optical Satellite Imagery

Plastic pollution is ubiquitous across marine environments, yet detection of anthropogenic debris in the global oceans is in its infancy. Here, we exploit high-resolution multispectral satellite imagery over the North Pacific Ocean and information from GPS-tracked floating plastic conglomerates to explore the potential for detecting marine plastic debris via spaceborne remote sensing platforms. Through an innovative method of estimating material abundance in mixed pixels, combined with an inverse spectral unmixing calculation, a spectral signature of aggregated plastic litter was derived from an 8-band WorldView-2 image. By leveraging the spectral characteristics of marine plastic debris in a real environment, plastic detectability was demonstrated and evaluated utilising a Spectral Angle Mapper (SAM) classification, Mixture Tuned Matched Filtering (MTMF), the Reed-Xiaoli Detector (RXD) algorithm, and spectral indices in a three-variable feature space. Results indicate that floating aggregations are detectable on sub-pixel scales, but as reliable ground truth information was restricted to a single confirmed target, detections were only validated by means of their respective spectral responses. Effects of atmospheric correction algorithms were evaluated using ACOLITE, ACOMP, and FLAASH, in which derived unbiased percentage differences ranged from 1% to 81% following a pairwise comparison. Building first steps towards an integrated marine monitoring system, the strengths and limitations of current remote sensing technology are identified and adopted to make suggestions for future improvements

Robust hyperspectral image reconstruction for scene simulation applications

This thesis presents the development of a spectral reconstruction method for multispectral (MSI) and hyperspectral (HSI) applications through an enhanced dictionary learning and spectral unmixing methodologies. Earth observation/surveillance is largely undertaken by MSI sensing such as that given by the Landsat, WorldView, Sentinel etc, however, the practical usefulness of the MSI data set is very limited. This is mainly because of the very limited number of wave bands that can be provided by the MSI imagery. One means to remedy this major shortcoming is to extend the MSI into HSI without the need of involving expensive hardware investment. Specifically, spectral reconstruction has been one of the most critical elements in applications such as Hyperspectral scene simulation. Hyperspectral scene simulation has been an important technique particularly for defence applications. Scene simulation creates a virtual scene such that modelling of the materials in the scene can be tailored freely to allow certain parameters of the model to be studied. In the defence sector this is the most cost-effective technique to allow the vulnerability of the soldiers/vehicles to be evaluated before they are deployed to a foreign ground. The simulation of a hyperspectral scene requires the details of materials in the scene, which is normally not available. Current state-of-the-art technology is trying to make use of the MSI satellite data, and to transform it into HSI for the hyperspectral scene simulation. One way to achieve this is through a reconstruction algorithm, commonly known as spectral reconstruction, which turns the MSI into HSI using an optimisation approach. The methodology that has been adopted in this thesis is the development of a robust dictionary learning to estimate the endmember (EM) robustly. Once the EM is found the abundance of materials in the scene can be subsequently estimated through a linear unmixing approach. Conventional approaches to the material allocation of most Hyperspectral scene simulator has been using the Texture Material Mapper (TMM) algorithm, which allocates materials from a spectral library (a collection of pre-compiled endmember iii iv materials) database according to the minimum spectral Euclidean distance difference to a candidate pixel of the scene. This approach has been shown (in this work) to be highly inaccurate with large scene reconstruction error. This research attempts to use a dictionary learning technique for material allocation, solving it as an optimisation problem with the objective of: (i) to reconstruct the scene as closely as possible to the ground truth with a fraction of error as that given by the TMM method, and (ii) to learn materials which are trace (2-3 times the number of species (i.e. intrinsic dimension) in the scene) cluster to ensure all material species in the scene is included for the scene reconstruction. Furthermore, two approaches complementing the goals of the learned dictionary through a rapid orthogonal matching pursuit (r-OMP) which enhances the performance of the orthogonal matching pursuit algorithm; and secondly a semi-blind approximation of the irradiance of all pixels in the scene including those in the shaded regions, have been proposed in this work. The main result of this research is the demonstration of the effectiveness of the proposed algorithms using real data set. The SCD-SOMP has been shown capable to learn both the background and trace materials even for a dictionary with small number of atoms (≈10). Also, the KMSCD method is found to be the more versatile with overcomplete (non-orthogonal) dictionary capable to learn trace materials with high scene reconstruction accuracy (2x of accuracy enhancement over that simulated using the TMM method. Although this work has achieved an incremental improvement in spectral reconstruction, however, the need of dictionary training using hyperspectral data set in this thesis has been identified as one limitation which is needed to be removed for the future direction of research

Detecting soil erosion in semi-arid Mediterranean environments using simulated EnMAP data

Soil is an essential nature resource. Management of this resource is vital for sustainability and the continued functioning of earths atmospheric, hydrospheric and lithospheric functioning. The assessment and continued monitoring of surface soil state provides the information required to effectively manage this resource. This research used a simulated Environmental Mapping and Analysis Program (EnMAP) hyperspectral image cube of an agricultural region in semi- arid Mediterranean Spain to classify soil erosion states. Multiple Endmember Spectral Mixture Analysis (MESMA) was used to derive within pixel fractions of eroded and accumulated soils. A Classification of the soil erosion states using the scene fraction outputs and digital terrain information. The information products generated in this research provided an optimistic outlook for the applicability of the future EnMAP sensor for soil erosion investigations in semi-arid Mediterranean environments. Additionally, this research verifies that the launch of the EnMAP satellite sensor in 2018 will provide the opportunity to further improve the monitoring of earth finite soil resources.NSERC create AMETHYST , Alberta Terrestrial Imaging Centre

Challenges and Opportunities of Multimodality and Data Fusion in Remote Sensing

International audience—Remote sensing is one of the most common ways to extract relevant information about the Earth and our environment. Remote sensing acquisitions can be done by both active (synthetic aperture radar, LiDAR) and passive (optical and thermal range, multispectral and hyperspectral) devices. According to the sensor, a variety of information about the Earth's surface can be obtained. The data acquired by these sensors can provide information about the structure (optical, synthetic aperture radar), elevation (LiDAR) and material content (multi and hyperspectral) of the objects in the image. Once considered together their comple-mentarity can be helpful for characterizing land use (urban analysis, precision agriculture), damage detection (e.g., in natural disasters such as floods, hurricanes, earthquakes, oil-spills in seas), and give insights to potential exploitation of resources (oil fields, minerals). In addition, repeated acquisitions of a scene at different times allows one to monitor natural resources and environmental variables (vegetation phenology, snow cover), anthropological effects (urban sprawl, deforestation), climate changes (desertification, coastal erosion) among others. In this paper, we sketch the current opportunities and challenges related to the exploitation of multimodal data for Earth observation. This is done by leveraging the outcomes of the Data Fusion contests, organized by the IEEE Geoscience and Remote Sensing Society since 2006. We will report on the outcomes of these contests, presenting the multimodal sets of data made available to the community each year, the targeted applications and an analysis of the submitted methods and results: How was multimodality considered and integrated in the processing chain? What were the improvements/new opportunities offered by the fusion? What were the objectives to be addressed and the reported solutions? And from this, what will be the next challenges

Caracterização e estudo comparativo de exsudações de hidrocarbonetos e plays petrolíferos em bacias terrestres das regiões central do Irã e sudeste do Brasil usando sensoriamento remoto espectral

Orientador: Carlos Roberto de Souza FilhoTese (doutorado) - Universidade Estadual de Campinas, Instituto de GeociênciasResumo: O objetivo desta pesquisa foi explorar as assinaturas de exsudações de hidrocarbonetos na superfície usando a tecnologia de detecção remota espectral. Isso foi alcançado primeiro, realizando uma revisão abrangente das capacidades e potenciais técnicas de detecção direta e indireta. Em seguida, a técnica foi aplicada para investigar dois locais de teste localizados no Irã e no Brasil, conhecidos por hospedar sistemas ativos de micro-exsudações e afloramentos betuminosos, respectivamente. A primeira área de estudo está localizada perto da cidade de Qom (Irã), e está inserida no campo petrolífero Alborz, enterrado sob sedimentos datados do Oligoceno da Formação Upper Red. O segundo local está localizado perto da cidade de Anhembi (SP), na margem oriental da bacia do Paraná, no Brasil, e inclui acumulações de betume em arenitos triássicos da Formação Pirambóia. O trabalho na área de Qom integrou evidências de (i) estudos petrográficos e geoquímicos em laboratório, (ii) investigações de afloramentos em campo, e (iii) mapeamento de anomalia em larga escala através de conjuntos de dados multi-espectrais ASTER e Sentinel-2. O resultado deste estudo se trata de novos indicadores mineralógicos e geoquímicos para a exploração de micro-exsudações e um modelo de micro-exsudações atualizado. Durante este trabalho, conseguimos desenvolver novas metodologias para análise de dados espectroscópicos. Através da utilização de dados simulados, indicamos que o instrumento de satélite WorldView-3 tem potencial para detecção direta de hidrocarbonetos. Na sequência do estudo, dados reais sobre afloramentos de arenitos e óleo na área de Anhembi foram investigados. A área foi fotografada novamente no chão e usando o sistema de imagem hiperespectral AisaFENIX. Seguiu-se estudos e amostragem no campo,incluindo espectroscopia de alcance fechado das amostras no laboratório usando instrumentos de imagem (ou seja, sisuCHEMA) e não-imagem (ou seja, FieldSpec-4). O estudo demonstrou que uma abordagem espectroscópica multi-escala poderia fornecer uma imagem completa das variações no conteúdo e composição do betume e minerais de alteração que acompanham. A assinatura de hidrocarbonetos, especialmente a centrada em 2300 nm, mostrou-se consistente e comparável entre as escalas e capaz de estimar o teor de betume de areias de petróleo em todas as escalas de imagemAbstract: The objective of this research was to explore for the signatures of seeping hydrocarbons on the surface using spectral remote sensing technology. It was achieved firstly by conducting a comprehensive review of the capacities and potentials of the technique for direct and indirect seepage detection. Next, the technique was applied to investigate two distinctive test sites located in Iran and Brazil known to retain active microseepage systems and bituminous outcrops, respectively. The first study area is located near the city of Qom in Iran, and consists of Alborz oilfield buried under Oligocene sediments of the Upper-Red Formation. The second site is located near the town of Anhembi on the eastern edge of the Paraná Basin in Brazil and includes bitumen accumulations in the Triassic sandstones of the Pirambóia Formation. Our work in Qom area integrated evidence from (i) petrographic, spectroscopic, and geochemical studies in the laboratory, (ii) outcrop investigations in the field, and (iii) broad-scale anomaly mapping via orbital remote sensing data. The outcomes of this study was novel mineralogical and geochemical indicators for microseepage characterization and a classification scheme for the microseepage-induced alterations. Our study indicated that active microseepage systems occur in large parts of the lithofacies in Qom area, implying that the extent of the petroleum reservoir is much larger than previously thought. During this work, we also developed new methodologies for spectroscopic data analysis and processing. On the other side, by using simulated data, we indicated that WorldView-3 satellite instrument has the potential for direct hydrocarbon detection. Following this demonstration, real datasets were acquired over oil-sand outcrops of the Anhembi area. The area was further imaged on the ground and from the air by using an AisaFENIX hyperspectral imaging system. This was followed by outcrop studies and sampling in the field and close-range spectroscopy in the laboratory using both imaging (i.e. sisuCHEMA) and nonimaging instruments. The study demonstrated that a multi-scale spectroscopic approach could provide a complete picture of the variations in the content and composition of bitumen and associated alteration mineralogy. The oil signature, especially the one centered at 2300 nm, was shown to be consistent and comparable among scales, and capable of estimating the bitumen content of oil-sands at all imaging scalesDoutoradoGeologia e Recursos NaturaisDoutor em Geociências2015/06663-7FAPES

Improving Hyperspectral Subpixel Target Detection Using Hybrid Detection Space

A Hyper-Spectral Image (HSI) has high spectral and low spatial resolution. As a result, most targets exist as subpixels, which pose challenges in target detection. Moreover, limitation of target and background samples always hinders the target detection performance. In this thesis, a hybrid method for subpixel target detection of an HSI using minimal prior knowledge is developed. The Matched Filter (MF) and Adaptive Cosine Estimator (ACE) are two popular algorithms in HSI target detection. They have different advantages in differentiating target from background. In the proposed method, the scores of MF and ACE algorithms are used to construct a hybrid detection space. First, some high abundance target spectra are randomly picked from the scene to perform initial detection to determine the target and background subsets. Then, the reference target spectrum and background covariance matrix are improved iteratively, using the hybrid detection space. As the iterations continue, the reference target spectrum gets closer and closer to the central line that connects the centers of target and background and resulting in noticeable improvement in target detection. Two synthetic datasets and two real datasets are used in the experiments. The results are evaluated based on the mean detection rate, Receiver Operating Characteristic (ROC) curve and observation of the detection results. Compared to traditional MF and ACE algorithms with Reed-Xiaoli Detector (RXD) background covariance matrix estimation, the new method shows much better performance on all four datasets. This method can be applied in environmental monitoring, mineral detection, as well as oceanography and forestry reconnaissance to search for extremely small target distribution in a large scene

Data-Driven Artificial Intelligence for Calibration of Hyperspectral Big Data

Near-earth hyperspectral big data present both huge opportunities and challenges for spurring developments in agriculture and high-throughput plant phenotyping and breeding. In this article, we present data-driven approaches to address the calibration challenges for utilizing near-earth hyperspectral data for agriculture. A data-driven, fully automated calibration workflow that includes a suite of robust algorithms for radiometric calibration, bidirectional reflectance distribution function (BRDF) correction and reflectance normalization, soil and shadow masking, and image quality assessments was developed. An empirical method that utilizes predetermined models between camera photon counts (digital numbers) and downwelling irradiance measurements for each spectral band was established to perform radiometric calibration. A kernel-driven semiempirical BRDF correction method based on the Ross Thick-Li Sparse (RTLS) model was used to normalize the data for both changes in solar elevation and sensor view angle differences attributed to pixel location within the field of view. Following rigorous radiometric and BRDF corrections, novel rule-based methods were developed to conduct automatic soil removal; and a newly proposed approach was used for image quality assessment; additionally, shadow masking and plot-level feature extraction were carried out. Our results show that the automated calibration, processing, storage, and analysis pipeline developed in this work can effectively handle massive amounts of hyperspectral data and address the urgent challenges related to the production of sustainable bioenergy and food crops, targeting methods to accelerate plant breeding for improving yield and biomass traits