Spatiotemporal Data Augmentation of MODIS-LANDSAT Water Bodies Using Generative Adversarial Networks

The monitoring of the shape and area of a water body is an essential component for many Earth science and Hydrological applications. For this purpose, these applications require remote sensing data which provides accurate analysis of the water bodies. In this thesis the same is being attempted, first, a model is created that can map the information from one kind of satellite that captures the data from a distance of 500m to another data that is captured by a different satellite at a distance of 30m. To achieve this, we first collected the data from both of the satellites and translated the data from one satellite to another using our proposed Hydro-GAN model. This translation gives us the accurate shape, boundary, and area of the water body. We evaluated the method by using several different similarity metrics for the area and the shape of the water body. The second part of this thesis involves augmenting the data that we obtained from the Hydro-GAN model with the original data and using this enriched data to predict the area of a water body in the future. We used the case study of Great Salt lake for this purpose. The results indicated that our proposed model was creating accurate area and shape of the water bodies. When we used our proposed model to generate data at a resolution of 30m it gave us better areal and shape accuracy. If we get more data at this resolution, we can use that data to better predict coastal lines, boundaries, as well as erosion monitoring

Space Object Identification Using Feature Space Trajectory Neural Networks

The Feature Space Trajectory Neural Network (FSTNN) is a simple yet powerful pattern recognition tool developed by Neiberg and Casasent for use in an Automatic Target Recognition System. Since the FSTNN was developed, it has been used on various problems including speaker identification and space object identification. However, in these types of problems, the test set represents time series data rather than an independent set of points. Since the distance metric of the standard FSTNN treats each test point independently without regard to its position in the sequence, the FSTNN can yield less than optimal results in these problems. Two methods for incorporating sequence information into the FSTNN algorithm are presented. These methods, Dynamic Time Warping (DTW) and Uniform Time Warping (UTW), are described and compared to the standard FSTNN performance on the space object identification problem. Both reduce error induced by improper synchronization of the test and training sequences and make the FSTNN more generally applicable to a wide variety of pattern recognition problems. They incorporate sequencing information by synchronizing the test and training trajectories. DTW accomplishes this \u27on-the-fly\u27 as the sequence progresses while UTW uniformly compensates for temporal differences across the trajectories. These algorithms improve the maximum probability of false alarm (PFA) of the standard FSTNN by an average of 10.18% and 27.69%, respectively, although UTW is less consistent in its results. A metric for determining the saliency of the features in an FSTNN is also presented and demonstrated

A Weighted DTW Approach for Similarity Matching over Uncertain Time Series

To measure uncertain time series similarity effectively and efficiently, in this paper, we propose a weighted DTW distance-based approach for uncertain time series with the expected distance. We introduce a weight function to assign weights to a reference point and a testing point. With this function and the WDTW, the accuracy of calculating uncertain time series similarity can be improved. Also, to reduce the storage space and time-consuming, we extend the lower bound function LB_Keogh for DTW into ULB_Keogh for our approach

Advanced methods for earth observation data synergy for geophysical parameter retrieval

The first part of the thesis focuses on the analysis of relevant factors to estimate the response time between satellite-based and in-situ soil moisture (SM) using a Dynamic Time Warping (DTW). DTW was applied to the SMOS L4 SM, and was compared to in-situ root-zone SM in the REMEDHUS network in Western Spain. The method was customized to control the evolution of time lag during wetting and drying conditions. Climate factors in combination with crop growing seasons were studied to reveal SM-related processes. The heterogeneity of land use was analyzed using high-resolution images of NDVI from Sentinel-2 to provide information about the level of spatial representativity of SMOS data to each in-situ station. The comparison of long-term precipitation records and potential evapotranspiration allowed estimation of SM seasons describing different SM conditions depending on climate and soil properties. The second part of the thesis focuses on data-driven methods for sea ice segmentation and parameter retrieval. A Bayesian framework is employed to segment sets of multi-source satellite data. The Bayesian unsupervised learning algorithm allows to investigate the ‘hidden link’ between multiple data. The statistical properties are accounted for by a Gaussian Mixture Model, and the spatial interactions are reflected using Hidden Markov Random Fields. The algorithm segments spatial data into a number of classes, which are represented as a latent field in physical space and as clusters in feature space. In a first application, a two-step probabilistic approach based on Expectation-Maximization and the Bayesian segmentation algorithm was used to segment SAR images to discriminate surface water from sea ice types. Information on surface roughness is contained in the radar backscattering images which can be - in principle - used to detect melt ponds and to estimate high-resolution sea ice concentration (SIC). In a second study, the algorithm was applied to multi-incidence angle TB data from the SMOS L1C product to harness the its sensitivity to thin ice. The spatial patterns clearly discriminate well-determined areas of open water, old sea ice and a transition zone, which is sensitive to thin sea ice thickness (SIT) and SIC. In a third application, SMOS and the AMSR2 data are used to examine the joint effect of CIMR-like observations. The information contained in the low-frequency channels allows to reveal ranges of thin sea ice, and thicker ice can be determined from the relationship between the high-frequency channels and changing conditions as the sea ice ages. The proposed approach is suitable for merging large data sets and provides metrics for class analysis, and to make informed choices about integrating data from future missions into sea ice products. A regression neural network approach was investigated with the goal to infer SIT using TB data from the Flexible Microwave Payload 2 (FMPL-2) of the FSSCat mission. Two models - covering thin ice up to 0.6m and the full-range of SIT - were trained on Arctic data using ground truth data derived from the SMOS and Cryosat-2. This work demonstrates that moderate-cost CubeSat missions can provide valuable data for applications in Earth observation.La primera parte de la tesis se centra en el análisis de los factores relevantes para estimar el tiempo de respuesta entre la humedad del suelo (SM) basada en el satélite y la in-situ, utilizando una deformación temporal dinámica (DTW). El DTW se aplicó al SMOS L4 SM, y se comparó con la SM in-situ en la red REMEDHUS en el oeste de España. El método se adaptó para controlar la evolución del desfase temporal durante diferentes condiciones de humedad y secado. Se estudiaron los factores climáticos en combinación con los períodos de crecimiento de los cultivos para revelar los procesos relacionados con la SM. La heterogeneidad del uso del suelo se analizó utilizando imágenes de alta resolución de NDVI de Sentinel-2 para proporcionar información sobre el nivel de representatividad espacial de los datos de SMOS a cada estación in situ. La comparación de los patrones de precipitación a largo plazo y la evapotranspiración potencial permitió estimar las estaciones de SM que describen diferentes condiciones de SM en función del clima y las propiedades del suelo. La segunda parte de esta tesis se centra en métodos dirigidos por datos para la segmentación del hielo marino y la obtención de parámetros. Se emplea un método de inferencia bayesiano para segmentar conjuntos de datos satelitales de múltiples fuentes. El algoritmo de aprendizaje bayesiano no supervisado permite investigar el “vínculo oculto” entre múltiples datos. Las propiedades estadísticas se contabilizan mediante un modelo de mezcla gaussiana, y las interacciones espaciales se reflejan mediante campos aleatorios ocultos de Markov. El algoritmo segmenta los datos espaciales en una serie de clases, que se representan como un campo latente en el espacio físico y como clústeres en el espacio de las variables. En una primera aplicación, se utilizó un enfoque probabilístico de dos pasos basado en la maximización de expectativas y el algoritmo de segmentación bayesiano para segmentar imágenes SAR con el objetivo de discriminar el agua superficial de los tipos de hielo marino. La información sobre la rugosidad de la superficie está contenida en las imágenes de backscattering del radar, que puede utilizarse -en principio- para detectar estanques de deshielo y estimar la concentración de hielo marino (SIC) de alta resolución. En un segundo estudio, el algoritmo se aplicó a los datos TB de múltiples ángulos de incidencia del producto SMOS L1C para aprovechar su sensibilidad al hielo fino. Los patrones espaciales discriminan claramente áreas bien determinadas de aguas abiertas, hielo marino viejo y una zona de transición, que es sensible al espesor del hielo marino fino (SIT) y al SIC. En una tercera aplicación, se utilizan los datos de SMOS y de AMSR2 para examinar el efecto conjunto de las observaciones tipo CIMR. La información contenida en los canales de baja frecuencia permite revelar rangos de hielo marino delgado, y el hielo más grueso puede determinarse a partir de la relación entre los canales de alta frecuencia y las condiciones cambiantes a medida que el hielo marino envejece. El enfoque propuesto es adecuado para fusionar grandes conjuntos de datos y proporciona métricas para el análisis de clases, y para tomar decisiones informadas sobre la integración de datos de futuras misiones en los productos de hielo marino. Se investigó un enfoque de red neuronal de regresión con el objetivo de inferir el SIT utilizando datos de TB de la carga útil de microondas flexible 2 (FMPL-2) de la misión FSSCat. Se entrenaron dos modelos - que cubren el hielo fino hasta 0.6 m y el rango completo del SIT - con datos del Ártico utilizando datos de “ground truth” derivados del SMOS y del Cryosat-2. Este trabajo demuestra que las misiones CubeSat de coste moderado pueden proporcionar datos valiosos para aplicaciones de observación de la Tierra.Postprint (published version

On the semantics of big Earth observation data for land classification

This paper discusses the challenges of using big Earth observation data for land classification. The approach taken is to consider pure data-driven methods to be insufficient to represent continuous change. I argue for sound theories when working with big data. After revising existing classification schemes such as FAO\u27s Land Cover Classification System (LCCS), I conclude that LCCS and similar proposals cannot capture the complexity of landscape dynamics. I then investigate concepts that are being used for analyzing satellite image time series; I show these concepts to be instances of events. Therefore, for continuous monitoring of land change, event recognition needs to replace object identification as the prevailing paradigm. The paper concludes by showing how event semantics can improve data-driven methods to fulfil the potential of big data

Pixel-level Image Fusion Algorithms for Multi-camera Imaging System

This thesis work is motivated by the potential and promise of image fusion technologies in the multi sensor image fusion system and applications. With specific focus on pixel level image fusion, the process after the image registration is processed, we develop graphic user interface for multi-sensor image fusion software using Microsoft visual studio and Microsoft Foundation Class library. In this thesis, we proposed and presented some image fusion algorithms with low computational cost, based upon spatial mixture analysis. The segment weighted average image fusion combines several low spatial resolution data source from different sensors to create high resolution and large size of fused image. This research includes developing a segment-based step, based upon stepwise divide and combine process. In the second stage of the process, the linear interpolation optimization is used to sharpen the image resolution. Implementation of these image fusion algorithms are completed based on the graphic user interface we developed. Multiple sensor image fusion is easily accommodated by the algorithm, and the results are demonstrated at multiple scales. By using quantitative estimation such as mutual information, we obtain the experiment quantifiable results. We also use the image morphing technique to generate fused image sequence, to simulate the results of image fusion. While deploying our pixel level image fusion algorithm approaches, we observe several challenges from the popular image fusion methods. While high computational cost and complex processing steps of image fusion algorithms provide accurate fused results, they also makes it hard to become deployed in system and applications that require real-time feedback, high flexibility and low computation abilit

Analyzing spatio-temporal factors to estimate the response time between SMOS and in-situ soil moisture at different depths

A comprehensive understanding of temporal variability of subsurface soil moisture (SM) is paramount in hydrological and agricultural applications such as rainfed farming and irrigation. Since the SMOS (Soil Moisture and Ocean Salinity) mission was launched in 2009, globally available satellite SM retrievals have been used to investigate SM dynamics, based on the fact that useful information about subsurface SM is contained in their time series. SM along the depth profile is influenced by atmospheric forcing and local SM properties. Until now, subsurface SM was estimated by weighting preceding information of remotely sensed surface SM time series according to an optimized depth-specific characteristic time length. However, especially in regions with extreme SM conditions, the response time is supposed to be seasonally variable and depends on related processes occurring at different timescales. Aim of this study was to quantify the response time by means of the time lag between the trend series of satellite and in-situ SM observations using a Dynamic Time Warping (DTW) technique. DTW was applied to the SMOS satellite SM L4 product at 1 km resolution developed by the Barcelona Expert Center (BEC), and in-situ near-surface and root-zone SM of four representative stations at multiple depths, located in the Soil Moisture Measurements Station Network of the University of Salamanca (REMEDHUS) in Western Spain. DTW was customized to control the rate of accumulation and reduction of time lag during wetting and drying conditions and to consider the onset dates of pronounced precipitation events to increase sensitivity to prominent features of the input series. The temporal variability of climate factors in combination with crop growing seasons were used to indicate prevailing SM-related processes. Hereby, a comparison of long-term precipitation recordings and estimations of potential evapotranspiration (PET) allowed us to estimate SM seasons. The spatial heterogeneity of land use was analyzed by means of high-resolution images of Normalized Difference Vegetation Index (NDVI) from Sentinel-2 to provide information about the level of spatial representativeness of SMOS observations to each in-situ station. Results of the spatio-temporal analysis of the study were then evaluated to understand seasonally and spatially changing patterns in time lag. The time lag evolution describes a variable characteristic time length by considering the relevant processes which link SMOS and in-situ SM observation, which is an important step to accurately infer subsurface SM from satellite time series. At a further stage, the approach needs to be applied to different SM networks to understand the seasonal, climate- and site-specific characteristic behaviour of time lag and to decide, whether general conclusions can be drawn.The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660050. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 713673. This study was also funded through the award “Unidad de Excelencia María de Maeztu” MDM-2016-0600 and by the Spanish Ministry of Science and Innovation through the projects ESP2017-89463-C3-1-R, ESP2017-89463-C3-2-R and ESP2017-89463-C3-3-R.Peer ReviewedPostprint (published version