Comparison of convolutional neural networks for cloudy optical images reconstruction from single or multitemporal joint SAR and optical images

With the increasing availability of optical and synthetic aperture radar (SAR) images thanks to the Sentinel constellation, and the explosion of deep learning, new methods have emerged in recent years to tackle the reconstruction of optical images that are impacted by clouds. In this paper, we focus on the evaluation of convolutional neural networks that use jointly SAR and optical images to retrieve the missing contents in one single polluted optical image. We propose a simple framework that ease the creation of datasets for the training of deep nets targeting optical image reconstruction, and for the validation of machine learning based or deterministic approaches. These methods are quite different in terms of input images constraints, and comparing them is a problematic task not addressed in the literature. We show how space partitioning data structures help to query samples in terms of cloud coverage, relative acquisition date, pixel validity and relative proximity between SAR and optical images. We generate several datasets to compare the reconstructed images from networks that use a single pair of SAR and optical image, versus networks that use multiple pairs, and a traditional deterministic approach performing interpolation in temporal domain.Comment: 17 page

Development of a methodology to fill gaps in MODIS LST data for Antarctica

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesLand Surface Temperature (LST) is an essential parameter for analyzing many environmental questions. Lack of high spatio-temporal resolution of LST data in Antarctica limits the understanding of climatological, ecological processes. The MODIS LST product is a promising source that provides daily LST data at 1 km spatial resolution, but MODIS LST data have gaps due to cloud cover. This research developed a method to fill those gaps with user-defined options to balance processing time and accuracy of MODIS LST data. The presented method combined temporal and spatial interpolation, using the nearest MODIS Aqua/Terra scene for temporal interpolation, Generalized Additive Model (GAM) using 3-dimensional spatial trend surface, elevation, and aspect as covariates. The moving window size controls the number of filled pixels and the prediction accuracy in the temporal interpolation. A large moving window filled more pixels with less accuracy but improved the overall accuracy of the method. The developed method's performance validated and compared to Local Weighted Regression (LWR) using 14 images and Thin Plate Spline (TPS) interpolation by filling different sizes of artificial gaps 3%, 10%, and 25% of valid pixels. The developed method performed better with a low percentage of cloud cover by RMSE ranged between 0.72 to 1.70 but tended to have a higher RMSE with a high percentage of cloud cover

Remote Sensing Image Scene Classification: Benchmark and State of the Art

Remote sensing image scene classification plays an important role in a wide range of applications and hence has been receiving remarkable attention. During the past years, significant efforts have been made to develop various datasets or present a variety of approaches for scene classification from remote sensing images. However, a systematic review of the literature concerning datasets and methods for scene classification is still lacking. In addition, almost all existing datasets have a number of limitations, including the small scale of scene classes and the image numbers, the lack of image variations and diversity, and the saturation of accuracy. These limitations severely limit the development of new approaches especially deep learning-based methods. This paper first provides a comprehensive review of the recent progress. Then, we propose a large-scale dataset, termed "NWPU-RESISC45", which is a publicly available benchmark for REmote Sensing Image Scene Classification (RESISC), created by Northwestern Polytechnical University (NWPU). This dataset contains 31,500 images, covering 45 scene classes with 700 images in each class. The proposed NWPU-RESISC45 (i) is large-scale on the scene classes and the total image number, (ii) holds big variations in translation, spatial resolution, viewpoint, object pose, illumination, background, and occlusion, and (iii) has high within-class diversity and between-class similarity. The creation of this dataset will enable the community to develop and evaluate various data-driven algorithms. Finally, several representative methods are evaluated using the proposed dataset and the results are reported as a useful baseline for future research.Comment: This manuscript is the accepted version for Proceedings of the IEE

OCM 2023 - Optical Characterization of Materials : Conference Proceedings

The state of the art in the optical characterization of materials is advancing rapidly. New insights have been gained into the theoretical foundations of this research and exciting developments have been made in practice, driven by new applications and innovative sensor technologies that are constantly evolving. The great success of past conferences proves the necessity of a platform for presentation, discussion and evaluation of the latest research results in this interdisciplinary field

Soil Moisture Estimation using Sentinel-1/-2 Imagery Coupled with cycleGAN for Time-series Gap Filing

Fast soil moisture content (SMC) mapping is necessary to support water resource management and to understand crops’ growth, quality and yield. Thereby, Earth Observation (EO) plays a key role due to its ability of almost real-time monitoring of large areas at a low cost. This study aimed to explore the possibility of taking advantage of freely available Sentinel-1 (S1) and Sentinel-2 (S2) EO data for the simultaneous prediction of SMC with cycle-consistent adversarial network (cycleGAN) for time-series gap filling. The proposed methodology, first, learns latent low-dimensional representation of the satellite images, then learns a simple machine learning model on top of these representations. To evaluate the methodology, a series of vineyards, located in South Australia’s Eden valley are chosen. Specifically, we presented an efficient framework for extracting latent features from S1 and S2 imagery. We showed how one could use S1 to S2 feature translation based on Cycle-GAN using S1&S2 time series when there are missing images acquired over an area of interest. The resulting data in our study is then used to fill gaps in time series data. We used the resulting latent representations to predict SMC with various ML tools. In the experiments, cycleGAN and the autoencoders were trained with data randomly chosen around the site of interest, so we could augment the existing dataset. The best performance was demonstrated with random forest algorithm, whereas linear regression model demonstrated significant overfitting. The experiments demonstrate that the proposed methodology outperforms the compared state-of-the-art methods if there are missing optical and synthetic-aperture radar (SAR) images

Sustainable marine ecosystems: deep learning for water quality assessment and forecasting

An appropriate management of the available resources within oceans and coastal regions is vital to guarantee their sustainable development and preservation, where water quality is a key element. Leveraging on a combination of cross-disciplinary technologies including Remote Sensing (RS), Internet of Things (IoT), Big Data, cloud computing, and Artificial Intelligence (AI) is essential to attain this aim. In this paper, we review methodologies and technologies for water quality assessment that contribute to a sustainable management of marine environments. Specifically, we focus on Deep Leaning (DL) strategies for water quality estimation and forecasting. The analyzed literature is classified depending on the type of task, scenario and architecture. Moreover, several applications including coastal management and aquaculture are surveyed. Finally, we discuss open issues still to be addressed and potential research lines where transfer learning, knowledge fusion, reinforcement learning, edge computing and decision-making policies are expected to be the main involved agents.Postprint (published version

A Comprehensive survey on deep future frame video prediction

El present projecte planteja l'estudi comprensiu i extens per a la tasca de predicció de fotogrames donada una seqüència de vídeo. Mitjançant l'anàlisi de l'estat de l'art en generació d'imatges, xarxes convolucionals i adversàries l'objectiu és establir les forces i utilitats d'aquesta tasca

Sensor Signal and Information Processing II

In the current age of information explosion, newly invented technological sensors and software are now tightly integrated with our everyday lives. Many sensor processing algorithms have incorporated some forms of computational intelligence as part of their core framework in problem solving. These algorithms have the capacity to generalize and discover knowledge for themselves and learn new information whenever unseen data are captured. The primary aim of sensor processing is to develop techniques to interpret, understand, and act on information contained in the data. The interest of this book is in developing intelligent signal processing in order to pave the way for smart sensors. This involves mathematical advancement of nonlinear signal processing theory and its applications that extend far beyond traditional techniques. It bridges the boundary between theory and application, developing novel theoretically inspired methodologies targeting both longstanding and emergent signal processing applications. The topic ranges from phishing detection to integration of terrestrial laser scanning, and from fault diagnosis to bio-inspiring filtering. The book will appeal to established practitioners, along with researchers and students in the emerging field of smart sensors processing