Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Study of the speckle noise effects over the eigen decomposition of polarimetric SAR data: a review

This paper is focused on considering the effects of speckle noise on the eigen decomposition of the co- herency matrix. Based on a perturbation analysis of the matrix, it is possible to obtain an analytical expression for the mean value of the eigenvalues and the eigenvectors, as well as for the Entropy, the Anisotroopy and the dif- ferent a angles. The analytical expressions are compared against simulated polarimetric SAR data, demonstrating the correctness of the different expressions.Peer ReviewedPostprint (published version

Crop development monitoring from Synthetic Aperture Radar (SAR) imagery

Satellite remote sensing plays a vital role in providing large-scale and timely data to stakeholders of the agricultural supply chain. This allows for informed decision-making that promotes sustainable and cost-effective crop management practices. In particular, data derived from satellite-based Synthetic Aperture Radar (SAR) systems, provide opportunities for continuous crop monitoring, taking advantage of its ability to acquire images during day or night and under almost all weather conditions. Moreover, an abundance of SAR data can be anticipated in the next 5 years with the launch of several international SAR missions. However, research on crop development monitoring with data from SAR satellites has not been as widely studied as with data derived from passive multi-spectral satellites and contributions can be made to the current state-of-the-art techniques. This thesis aims at improving the current knowledge on the use of satellite-based SAR imagery for crop development monitoring. This is approached by developing novel methodologies and detailed interpretations of multitemporal SAR and Polarimetric SAR (PolSAR) responses to crop growth in three different test sites. Chapter two presents a detailed analysis of the Sentinel-1 SAR satellite response to asparagus crop development in Peru, investigating the capabilities of the sensor to capture seasonality effects as well as providing an interpretation of the temporal backscatter signature. This is complemented with a case study where a multiple-output random forest regression algorithm is used to successfully retrieve crop growth stage from Sentinel-1 data and temperature measurements. Following the limitations identified with this approach, a methodology that builds upon ideas of Bayesian Filtering Frameworks (BFFs) for crop monitoring is proposed in chapter three. It incorporates Gaussian processes to model crop dynamics as well as to model the remote sensing response to the crop state. Using this approach, it is possible to derive daily predictions with the associated uncertainties, to combine in near-real-time data from active and passive satellites as well as to estimate past and future crop key events that are of strategic importance for different stakeholders. The final section of this thesis looks at the new developments of the SAR technology considering that future open access missions will provide Quad Polarimetric SAR data. An algorithm based on multitemporal PolSAR change detection is introduced in chapter four. It defines a Change Matrix to encode an interpretable representation of the crop dynamics as captured by the evolution of the scattering mechanisms over time. We use rice fields in Spain and multiple cereal crops in Canada to test the use of the algorithm for crop monitoring. A supervised learning-based crop type classification methodology is then proposed with the same method by using the encoded scattering mechanisms as input for a neural-network-based classifier, achieving comparable performances to state-of-the-art classifiers. The results obtained in this thesis represent novel additions to the literature that contribute to our understanding and successful use of SAR imagery for agricultural monitoring. For the first time, a detailed analysis of asparagus crops is presented. It is a key crop for agricultural exports of Peru, the largest exporter of asparagus in the world. Secondly, two key contributions to the state of the art BFFs for crop monitoring are presented: a) A better exploitation of the SAR temporal dimension and an application with freely available data and b) given that it is a learning-based approach, it overcomes current limitations of transferability among crop types and regions. Finally, the PolSAR change detection approach presented in the last thesis chapter, provides a novel and easy-to-interpret tool for both crop monitoring and crop type mapping applications

Classification of Compact Polarimetric Synthetic Aperture Radar Images

The RADARSAT Constellation Mission (RCM) was launched in June 2019. RCM, in addition to dual-polarization (DP) and fully quad-polarimetric (QP) imaging modes, provides compact polarimetric (CP) mode data. A CP synthetic aperture radar (SAR) is a coherent DP system in which a single circular polarization is transmitted followed by the reception in two orthogonal linear polarizations. A CP SAR fully characterizes the backscattered field using the Stokes parameters, or equivalently, the complex coherence matrix. This is the main advantage of a CP SAR over the traditional (non-coherent) DP SAR. Therefore, designing scene segmentation and classification methods using CP complex coherence matrix data is advocated in this thesis. Scene classification of remotely captured images is an important task in monitoring the Earth's surface. The high-resolution RCM CP SAR data can be used for land cover classification as well as sea-ice mapping. Mapping sea ice formed in ocean bodies is important for ship navigation and climate change modeling. The Canadian Ice Service (CIS) has expert ice analysts who manually generate sea-ice maps of Arctic areas on a daily basis. An automated sea-ice mapping process that can provide detailed yet reliable maps of ice types and water is desirable for CIS. In addition to linear DP SAR data in ScanSAR mode (500km), RCM wide-swath CP data (350km) can also be used in operational sea-ice mapping of the vast expanses in the Arctic areas. The smaller swath coverage of QP SAR data (50km) is the reason why the use of QP SAR data is limited for sea-ice mapping. This thesis involves the design and development of CP classification methods that consist of two steps: an unsupervised segmentation of CP data to identify homogeneous regions (superpixels) and a labeling step where a ground truth label is assigned to each super-pixel. An unsupervised segmentation algorithm is developed based on the existing Iterative Region Growing using Semantics (IRGS) for CP data and is called CP-IRGS. The constituents of feature model and spatial context model energy terms in CP-IRGS are developed based on the statistical properties of CP complex coherence matrix data. The superpixels generated by CP-IRGS are then used in a graph-based labeling method that incorporates the global spatial correlation among super-pixels in CP data. The classifications of sea-ice and land cover types using test scenes indicate that (a) CP scenes provide improved sea-ice classification than the linear DP scenes, (b) CP-IRGS performs more accurate segmentation than that using only CP channel intensity images, and (c) using global spatial information (provided by a graph-based labeling approach) provides an improvement in classification accuracy values over methods that do not exploit global spatial correlation

Monitoring Snow Cover and Snowmelt Dynamics and Assessing their Influences on Inland Water Resources

Snow is one of the most vital cryospheric components owing to its wide coverage as well as its unique physical characteristics. It not only affects the balance of numerous natural systems but also influences various socio-economic activities of human beings. Notably, the importance of snowmelt water to global water resources is outstanding, as millions of populations rely on snowmelt water for daily consumption and agricultural use. Nevertheless, due to the unprecedented temperature rise resulting from the deterioration of climate change, global snow cover extent (SCE) has been shrinking significantly, which endangers the sustainability and availability of inland water resources. Therefore, in order to understand cryo-hydrosphere interactions under a warming climate, (1) monitoring SCE dynamics and snowmelt conditions, (2) tracking the dynamics of snowmelt-influenced waterbodies, and (3) assessing the causal effect of snowmelt conditions on inland water resources are indispensable. However, for each point, there exist many research questions that need to be answered. Consequently, in this thesis, five objectives are proposed accordingly. Objective 1: Reviewing the characteristics of SAR and its interactions with snow, and exploring the trends, difficulties, and opportunities of existing SAR-based SCE mapping studies; Objective 2: Proposing a novel total and wet SCE mapping strategy based on freely accessible SAR imagery with all land cover classes applicability and global transferability; Objective 3: Enhancing total SCE mapping accuracy by fusing SAR- and multi-spectral sensor-based information, and providing total SCE mapping reliability map information; Objective 4: Proposing a cloud-free and illumination-independent inland waterbody dynamics tracking strategy using freely accessible datasets and services; Objective 5: Assessing the influence of snowmelt conditions on inland water resources

Synthetic Aperture Radar (SAR) Meets Deep Learning

This reprint focuses on the application of the combination of synthetic aperture radars and depth learning technology. It aims to further promote the development of SAR image intelligent interpretation technology. A synthetic aperture radar (SAR) is an important active microwave imaging sensor, whose all-day and all-weather working capacity give it an important place in the remote sensing community. Since the United States launched the first SAR satellite, SAR has received much attention in the remote sensing community, e.g., in geological exploration, topographic mapping, disaster forecast, and traffic monitoring. It is valuable and meaningful, therefore, to study SAR-based remote sensing applications. In recent years, deep learning represented by convolution neural networks has promoted significant progress in the computer vision community, e.g., in face recognition, the driverless field and Internet of things (IoT). Deep learning can enable computational models with multiple processing layers to learn data representations with multiple-level abstractions. This can greatly improve the performance of various applications. This reprint provides a platform for researchers to handle the above significant challenges and present their innovative and cutting-edge research results when applying deep learning to SAR in various manuscript types, e.g., articles, letters, reviews and technical reports

Method for landslides detection with semi-automatic procedures: The case in the zone center-east of Cauca department, Colombia

Landslides are a common natural hazard that causes human casualties, but also infrastructure damage and land-use degradation. Therefore, a quantitative assessment of their presence is required by means of detecting and recognizing the potentially unstable areas. This research aims to develop a method supported on semiautomatic methods to detect potential mass movements at a regional scale. Five techniques were studied: Morphometry, SAR interferometry (InSAR), Persistent Scatterer InSAR (PS-InSAR), SAR polarimetry (PolSAR) and NDVI composites of Landsat 5, Landsat 7, and Landsat 8. The case study was chosen within the mid-eastern area of the Cauca state, which is characterised by its mountainous terrain and the presence of slope instabilities, officially registered in the CGS-SIMMA landslide inventory. This inventory revealed that the type `slide' occurred with 77.4% from the entire registries, `fall' with 16.5%, followed by `creeps' with 3%, flows with 2.6%, and `lateral spread' with 0.43%. As a result, we obtained the morphometric variables: slope, CONVI, TWI, landform, which were highly associated with landslides. The effect of a DEM in the processing flow of the InSAR method was similar for the InSAR coherence variable using the DEMs ASTER, PALSAR RTC, Topo-map, and SRTM. Then, a multiInSAR analysis gave displacement velocities in the LOS direction between -10 and 10 mm/year. With the dual-PolSAR analysis (Sentinel-1), VH and VV C-band polarised radar energy emitted median values of backscatters, for landslides, about of -14.5 dB for VH polarisation and -8.5 dB for VV polarisation. Also, L-band fully polarimetric NASA-UAVSAR data allowed to nd the mechanism of dispersion of CGS landslide inventory: 39% for surface scattering, 46.4% for volume dispersion, and 14.6% for double-bounce scattering. The optical remote sensing provided NDVI composites derived from Landsat series between 2012 and 2016, showing that NDVI values between 0.40 and 0.70 had a high correlation to landslides. In summary, we found the highest categories related to landslides by Weight of Evidence method (WofE) for each spaceborne technique applied. Finally, these results were merged to generate the landslide detection model by using the supervised machine learning method of Random Forest. By taking training and test samples, the precision of the detection model was of about 70% for the rotational and translational types.Los deslizamientos son una amenaza natural que causa pérdidas humanas, daños a la infraestructura y degradación del suelo. Una evaluación cuantitativa de su presencia se requiere mediante la detección y el reconocimiento de potenciales áreas inestables. Esta investigación tuvo como alcance desarrollar un método soportado en métodos semi-automáticos para detectar potenciales movimientos en masa a escala regional. Cinco técnicas fueron estudiadas: Morfometría, Interferometría radar, Interferometría con Persistent Scatterers, Polarimetría radar y composiciones del NDVI con los satélites Landsat 5, Landsat 7 y Landsat 8. El caso de estudio se seleccionó dentro de la región intermedia al este del departamento del Cauca, la cual se caracteriza por terreno montañoso y la presencia de inestabilidades de la pendiente oficialmente registrados en el servicio SIMMA del Servicio Geológico Colombiano. Este inventario reveló que el tipo de movimiento deslizamiento ocurrió con una frecuencia relativa de 77.4%, caidos con el 16.5% de los casos y reptaciones con 3%, flujos con 2.6% y propagación lateral con 0.43%. Como resultado, se obtuvo las variables morfométricas: pendiente, convergencia, índice topográfico de humedad y forma del terreno altamente asociados con los deslizamientos. El efecto de un DEM en el procesamiento del método InSAR fue similar para la variable coherencia usando los DEMs: ASTER, PAlSAR RTC, Topo-map y SRTM. Un análisis Multi-InSAR estimó velocidades de desplazamiento en dirección de vista del radar entre -10 y 10 mm/año. El análisis de polarimetría dual del Sentinel-1 arrojó valores de retrodispersión promedio de -14.5 dB en la banda VH y -8.5dB en la banda VV. Las cuatro polarimetrías del sensor aéreo UAVSAR permitió caracterizar el mecanismo de dispersión del Inventario de Deslizamiento así: 39% en el mecanismo de superficie, 46.4% en el mecanismo de volumen y 14.6% en el mecanismo de doble rebote. La información generada en el rango óptico permitió obtener composiciones de NDVI derivados de la plataforma Landsat entre los años 2012 y 2016, mostrando que el rango entre 0.4 y 0.7 tuvieron una alta asociación con los deslizamientos. En esta investigación se determinaron las categorías de las variables de Teledetección más altamente relacionadas con los movimientos en masa mediante el método de Pesos de Evidencias (WofE). Finalmente, estos resultados se fusionaron para generar el modelo de detección de deslizamientos usando el método supervisado de aprendizaje de máquina Random Forest. Tomando muestras aleatorias para entrenar y validar el modelo en una proporción 70:30, el modelo de detección, especialmente los movimientos de tipo rotacional y traslacional fueron clasificados con una tasa general de éxito del 70%.Ministerio de CienciasConvocatoria 647 de 2014Research line: Geotechnics and Geoenvironmental HazardDoctorad

Advanced pixel selection and optimization algorithms for Persistent Scatterer Interferometry (PSI)

Tesi amb diferents seccions retallades per dret de l'editorPremi Extraordinari de Doctorat, promoció 2018-2019. Àmbit de les TICGround deformation measurements can provide valuable information for minimization of associated loss and damage caused by natural and environmental hazards. As a kind of remote sensing technique, Persistent Scatterer Interferometry (PSI) SAR is able to measure ground deformation with high spatial resolution, efficiently. Moreover, the ground deformation monitoring accuracy of PSI techniques can reach up to millimeter level. However, low coherence could hinderthe exploitation of SAR data, and high-accuracy deformation monitoring can only be achieved by PSI for high quality pixels. Therefore, pixel optimization and identification of coherent pixels are crucial for PSI techniques. In this thesis, advanced pixel selection and optimization algorithms have been investigated. Firstly, a full-resolution pixel selection method based on the Temporal Phase Coherence (TPC) has been proposed. This method first estimates noise phase term of each pixel at interferogram level. Then, for each pixel, its noise phase terms of all interferograms are used to assess this pixel’s temporal phase quality (i.e., TPC). In the next, based on the relationship between TPC and phase Standard Deviation (STD), a threshold can be posed on TPC to identify high phase quality pixels. This pixel selection method can work with both Deterministic Scatterers (PSs) and Distributed Scatterers (DSs). To valid the effectiveness of the developed method, it has been used to monitor the Canillo (Andorra) landslide. The results show that the TPC method can obtained highest density of valid pixels among the employed three approaches in this challenging area with X-band SAR data. Second, to balance the polarimetric DInSAR phase optimization effect and the computation cost, a new PolPSI algorithm is developed. This proposed PolPSI algorithm is based on the Coherency Matrix Decomposition result to determine the optimal scattering mechanism of each pixel, thus it is named as CMD-PolPSI. CMDPolPSI need not to search for solution within the full space of solution, it is therefore much computationally faster than the classical Equal Scattering Mechanism (ESM) method, but with lower optimization performance. On the other hand, its optimization performance outperforms the less computational costly BEST method. Third, an adaptive algorithm SMF-POLOPT has been proposed to adaptive filtering and optimizing PolSAR pixels for PolPSI applications. This proposed algorithm is based on PolSAR classification results to firstly identify Polarimetric Homogeneous Pixels (PHPs) for each pixel, and at the same time classify PS and DS pixels. After that, DS pixels are filtered by their associated PHPs, and then optimized based on the coherence stability phase quality metric; PS pixels are unfiltered and directly optimized based on the DA phase quality metric. SMF-POLOPT can simultaneously reduce speckle noise and retain structures’ details. Meanwhile, SMF-POLOPT is able to obtain much higher density of valid pixels for deformation monitoring than the ESM method. To conclude, one pixel selection method has been developed and tested, two PolPSI algorithms have been proposed in this thesis. This work make contributions to the research of “Advanced Pixel Selection and Optimization Algorithms for Persistent Scatterer InterferometryLes mesures de deformació del sòl poden proporcionar informació valuosa per minimitzar les pèrdues i els danys associats causats pels riscos naturals i ambientals. Com a tècnica de teledetecció, la interferometria de dispersors persistents (Persistent Scatter Interferometry, PSI) SAR és capaç de mesurar de forma eficient la deformació del terreny amb una alta resolució espacial. A més, la precisió de monitorització de la deformació del sòl de les tècniques PSI pot arribar a arribar a nivells del mil·límetre. No obstant això, una baixa coherència pot dificultar l’explotació de dades SAR i el control de deformació d’alta precisió només es pot aconseguir mitjançant PSI per a píxels d’alta qualitat. Per tant, l’optimització de píxels i la identificació de píxels coherents són crucials en les tècniques PSI. En aquesta tesi s¿han investigat algorismes avançats de selecció i optimització de píxels. En primer lloc, s'ha proposat un mètode de selecció de píxels de resolució completa basat en la coherència temporal de fase (Temporal Phase Coherence, TPC). Aquest mètode estima per primera vegada el terme de fase de soroll de cada píxel a nivell d’interferograma. A continuació, per a cada píxel, s'utilitzen els termes de la fase de soroll de tots els interferogrames per avaluar la qualitat de fase temporal d'aquest píxel (és a dir, TPC). A la següent, basant-se en la relació entre el TPC i la desviació estàndard de fase (STD), es pot plantejar un llindar de TPC per identificar píxels de qualitat de fase alta. Aquest mètode de selecció de píxels es capaç de detectar tant els dispersors deterministes (PS) com els distribuïts (DS). Per validar l’eficàcia del mètode desenvolupat, s’ha utilitzat per controlar l’esllavissada de Canillo (Andorra). Els resultats mostren que el mètode TPC pot obtenir la major densitat de píxels vàlids, comparat amb els mètodes clàssics de selecció, en aquesta àrea difícil amb dades de SAR de banda X. En segon lloc, per equilibrar l’efecte d’optimització de fase DInSAR polarimètrica i el cost de càlcul, es desenvolupa un nou algorisme de PolPSI. Aquest algorisme proposat de PolPSI es basa en el resultat de la descomposició de la matriu de coherència per determinar el mecanisme de dispersió òptim de cada píxel, de manera que es denomina CMD-PolPSI. CMDPolPSI no necessita buscar solucions dins de l’espai complet de la solució, per tant, és molt més eficient computacionalment que el mètode clàssic de mecanismes d’igualtat de dispersió (Equal Scattering Mechanism, ESM), però amb un efecte d’optimització no tant òptim. D'altra banda, el seu efecte d'optimització supera el mètode BEST, el que te un menor cost computacional. En tercer lloc, s'ha proposat un algoritme adaptatiu SMF-POLOPT per al filtratge adaptatiu i l'optimització de píxels PolSAR per a aplicacions PolPSI. Aquest algorisme proposat es basa en els resultats de classificació PolSAR per identificar primer els píxels homogenis polarimètrics (PHP) per a cada píxel i, alhora, classificar els píxels PS i DS. Després d'això, els píxels DS es filtren pels seus PHP associats i, a continuació, s'optimitzen en funció de la mètrica de qualitat de la fase d'estabilitat de coherència; els píxels classificats com PS no es filtren i s'optimitzen directament en funció de la mètrica de qualitat de la fase DA. SMF-POLOPT pot reduir simultàniament el soroll de la fase interferomètrica i conservar els detalls de les estructures. Mentrestant, SMF-POLOPT aconsegueix obtenir una densitat molt més alta de píxels vàlids per al seguiment de la deformació que el mètode ESM. Per concloure, en aquesta tesi s’ha desenvolupat i provat un mètode de selecció de píxels, i s’han proposat dos algoritmes PolPSI. Aquest treball contribueix a la recerca en "Advanced Pixel Selection and Optimization Algorithms for Persistent Scatterer Interferometry"Postprint (published version