A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

High-resolution optical and SAR image fusion for building database updating

This paper addresses the issue of cartographic database (DB) creation or updating using high-resolution synthetic aperture radar and optical images. In cartographic applications, objects of interest are mainly buildings and roads. This paper proposes a processing chain to create or update building DBs. The approach is composed of two steps. First, if a DB is available, the presence of each DB object is checked in the images. Then, we verify if objects coming from an image segmentation should be included in the DB. To do those two steps, relevant features are extracted from images in the neighborhood of the considered object. The object removal/inclusion in the DB is based on a score obtained by the fusion of features in the framework of Dempster–Shafer evidence theory

Spaceborne radar observations: A guide for Magellan radar-image analysis

Geologic analyses of spaceborne radar images of Earth are reviewed and summarized with respect to detecting, mapping, and interpreting impact craters, volcanic landforms, eolian and subsurface features, and tectonic landforms. Interpretations are illustrated mostly with Seasat synthetic aperture radar and shuttle-imaging-radar images. Analogies are drawn for the potential interpretation of radar images of Venus, with emphasis on the effects of variation in Magellan look angle with Venusian latitude. In each landform category, differences in feature perception and interpretive capability are related to variations in imaging geometry, spatial resolution, and wavelength of the imaging radar systems. Impact craters and other radially symmetrical features may show apparent bilateral symmetry parallel to the illumination vector at low look angles. The styles of eruption and the emplacement of major and minor volcanic constructs can be interpreted from morphological features observed in images. Radar responses that are governed by small-scale surface roughness may serve to distinguish flow types, but do not provide unambiguous information. Imaging of sand dunes is rigorously constrained by specific angular relations between the illumination vector and the orientation and angle of repose of the dune faces, but is independent of radar wavelength. With a single look angle, conditions that enable shallow subsurface imaging to occur do not provide the information necessary to determine whether the radar has recorded surface or subsurface features. The topographic linearity of many tectonic landforms is enhanced on images at regional and local scales, but the detection of structural detail is a strong function of illumination direction. Nontopographic tectonic lineaments may appear in response to contrasts in small-surface roughness or dielectric constant. The breakpoint for rough surfaces will vary by about 25 percent through the Magellan viewing geometries from low to high Venusian latitudes. Examples of anomalies and system artifacts that can affect image interpretation are described

Terrain analysis using radar shape-from-shading

This paper develops a maximum a posteriori (MAP) probability estimation framework for shape-from-shading (SFS) from synthetic aperture radar (SAR) images. The aim is to use this method to reconstruct surface topography from a single radar image of relatively complex terrain. Our MAP framework makes explicit how the recovery of local surface orientation depends on the whereabouts of terrain edge features and the available radar reflectance information. To apply the resulting process to real world radar data, we require probabilistic models for the appearance of terrain features and the relationship between the orientation of surface normals and the radar reflectance. We show that the SAR data can be modeled using a Rayleigh-Bessel distribution and use this distribution to develop a maximum likelihood algorithm for detecting and labeling terrain edge features. Moreover, we show how robust statistics can be used to estimate the characteristic parameters of this distribution. We also develop an empirical model for the SAR reflectance function. Using the reflectance model, we perform Lambertian correction so that a conventional SFS algorithm can be applied to the radar data. The initial surface normal direction is constrained to point in the direction of the nearest ridge or ravine feature. Each surface normal must fall within a conical envelope whose axis is in the direction of the radar illuminant. The extent of the envelope depends on the corrected radar reflectance and the variance of the radar signal statistics. We explore various ways of smoothing the field of surface normals using robust statistics. Finally, we show how to reconstruct the terrain surface from the smoothed field of surface normal vectors. The proposed algorithm is applied to various SAR data sets containing relatively complex terrain structure

Using the space-borne NASA scatterometer (NSCAT) to determine the frozen and thawed seasons

We hypothesize that the strong sensitivity of radar backscatter to surface dielectric properties, and hence to the phase (solid or liquid) of any water near the surface should make space-borne radar observations a powerful tool for large-scale spatial monitoring of the freeze/thaw state of the land surface, and thus ecosystem growing season length. We analyzed the NASA scatterometer (NSCAT) backscatter from September 1996 to June 1997, along with temperature and snow depth observations and ecosystem modeling, for three BOREAS sites in central Canada. Because of its short wavelength (2.14 cm), NSCAT was sensitive to canopy and surface water. NSCAT had 25 km spatial resolution and approximately twice-daily temporal coverage at the BOREAS latitude. At the northern site the NSCAT signal showed strong seasonality, with backscatter around −8 dB in winter and −12 dB in early summer and fall. The NSCAT signal for the southern sites had less seasonality. At all three sites there was a strong decrease in backscatter during spring thaw (4–6 dB). At the southern deciduous site, NSCAT backscatter rose from −11 to −9.2 dB during spring leaf-out. All sites showed 1–2 dB backscatter shifts corresponding to changes in landscape water state coincident with brief midwinter thaws, snowfall, and extreme cold (Tmax\u3c−25°C). Freeze/thaw detection algorithms developed for other radar instruments gave reasonable results for the northern site but were not successful at the two southern sites. We developed a change detection algorithm based on first differences of 5-day smoothed NSCAT backscatter measurements. This algorithm had some success in identifying the arrival of freezing conditions in the autumn and the beginning of thaw in the spring. Changes in surface freeze/thaw state generally coincided with the arrival and departure of the seasonal snow cover and with simulated shifts in the directions of net carbon exchange at each of the study sites