Fusion of VNIR Optical and C-Band Polarimetric SAR Satellite Data for Accurate Detection of Temporal Changes in Vegetated Areas

In this paper, we propose a processing chain jointly employing Sentinel-1 and Sentinel-2 data, aiming to monitor changes in the status of the vegetation cover by integrating the four 10 m visible and near-infrared (VNIR) bands with the three red-edge (RE) bands of Sentinel-2. The latter approximately span the gap between red and NIR bands (700 nm–800 nm), with bandwidths of 15/20 nm and 20 m pixel spacing. The RE bands are sharpened to 10 m, following the hypersharpening protocol, which holds, unlike pansharpening, when the sharpening band is not unique. The resulting 10 m fusion product may be integrated with polarimetric features calculated from the Interferometric Wide (IW) Ground Range Detected (GRD) product of Sentinel-1, available at 10 m pixel spacing, before the fused data are analyzed for change detection. A key point of the proposed scheme is that the fusion of optical and synthetic aperture radar (SAR) data is accomplished at level of change, through modulation of the optical change feature, namely the difference in normalized area over (reflectance) curve (NAOC), calculated from the sharpened RE bands, by the polarimetric SAR change feature, achieved as the temporal ratio of polarimetric features, where the latter is the pixel ratio between the co-polar and the cross-polar channels. Hyper-sharpening of Sentinel-2 RE bands, calculation of NAOC and modulation-based integration of Sentinel-1 polarimetric change features are applied to multitemporal datasets acquired before and after a fire event, over Mount Serra, in Italy. The optical change feature captures variations in the content of chlorophyll. The polarimetric SAR temporal change feature describes depolarization effects and changes in volumetric scattering of canopies. Their fusion shows an increased ability to highlight changes in vegetation status. In a performance comparison achieved by means of receiver operating characteristic (ROC) curves, the proposed change feature-based fusion approach surpasses a traditional area-based approach and the normalized burned ratio (NBR) index, which is widespread in the detection of burnt vegetation

Advantages of nonlinear intensity components for contrast-based multispectral pansharpening

In this study, we investigate whether a nonlinear intensity component can be beneficial for multispectral (MS) pansharpening based on component-substitution (CS). In classical CS methods, the intensity component is a linear combination of the spectral components and lies on a hyperplane in the vector space that contains the MS pixel values. Starting from the hyperspherical color space (HCS) fusion technique, we devise a novel method, in which the intensity component lies on a hyper-ellipsoidal surface instead of on a hyperspherical surface. The proposed method is insensitive to the format of the data, either floating-point spectral radiance values or fixed-point packed digital numbers (DNs), thanks to the use of a multivariate linear regression between the squares of the interpolated MS bands and the squared lowpass filtered Pan. The regression of squared MS, instead of the Euclidean radius used by HCS, makes the intensity component no longer lie on a hypersphere in the vector space of the MS samples, but on a hyperellipsoid. Furthermore, before the fusion is accomplished, the interpolated MS bands are corrected for atmospheric haze, in order to build a multiplicative injection model with approximately de-hazed components. Experiments on GeoEye-1 and WorldView-3 images show consistent advantages over the baseline HCS and a performance slightly superior to those of some of the most advanced methodsPeer ReviewedPostprint (published version

Optimization of Optical Image Geometric Modeling, Application to Topography Extraction and Topographic Change Measurements Using PlanetScope and SkySat Imagery

The volume of data generated by earth observation satellites has increased tremendously over the last few decades and will increase further in the coming decade thanks in particular to the launch of nanosatellites constellations. These data should open new avenues for Earth surface monitoring due to highly improved spectral, spatial and temporal resolution. Many applications depend, however, on the accuracy of the image geometric model. The geometry of optical images, whether acquired from pushbroom or frame systems, is now commonly represented using a Rational Function Model (RFM). While the formalism has become standard, the procedures used to generate these models and their accuracies are diverse. As a result, the RFM models delivered with commercial data are commonly not accurate enough for 3-D extraction, subpixel registration or ground deformation measurements. In this study, we present a methodology for RFM optimization and demonstrate its potential for 3D reconstruction using tri-stereo and multi-date Cubesat images provided by SkySat and PlanetScope, respectively. We use SkySat data over the Morenci Mine, Arizona, which is the largest copper mine in the United States. The re-projection error after the RFM refinement is 0.42 pix without using ground control points (GCPs). Comparison of our Digital Elevation Model (DEM with ~3 m GSD) with a reference DEM obtained from an airborne LiDAR survey (with ~1 m GSD) over stable areas yields a standard deviation of the elevation differences of ~3.9 m. The comparison of the two DEMs allows detecting and measuring the topographic changes due to the mine activity (excavation and stockpiles). We assess the potential of PlanetScope data, using multi-date DOVE-C images from the Shisper glacier, located in the Karakoram (Pakistan), which is known for its recent surge. We extracted DEMs in 2017 and 2019 before and after the surge. The re-projection error after the RFM refinement is 0.38 pix without using GCPs. The accuracy of our DEMs (with ~9 m GSD) is evaluated through comparison with the SRTM DEM (GSD ~30 m) and with a DEM (GSD ~2 m) calculated from Geoeye-1 (GE-1) and World-View-2 (WV-2) stereo images. The standard deviation of the elevation differences in stable areas between the PlanetScope DEM and SRTM is ~12 m, and ~7 m with the GE-1&WV-2 DEM. The mass transfer due to the surge is clearly revealed from a comparison of the 2017 and 2019 DEMs. The study demonstrates that, with the proposed scheme for RFM optimization, times series of DEM extracted from SkySat and PlanetScope images can be used to measure topographic changes due to mining activities or ice flow, and could also be used to monitor geomorphic processes such as landslides, or coastal erosion for example

Optimization of Optical Image Geometric Modeling, Application to Topography Extraction and Topographic Change Measurements Using PlanetScope and SkySat Imagery

The volume of data generated by earth observation satellites has increased tremendously over the last few decades and will increase further in the coming decade thanks in particular to the launch of nanosatellites constellations. These data should open new avenues for Earth surface monitoring due to highly improved spectral, spatial and temporal resolution. Many applications depend, however, on the accuracy of the image geometric model. The geometry of optical images, whether acquired from pushbroom or frame systems, is now commonly represented using a Rational Function Model (RFM). While the formalism has become standard, the procedures used to generate these models and their accuracies are diverse. As a result, the RFM models delivered with commercial data are commonly not accurate enough for 3-D extraction, subpixel registration or ground deformation measurements. In this study, we present a methodology for RFM optimization and demonstrate its potential for 3D reconstruction using tri-stereo and multi-date Cubesat images provided by SkySat and PlanetScope, respectively. We use SkySat data over the Morenci Mine, Arizona, which is the largest copper mine in the United States. The re-projection error after the RFM refinement is 0.42 pix without using ground control points (GCPs). Comparison of our Digital Elevation Model (DEM with ~3 m GSD) with a reference DEM obtained from an airborne LiDAR survey (with ~1 m GSD) over stable areas yields a standard deviation of the elevation differences of ~3.9 m. The comparison of the two DEMs allows detecting and measuring the topographic changes due to the mine activity (excavation and stockpiles). We assess the potential of PlanetScope data, using multi-date DOVE-C images from the Shisper glacier, located in the Karakoram (Pakistan), which is known for its recent surge. We extracted DEMs in 2017 and 2019 before and after the surge. The re-projection error after the RFM refinement is 0.38 pix without using GCPs. The accuracy of our DEMs (with ~9 m GSD) is evaluated through comparison with the SRTM DEM (GSD ~30 m) and with a DEM (GSD ~2 m) calculated from Geoeye-1 (GE-1) and World-View-2 (WV-2) stereo images. The standard deviation of the elevation differences in stable areas between the PlanetScope DEM and SRTM is ~12 m, and ~7 m with the GE-1&WV-2 DEM. The mass transfer due to the surge is clearly revealed from a comparison of the 2017 and 2019 DEMs. The study demonstrates that, with the proposed scheme for RFM optimization, times series of DEM extracted from SkySat and PlanetScope images can be used to measure topographic changes due to mining activities or ice flow, and could also be used to monitor geomorphic processes such as landslides, or coastal erosion for example

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

Multispectral Image Road Extraction Based Upon Automated Map Conflation

Road network extraction from remotely sensed imagery enables many important and diverse applications such as vehicle tracking, drone navigation, and intelligent transportation studies. There are, however, a number of challenges to road detection from an image. Road pavement material, width, direction, and topology vary across a scene. Complete or partial occlusions caused by nearby buildings, trees, and the shadows cast by them, make maintaining road connectivity difficult. The problems posed by occlusions are exacerbated with the increasing use of oblique imagery from aerial and satellite platforms. Further, common objects such as rooftops and parking lots are made of materials similar or identical to road pavements. This problem of common materials is a classic case of a single land cover material existing for different land use scenarios. This work addresses these problems in road extraction from geo-referenced imagery by leveraging the OpenStreetMap digital road map to guide image-based road extraction. The crowd-sourced cartography has the advantages of worldwide coverage that is constantly updated. The derived road vectors follow only roads and so can serve to guide image-based road extraction with minimal confusion from occlusions and changes in road material. On the other hand, the vector road map has no information on road widths and misalignments between the vector map and the geo-referenced image are small but nonsystematic. Properly correcting misalignment between two geospatial datasets, also known as map conflation, is an essential step. A generic framework requiring minimal human intervention is described for multispectral image road extraction and automatic road map conflation. The approach relies on the road feature generation of a binary mask and a corresponding curvilinear image. A method for generating the binary road mask from the image by applying a spectral measure is presented. The spectral measure, called anisotropy-tunable distance (ATD), differs from conventional measures and is created to account for both changes of spectral direction and spectral magnitude in a unified fashion. The ATD measure is particularly suitable for differentiating urban targets such as roads and building rooftops. The curvilinear image provides estimates of the width and orientation of potential road segments. Road vectors derived from OpenStreetMap are then conflated to image road features by applying junction matching and intermediate point matching, followed by refinement with mean-shift clustering and morphological processing to produce a road mask with piecewise width estimates. The proposed approach is tested on a set of challenging, large, and diverse image data sets and the performance accuracy is assessed. The method is effective for road detection and width estimation of roads, even in challenging scenarios when extensive occlusion occurs

Deep learning for inverse problems in remote sensing: super-resolution and SAR despeckling

L'abstract è presente nell'allegato / the abstract is in the attachmen

Polarimetric Image Reconstruction Algorithms.

In the field of imaging polarimetry Stokes parameters are sought and must be inferred from noisy and blurred intensity measurements. Using a penalized-likelihood estimation framework we investigate reconstruction quality when estimating intensity images and then transforming to Stokes parameters (traditional estimator), and when estimating Stokes parameters directly (Stokes estimator). We define our cost function for reconstruction by a weighted least squares data fit term and a regularization penalty. It is shown that under quadratic regularization, the traditional and Stokes estimators can be made equal by appropriate choice of regularization parameters. It is empirically shown that, when using edge preserving regularization, estimating the Stokes parameters directly leads to lower textsc{RMS} error in reconstruction. Also, the addition of a cross channel regularization term further lowers the textsc{RMS} error for both methods especially in the case of low textsc{SNR}. The technique of phase diversity has been used in traditional incoherent imaging systems to jointly estimate an object and optical system aberrations. We extend the technique of phase diversity to polarimetric imaging systems. Specifically, we describe penalized-likelihood methods for jointly estimating Stokes images and optical system aberrations from measurements that contain phase diversity. Jointly estimating Stokes images and optical system aberrations involves a large parameter space. A closed-form expression for the estimate of the Stokes images in terms of the aberration parameters is derived and used in a formulation that reduces the dimensionality of the search space to the number of aberration parameters only. We compare the performance of the joint estimator under both quadratic and edge-preserving regularization. The joint estimator with edge-preserving regularization yields higher fidelity polarization estimates than with quadratic regularization. Under quadratic regularization, using the reduced-parameter search strategy, accurate aberration estimates can be obtained without recourse to regularization ``tuning''. Phase-diverse wavefront sensing is emerging as a viable candidate wavefront sensor for adaptive-optics systems. In a quadratically penalized weighted least squares estimation framework a closed form expression for the object being imaged in terms of the aberrations in the system is available.Ph.D.Applied PhysicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/77925/1/jvalenz_1.pd

Earth imaging with microsatellites: An investigation, design, implementation and in-orbit demonstration of electronic imaging systems for earth observation on-board low-cost microsatellites.

This research programme has studied the possibilities and difficulties of using 50 kg microsatellites to perform remote imaging of the Earth. The design constraints of these missions are quite different to those encountered in larger, conventional spacecraft. While the main attractions of microsatellites are low cost and fast response times, they present the following key limitations: Payload mass under 5 kg, Continuous payload power under 5 Watts, peak power up to 15 Watts, Narrow communications bandwidths (9.6 / 38.4 kbps), Attitude control to within 5°, No moving mechanics. The most significant factor is the limited attitude stability. Without sub-degree attitude control, conventional scanning imaging systems cannot preserve scene geometry, and are therefore poorly suited to current microsatellite capabilities. The foremost conclusion of this thesis is that electronic cameras, which capture entire scenes in a single operation, must be used to overcome the effects of the satellite's motion. The potential applications of electronic cameras, including microsatellite remote sensing, have erupted with the recent availability of high sensitivity field-array CCD (charge-coupled device) image sensors. The research programme has established suitable techniques and architectures necessary for CCD sensors, cameras and entire imaging systems to fulfil scientific/commercial remote sensing despite the difficult conditions on microsatellites. The author has refined these theories by designing, building and exploiting in-orbit five generations of electronic cameras. The major objective of meteorological scale imaging was conclusively demonstrated by the Earth imaging camera flown on the UoSAT-5 spacecraft in 1991. Improved cameras have since been carried by the KITSAT-1 (1992) and PoSAT-1 (1993) microsatellites. PoSAT-1 also flies a medium resolution camera (200 metres) which (despite complete success) has highlighted certain limitations of microsatellites for high resolution remote sensing. A reworked, and extensively modularised, design has been developed for the four camera systems deployed on the FASat-Alfa mission (1995). Based on the success of these missions, this thesis presents many recommendations for the design of microsatellite imaging systems. The novelty of this research programme has been the principle of designing practical camera systems to fit on an existing, highly restrictive, satellite platform, rather than conceiving a fictitious small satellite to support a high performance scanning imager. This pragmatic approach has resulted in the first incontestable demonstrations of the feasibility of remote sensing of the Earth from inexpensive microsatellites