A hybrid pan-sharpening approach using maximum local extrema

Mixing or combining different elements for getting enhanced version, is practiced across various areas in real life. Pan-sharpening is a similar technique used in the digital world; a process to combine two images into a fused image that comprises more detailed information. Images referred herein are Panchromatic (PAN) and Multispectral (MS) images. This paper presents a pansharpening algorithm which integrates multispectral and panchromatic images to generate an improved multispectral image. This technique merges the Discrete wavelet transform (WT) and Intensity-Hue-Saturation (IHS) through separate fusing criterion for choosing an approximate and detail sub-images. Whereas the maximal local extrema are used for merging detail sub-images and finally merged high-resolution image is reconstructed through inverse transform of wavelet and IHS. The proposed fusion approach enhances the superiority of the resultant fused image is demonstrated by quality measures like CORR, RMSE, PFE, SSIM, SNR and PSNR with the help of satellite Worldview-II images. The proposed algorithm is correlated with the other fusion techniques through empirical outcomes proves the superiority of the final merged image in terms of resolutions than the others

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

A New Pansharpening Approach for Hyperspectral Images

We first briefly review recent papers for pansharpening of hyperspectral (HS) images. We then present a recent pansharpening approach called hybrid color mapping (HCM). A few variants of HCM are then summarized. Using two hyperspectral images, we illustrate the advantages of HCM by comparing HCM with 10 state-of-the-art algorithms

Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

International audienceHyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Non-linear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand, an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that, an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time

Evaluation of Pan-Sharpening Techniques Using Lagrange Optimization

Earth’s observation satellites, such as IKONOS, provide simultaneously multispectral and panchromatic images. A multispectral image comes with a lower spatial and higher spectral resolution in contrast to a panchromatic image which usually has a high spatial and a low spectral resolution. Pan-sharpening represents a fusion of these two complementary images to provide an output image that has both spatial and spectral high resolutions. The objective of this paper is to propose a new method of pan-sharpening based on pixel-level image manipulation and to compare it with several state-of-art pansharpening methods using different evaluation criteria.  The paper presents an image fusion method based on pixel-level optimization using the Lagrange multiplier. Two cases are discussed: (a) the maximization of spectral consistency and (b) the minimization of the variance difference between the original data and the computed data. The paper compares the results of the proposed method with several state-of-the-art pan-sharpening methods. The performance of the pan-sharpening methods is evaluated qualitatively and quantitatively using evaluation criteria, such as the Chi-square test, RMSE, SNR, SD, ERGAS, and RASE. Overall, the proposed method is shown to outperform all the existing methods

Assessing the effect of band selection on accuracy of pansharpened imagery: application to young woody vegetation mapping

Expansion of woody vegetation has adverse effects on ecosystem services, and thus it is desirable to contain the problem at the early developmental stages. This can be aided by using high spatial resolution remotely-sensed data. The study investigated the effect of band selection during pansharpening on the ability to discriminate young woody vegetation from coexisting land cover types. Red-green-blue (RGB) spectral bands (30 m) of Landsat 8 imagery was pansharpened using the panchromatic band (15 m) of the same image to improve spatial resolution. Near-infrared (NIR), shortwave-infrared 1 (SWIR1) and shortwave-infrared 2 (SWIR2), bands were used respectively as the fourth spectral band during pansharpening, resulting in three pansharpened images. Unsupervised classification was performed on each pansharpened image as well as non-pansharpened multispectral image. The overall accuracies of classification derived from the pansharpened image was higher (87% − 89%) than that derived from the non-pansharpened multispectral image (83%). The study shows that band selection did not affect the classification accuracy of woody vegetation significantly. In addition, the study shows the potential of pansharpened Landsat data in detecting woody vegetation encroachment at the early growth stage.Keywords: Young woody vegetation, Landsat, pansharpening, unsupervised classificatio

Extraction of low cost houses from a high spatial resolution satellite imagery using Canny edge detection filter

Since its democratic dispensation in 1994, the South African government enacted a number of legislative and policy interventions aimed at availing equal housing opportunities to the previously marginalized citizens. Mismanagement and unreliable reporting has been widely reported in publicly funded housing programmes which necessitated the government to audit and monitor housing development projects in municipalities using more robust and independent methodologies. The objective of this study was therefore to test and demonstrate the effectiveness of high spatial resolution satellite imagery in validating the presence of government funded houses using an object-oriented classification technique that applies a Canny edge detection filter. The results of this study demonstrate that object-orientated classification applied on pan-sharpened SPOT 6 satellite imagery can be used to conduct a reliable inventory and validate the number of houses. The application of the multi-resolution segmentation and Canny edge detection filtering technique proved to be an effective means of mapping individual houses as shown by the high detection accuracy of 99% and quality percentage of 96%.Keywords: Houses, Remote Sensing, SPOT 6, Canny edge detection, Multi-resolution Segmentation, Object-Oriented Classificatio

Bayesian Fusion of Multi-Band Images

International audienceThis paper presents a Bayesian fusion technique for remotely sensed multi-band images. The observed images are related to the high spectral and high spatial resolution image to be recovered through physical degradations, e.g., spatial and spectral blurring and/or subsampling defined by the sensor characteristics. The fusion problem is formulated within a Bayesian estimation framework. An appropriate prior distribution exploiting geometrical considerations is introduced. To compute the Bayesian estimator of the scene of interest from its posterior distribution, a Markov chain Monte Carlo algorithm is designed to generate samples asymptotically distributed according to the target distribution. To efficiently sample from this high-dimension distribution, a Hamiltonian Monte Carlo step is introduced within a Gibbs sampling strategy. The efficiency of the proposed fusion method is evaluated with respect to several state-of-the-art fusion techniques

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin