Registration of Multisensor Images through a Conditional Generative Adversarial Network and a Correlation-Type Similarity Measure

The automatic registration of multisensor remote sensing images is a highly challenging task due to the inherently different physical, statistical, and textural characteristics of the input data. Information-theoretic measures are often used to favor comparing local intensity distributions in the images. In this paper, a novel method based on the combination of a deep learning architecture and a correlation-type area-based functional is proposed for the registration of a multisensor pair of images, including an optical image and a synthetic aperture radar (SAR) image. The method makes use of a conditional generative adversarial network (cGAN) in order to address image-to-image translation across the optical and SAR data sources. Then, once the optical and SAR data are brought to a common domain, an area-based ℓ2 similarity measure is used together with the COBYLA constrained maximization algorithm for registration purposes. While correlation-type functionals are usually ineffective in the application to multisensor registration, exploiting the image-to-image translation capabilities of cGAN architectures allows moving the complexity of the comparison to the domain adaptation step, thus enabling the use of a simple ℓ2 similarity measure, favoring high computational efficiency, and opening the possibility to process a large amount of data at runtime. Experiments with multispectral and panchromatic optical data combined with SAR images suggest the effectiveness of this strategy and the capability of the proposed method to achieve more accurate registration as compared to state-of-the-art approaches

Enhanced phase congruency feature-based image registration for multimodal remote sensing imagery

Multimodal image registration is an essential image processing task in remote sensing. Basically, multimodal image registration searches for optimal alignment between images captured by different sensors for the same scene to provide better visualization and more informative images. Manual image registration is a tedious task and requires more effort, hence developing an automated image registration is very crucial to provide a faster and reliable solution. However, image registration faces many challenges from the nature of remote sensing image, the environment, and the technical shortcoming of the current methods that cause three issues, namely intensive processing power, local intensity variation, and rotational distortion. Since not all image details are significant, relying on the salient features will be more efficient in terms of processing power. Thus, the feature-based registration method was adopted as an efficient method to avoid intensive processing. The proposed method resolves rotation distortion issue using Oriented FAST and Rotated BRIEF (ORB) to produce invariant rotation features. However, since it is not intensity invariant, it cannot support multimodal data. To overcome the intensity variations issue, Phase Congruence (PC) was integrated with ORB to introduce ORB-PC feature extraction to generate feature invariance to rotation distortion and local intensity variation. However, the solution is not complete since the ORB-PC matching rate is below the expectation. Enhanced ORB-PC was proposed to solve the matching issue by modifying the feature descriptor. While better feature matches were achieved, a high number of outliers from multimodal data makes the common outlier removal methods unsuccessful. Therefore, the Normalized Barycentric Coordinate System (NBCS) outlier removal was utilized to find precise matches even with a high number of outliers. The experiments were conducted to verify the registration qualitatively and quantitatively. The qualitative experiment shows the proposed method has a broader and better features distribution, while the quantitative evaluation indicates improved performance in terms of registration accuracy by 18% compared to the related works

Deep learning-based change detection in remote sensing images:a review

Images gathered from different satellites are vastly available these days due to the fast development of remote sensing (RS) technology. These images significantly enhance the data sources of change detection (CD). CD is a technique of recognizing the dissimilarities in the images acquired at distinct intervals and are used for numerous applications, such as urban area development, disaster management, land cover object identification, etc. In recent years, deep learning (DL) techniques have been used tremendously in change detection processes, where it has achieved great success because of their practical applications. Some researchers have even claimed that DL approaches outperform traditional approaches and enhance change detection accuracy. Therefore, this review focuses on deep learning techniques, such as supervised, unsupervised, and semi-supervised for different change detection datasets, such as SAR, multispectral, hyperspectral, VHR, and heterogeneous images, and their advantages and disadvantages will be highlighted. In the end, some significant challenges are discussed to understand the context of improvements in change detection datasets and deep learning models. Overall, this review will be beneficial for the future development of CD methods

Multisource Remote Sensing based Impervious Surface Mapping

Impervious surface (IS) not only serves as a key indicator of urbanization, but also affects the micro-ecosystem. Therefore, it is essential to monitor IS distribution timely and accurately. Remote sensing is an effective approach as it can provide straightforward and consistent information over large area with low cost. This thesis integrates multi-source remote sensing data to interpretate urban patterns and provide more reliable IS mapping results. Registration of optical daytime and nighttime lights (NTL) data is developed in the first contribution. An impervious surface based optical-to-NTL image registration algorithm with iterative blooming effect reduction (IS_iBER) algorithm is proposed. This coarse-to-fine procedure investigates the correlation between optical and NTL features. The iterative registration and blooming effect reduction method obtains precise matching results and reduce the spatial extension of NTL. Considering the spatial transitional nature of urban-rural fringes (URF) areas, the second study proposed approach for URF delineation, namely optical and nighttime lights (NTL) data based multi-scale URF (msON_URF).The landscape heterogeneity and development vitality derived from optical and NTL features are analyzed at a series of scales to illustrate the urban-URF-rural pattern. Results illustrate that msON_URF is effective and practical for not only concentric, but also polycentric urban patterns. The third study proposes a nighttime light adjusted impervious surface index (NAISI) to detect IS area. Parallel to baseline subtraction approaches, NAISI takes advantage of features, rather than spectral band information to map IS. NAISI makes the most of independence between NTL-ISS and pervious surface to address the high spectral similarity between IS and bare soil in optical image. An optical and NTL based spectral mixture analysis (ON_SMA) is proposed to achieve sub-pixel IS mapping result in the fourth study. It integrates characteristics of optical and NTL imagery to adaptively select local endmembers. Results illustrate the proposed method yields effective improvement and highlight the potential of NTL data in IS mapping. In the fifth study, GA-SVM IS mapping algorithm is investigated with introduction of the achieved urban-URF-rural spatial structure. The combination of optical, NTL and SAR imagery is discussed. GA is implemented for feature selection and parameter optimization in each urban scenario

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

An Evaluation of Performance Enhancements to Particle Swarm Optimisation on Real-World Data

Swarm Computation is a relatively new optimisation paradigm. The basic premise is to model the collective behaviour of self-organised natural phenomena such as swarms, flocks and shoals, in order to solve optimisation problems. Particle Swarm Optimisation (PSO) is a type of swarm computation inspired by bird flocks or swarms of bees by modelling their collective social influence as they search for optimal solutions. In many real-world applications of PSO, the algorithm is used as a data pre-processor for a neural network or similar post processing system, and is often extensively modified to suit the application. The thesis introduces techniques that allow unmodified PSO to be applied successfully to a range of problems, specifically three extensions to the basic PSO algorithm: solving optimisation problems by training a hyperspatial matrix, using a hierarchy of swarms to coordinate optimisation on several data sets simultaneously, and dynamic neighbourhood selection in swarms. Rather than working directly with candidate solutions to an optimisation problem, the PSO algorithm is adapted to train a matrix of weights, to produce a solution to the problem from the inputs. The search space is abstracted from the problem data. A single PSO swarm optimises a single data set and has difficulties where the data set comprises disjoint parts (such as time series data for different days). To address this problem, we introduce a hierarchy of swarms, where each child swarm optimises one section of the data set whose gbest particle is a member of the swarm above in the hierarchy. The parent swarm(s) coordinate their children and encourage more exploration of the solution space. We show that hierarchical swarms of this type perform better than single swarm PSO optimisers on the disjoint data sets used. PSO relies on interaction between particles within a neighbourhood to find good solutions. In many PSO variants, possible interactions are arbitrary and fixed on initialisation. Our third contribution is a dynamic neighbourhood selection: particles can modify their neighbourhood, based on the success of the candidate neighbour particle. As PSO is intended to reflect the social interaction of agents, this change significantly increases the ability of the swarm to find optimal solutions. Applied to real-world medical and cosmological data, this modification is and shows improvements over standard PSO approaches with fixed neighbourhoods