Nonparametric image registration of airborne LiDAR, hyperspectral and photographic imagery of wooded landscapes

There is much current interest in using multisensor airborne remote sensing to monitor the structure and biodiversity of woodlands. This paper addresses the application of nonparametric (NP) image-registration techniques to precisely align images obtained from multisensor imaging, which is critical for the successful identification of individual trees using object recognition approaches. NP image registration, in particular, the technique of optimizing an objective function, containing similarity and regularization terms, provides a flexible approach for image registration. Here, we develop a NP registration approach, in which a normalized gradient field is used to quantify similarity, and curvature is used for regularization (NGF-Curv method). Using a survey of woodlands in southern Spain as an example, we show that NGF-Curv can be successful at fusing data sets when there is little prior knowledge about how the data sets are interrelated (i.e., in the absence of ground control points). The validity of NGF-Curv in airborne remote sensing is demonstrated by a series of experiments. We show that NGF-Curv is capable of aligning images precisely, making it a valuable component of algorithms designed to identify objects, such as trees, within multisensor data sets.This work was supported by the Airborne Research and Survey Facility of the U.K.’s Natural Environment Research Council (NERC) for collecting and preprocessing the data used in this research project [EU11/03/100], and by the grants supported from King Abdullah University of Science Technology and Wellcome Trust (BBSRC). D. Coomes was supported by a grant from NERC (NE/K016377/1) and funding from DEFRA and the BBSRC to develop methods for monitoring ash dieback from aircraft.This is the final version. It was first published by IEEE at http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7116541&sortType%3Dasc_p_Sequence%26filter%3DAND%28p_Publication_Number%3A36%29%26pageNumber%3D5

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Intensity based image registration of satellite images using evolutionary techniques

Image registration is the fundamental image processing technique to determine geometrical transformation that gives the most accurate match between reference and floating images. Its main aim is to align two images. Satellite images to be fused for numerous applications must be registered before use. The main challenges in satellite image registration are finding out the optimum transformation parameters. Here in this work the non-alignment parameters are considered to be rigid and affine transformation. An intensity based satellite image registration technique is being used to register the floating image to the native co-ordinate system where the normalized mutual information (NMI) is taken as the similarity metric for optimizing and updating transform parameters. Because of no assumptions are made regarding the nature of the relationship between the image intensities in both modalities NMI is very general and powerful and can be applied automatically without prior segmentation on a large variety of data and as well works better for overlapped images as compared to mutual information(MI). In order to get maximum accuracy of registration the NMI is optimized using Genetic algorithm, particle swarm optimization and hybrid GA-PSO. The random initialization and computational complexity makes GA oppressive, whereas weak local search ability with a premature convergence is the main drawback of PSO. Hybrid GA-PSO makes a trade-off between the local and global search in order to achieve a better balance between convergence speed and computational complexity. The above registration algorithm is being validated with several satellite data sets. The hybrid GA-PSO outperforms in terms of optimized NMI value and percentage of mis-registration error

Using redundant information from multiple aerial images for the detection of bomb craters based on marked point processes

Many countries were the target of air strikes during World War II. Numerous unexploded bombs still exist in the ground. These duds can be tracked down with the help of bomb craters, indicating areas where unexploded bombs may be located. Such areas are documented in so-called impact maps based on detected bomb craters. In this paper, a stochastic approach based on marked point processes (MPPs) for the automatic detection of bomb craters in aerial images taken during World War II is presented. As most areas are covered by multiple images, the influence of redundant image information on the object detection result is investigated: We compare the results generated based on single images with those obtained by our new approach that combines the individual detection results of multiple images covering the same location. The object model for the bomb craters is represented by circles. Our MPP approach determines the most likely configuration of objects within the scene. The goal is reached by minimizing an energy function that describes the conformity with a predefined model by Reversible Jump Markov Chain Monte Carlo sampling in combination with simulated annealing. Afterwards, a probability map is generated from the automatic detections via kernel density estimation. By setting a threshold, areas around the detections are classified as contaminated or uncontaminated sites, respectively, which results in an impact map. Our results show a significant improvement with respect to its quality when redundant image information is used. © 2020 Copernicus GmbH. All rights reserved

Building Development Monitoring in Multitemporal Remotely Sensed Image Pairs with Stochastic Birth-Death Dynamics

International audienceIn this paper we introduce a new probabilistic method which integrates building extraction with change detection in remotely sensed image pairs. A global optimization process attempts to find the optimal configuration of buildings, considering the observed data, prior knowledge, and interactions between the neighboring building parts. We present methodological contributions in three key issues: (1) We implement a novel object-change modeling approach based on Multitemporal Marked Point Processes, which simultaneously exploits low level change information between the time layers and object level building description to recognize and separate changed and unaltered buildings. (2) To answering the challenges of data heterogeneity in aerial and satellite image repositories, we construct a flexible hierarchical framework which can create various building appearance models from different elementary feature based modules. (3) To simultaneously ensure the convergence, optimality and computation complexity constraints raised by the increased data quantity, we adopt the quick Multiple Birth and Death optimization technique for change detection purposes, and propose a novel non-uniform stochastic object birth process, which generates relevant objects with higher probability based on low-level image features

Continuous Modeling of 3D Building Rooftops From Airborne LIDAR and Imagery

In recent years, a number of mega-cities have provided 3D photorealistic virtual models to support the decisions making process for maintaining the cities' infrastructure and environment more effectively. 3D virtual city models are static snap-shots of the environment and represent the status quo at the time of their data acquisition. However, cities are dynamic system that continuously change over time. Accordingly, their virtual representation need to be regularly updated in a timely manner to allow for accurate analysis and simulated results that decisions are based upon. The concept of "continuous city modeling" is to progressively reconstruct city models by accommodating their changes recognized in spatio-temporal domain, while preserving unchanged structures. However, developing a universal intelligent machine enabling continuous modeling still remains a challenging task. Therefore, this thesis proposes a novel research framework for continuously reconstructing 3D building rooftops using multi-sensor data. For achieving this goal, we first proposes a 3D building rooftop modeling method using airborne LiDAR data. The main focus is on the implementation of an implicit regularization method which impose a data-driven building regularity to noisy boundaries of roof planes for reconstructing 3D building rooftop models. The implicit regularization process is implemented in the framework of Minimum Description Length (MDL) combined with Hypothesize and Test (HAT). Secondly, we propose a context-based geometric hashing method to align newly acquired image data with existing building models. The novelty is the use of context features to achieve robust and accurate matching results. Thirdly, the existing building models are refined by newly proposed sequential fusion method. The main advantage of the proposed method is its ability to progressively refine modeling errors frequently observed in LiDAR-driven building models. The refinement process is conducted in the framework of MDL combined with HAT. Markov Chain Monte Carlo (MDMC) coupled with Simulated Annealing (SA) is employed to perform a global optimization. The results demonstrates that the proposed continuous rooftop modeling methods show a promising aspects to support various critical decisions by not only reconstructing 3D rooftop models accurately, but also by updating the models using multi-sensor data

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Marked point processes for the automatic detection of bomb craters in aerial wartime images

Many countries were the target of air strikes during the Second World War. The aftermath of such attacks is felt until today, as numerous unexploded bombs or duds still exist in the ground. Typically, such areas are documented in so-called impact maps, which are based on detected bomb craters. This paper proposes a stochastic approach to automatically detect bomb craters in aerial wartime images that were taken during World War II. In this work, one aspect we investigate is the type of object model for the crater: we compare circles with ellipses. The respective models are embedded in the probabilistic framework of marked point processes. By means of stochastic sampling the most likely configuration of objects within the scene is determined. Each configuration is evaluated using an energy function which describes the conformity with a predefined model. High gradient magnitudes along the border of the object are favoured and overlapping objects are penalized. In addition, a term that requires the grey values inside the object to be homogeneous is investigated. Reversible Jump Markov Chain Monte Carlo sampling in combination with simulated annealing provides the global optimum of the energy function. Afterwards, a probability map is generated from the automatic detections via kernel density estimation. By setting a threshold, areas around the detections are classified as contaminated or uncontaminated sites, respectively, which results in an impact map. Our results, based on 22 aerial wartime images, show the general potential of the method for the automated detection of bomb craters and the subsequent automatic generation of an impact map. © Authors 2019