Advances in Multi-Sensor Data Fusion: Algorithms and Applications

With the development of satellite and remote sensing techniques, more and more image data from airborne/satellite sensors have become available. Multi-sensor image fusion seeks to combine information from different images to obtain more inferences than can be derived from a single sensor. In image-based application fields, image fusion has emerged as a promising research area since the end of the last century. The paper presents an overview of recent advances in multi-sensor satellite image fusion. Firstly, the most popular existing fusion algorithms are introduced, with emphasis on their recent improvements. Advances in main applications fields in remote sensing, including object identification, classification, change detection and maneuvering targets tracking, are described. Both advantages and limitations of those applications are then discussed. Recommendations are addressed, including: (1) Improvements of fusion algorithms; (2) Development of “algorithm fusion” methods; (3) Establishment of an automatic quality assessment scheme

Mapping of shifting tidal estuaries to support inshore rescue

Across the world, many coastal tidal regions are unsafe to navigate due to shifting mud and sand pushed by water currents. Ability to regularly map the current location of a channel will aid safe passage for commercial, leisure and rescue craft. This work investigates the use of synthetic aperture radar data derived from satellites to provide accurate mapping of moving channels in coastal regions. As images must be collected at low tide, data availability is assessed considering the relationship between the orbital motion of the satellites and the tides. Change detection methods are applied to suitable images to map changes in the location of navigable channels. Pixels that undergo similar changes over time (e.g. from water covered to exposed sand) are grouped together by examining the principal component of the covariance matrix, for a vector composed of pixel values from the same location at different times. The Solway Firth in Great Britain is selected as a trial site as it is exposed to some of Europe's fastest tidal movements and ranges, and hence is one of Great Britain's most treacherous stretches of coastline

Combining Multiple Algorithms for Road Network Tracking from Multiple Source Remotely Sensed Imagery: a Practical System and Performance Evaluation

In light of the increasing availability of commercial high-resolution imaging sensors, automatic interpretation tools are needed to extract road features. Currently, many approaches for road extraction are available, but it is acknowledged that there is no single method that would be successful in extracting all types of roads from any remotely sensed imagery. In this paper, a novel classification of roads is proposed, based on both the roads' geometrical, radiometric properties and the characteristics of the sensors. Subsequently, a general road tracking framework is proposed, and one or more suitable road trackers are designed or combined for each type of roads. Extensive experiments are performed to extract roads from aerial/satellite imagery, and the results show that a combination strategy can automatically extract more than 60% of the total roads from very high resolution imagery such as QuickBird and DMC images, with a time-saving of approximately 20%, and acceptable spatial accuracy. It is proven that a combination of multiple algorithms is more reliable, more efficient and more robust for extracting road networks from multiple-source remotely sensed imagery than the individual algorithms

Generación de datos georreferenciados de muy alta resolución a partir de imágenes de GeoEye-1.

En 2008 y con la cooperación del Departamento de Defensa de los Estados Unidos, se lanzó un nuevo satélite comercial de muy alta resolución (VHR), llamado GeoEye-1, que actualmente es el satélite comercial con mayor resolución geométrica, tanto en pancromática (PAN) (0,41 m) como en multiespectral (MS) (1,65 m). El objetivo principal de este trabajo es estudiar el nivel de precisión alcanzable, en condiciones operacionales, en orotoimágenes generadas a partir de imágenes PAN y MS del nuevo satélite VHR GeoEye-1. Para ello se realiza un estudio estadístico, basado en cinco repeticiones, que consta de dos fases, (i) fase de orientación del sensor, donde se estudiará la influencia del modelo de sensor, el número y la distribución de los puntos de control (GCPs) y (ii) fase de ortorectificación, donde se estudirá la precisión que se obtiene al crear la ortoimagen con la participación de diferentes modelos digitales de elevaciones (DEMs). Para llevar a cabo el estudio se adquirió una imagen de archivo GeoEye-1 (PAN + MS), capturada el 29 de septiembre de 2010, con una ángulo off-nadir de unos 20º. Tanto para la imagen PAN como para la MS, el modelo de sensor con el que se llega a mejores resultados en la fase de orientación, es RPC0, siendo su comportamiento prácticamente independiente al número y distribución de GCPs. En la fase de ortorectificación, la generación de ortoimágenes PAN, requiere un DEM con mayor precisión que para la generación de ortoimágenes MS

Détection des bâtiments à partir des images multispectrales à très haute résolution spatiale par la transformation Hit-or-Miss

Résumé : La détection des bâtiments dans les images à très haute résolution spatiale (THRS) a plusieurs applications pratiques et représente un domaine de recherche scientifique intensive ces dernières années. Elle fait face à la complexité du milieu urbain et aux spécificités des images provenant des différents capteurs. La performance des méthodes existantes pour l’extraction des bâtiments n’est pas encore suffisante pour qu’elles soient généralisées à grande échelle (différents types de tissus urbains et capteurs). Les opérateurs morphologiques se sont montrés efficaces pour la détection des bâtiments dans les images panchromatiques (images en niveaux de gris) à très haute résolution spectrale (THRS). L’information spectrale issue des images multispectrales est jugée nécessaire pour l’amélioration de leur performance. L’extension des opérateurs morphologiques pour les images multispectrales exige l’adoption d’une stratégie qui permet le traitement des pixels sous forme de vecteurs, dont les composantes sont les valeurs dans les différentes bandes spectrales. Ce travail de recherche vise l’application de la transformation morphologique dite Hit-or-Miss (HMT) à des images multispectrales à THRS, afin de détecter des bâtiments. Pour répondre à la problématique de l’extension des opérateurs morphologiques pour les images multispectrales, nous proposons deux solutions. Comme une première solution nous avons généré des images en niveaux de gris à partir les bandes multispectrales. Dans ces nouvelles images les bâtiments potentiels sont rehaussés par rapport à l’arrière-plan. La HMT en niveaux de gris est alors appliquée à ces images afin de détecter les bâtiments. Pour rehausser les bâtiments nous avons proposé un nouvel indice, que nous avons appelé Spectral Similarity Ratio (SSR). Pour éviter de définir des configurations, des ensembles d’éléments structurants (ES), nécessaires pour l’application de la HMT, au préalable, nous avons utilisé l’érosion et la dilatation floues et poursuivi la réponse des pixels aux différentes valeurs des ES. La méthode est testée sur des extraits d’images représentant des quartiers de type résidentiel. Le taux moyen de reconnaissance obtenu pour les deux capteurs Ikonos et GeoEye est de 85 % et de 80 %, respectivement. Le taux moyen de bonne identification, quant à lui, est de 85 % et 84 % pour les images Ikonos et GeoEye, respectivement. Après certaines améliorations, la méthode a été appliquée sur des larges scènes Ikonos et WorldView-2, couvrant différents tissus urbains. Le taux moyen des bâtiments reconnus est de 82 %. Pour sa part, le taux de bonne identification est de 81 %. Dans la deuxième solution, nous adoptons une stratégie vectorielle pour appliquer la HMT directement sur les images multispectrales. La taille des ES de cette transformation morphologique est définie en utilisant la transformation dite chapeau haut-de-forme par reconstruction. Une étape de post-traitement inclut le filtrage de la végétation par l’indice de la végétation NDVI et la validation de la localisation des bâtiments par l’information d’ombre. La méthode est appliquée sur un espace urbain de type résidentiel. Des extraits d’images provenant des capteurs satellitaires Ikonos, GeoEye et WorldView 2 ont été traités. Le taux des bâtiments reconnus est relativement élevé pour tous les extraits - entre 85 % et 97 %. Le taux de bonne identification démontre des résultats entre 74 % et 88 %. Les résultats obtenus nous permettent de conclure que les objectifs de ce travail de recherche, à savoir, la proposition d’une technique pour l’estimation de la similarité spectrale entre les pixels formant le toit d’un bâtiment, l’intégration de l’information multispectrale dans la HMT dans le but de détecter les bâtiments, et la proposition d’une technique qui permet la définition semi-automatique des configurations bâtiment/voisinage dans les images multispectrales, ont été atteints. // Abstract : Detection of buildings in very high spatial resolution images (THRS) has various practical applications and is recently a subject of intensive scientific research. It faces the complexity of the urban environment and the variety of image characteristics depending on the type of the sensor. The performance of existing building extraction methods is not yet sufficient to be generalized to a large scale (different urban patterns and sensors). Morphological operators have been proven effective for the detection of buildings in panchromatic (greyscale) very high spectral resolution (VHSR) images. The spectral information of multispectral images is jugged efficient to improve the results of the detection. The extension of morphological operators to multispectral images is not straightforward. As pixels of multispectral images are pixels vectors the components of which are the intensity values in the different bands, a strategy to order vectors must be adopted. This research thesis focuses on the application of the morphological transformation called Hit-or-Miss (HMT) on multispectral VHSR images in order to detect buildings. To address the issue of the extension of morphological operators to multispectral images we have proposed two solutions. The first one employs generation of greyscale images from multispectral bands, where potential buildings are enhanced. The grayscale HMT is then applied to these images in order to detect buildings. To enhance potential building locations we have proposed the use of Spectral Similarity Ratio (SSR). To avoid the need to set multiple configurations of structuring elements (SE) necessary for the implementation of the HMT, we have used fuzzy erosion and fuzzy dilation and examined the pixel response to different values of SE. The method has been tested on image subsets taken over residential areas. The average rate of recognition for the two sensors, Ikonos and GeoEye, is 85% and 80%, respectively. The average rate of correct identification is 85% and 84%, for Ikonos and GeoEye subsets, respectively. Having made some improvements, we then applied the method to large scenes from Ikonos and WorldView-2 images covering different urban patterns. The average rate of recognized buildings is 82%. The rate of correct identification is 81%. As a second solution, we have proposed a new vector based strategy which allows the multispectral information to be integrated into the percent occupancy HMT (POHMT). Thus, the POHMT has been directly applied on multispectral images. The parameters for the POHMT have been defined using the morphological transformation dubbed top hat by reconstruction. A post-processing step included filtering the vegetation and validating building locations by proximity to shadow. The method has been applied to urban residential areas. Image subsets from Ikonos, GeoEye and WorldView2 have been processed. The rate of recognized buildings is relatively high for all subsets - between 85% and 97%. The rate of correct identification is between 74 % and 88 %. The results allow us to conclude that the objectives of this research, namely, suggesting a technique for estimating the spectral similarity between the pixels forming the roof of a building, the integration of multispectral information in the HMT in order to detect buildings and the proposition of a semiautomatic technique for the definition of the configurations building/neighbourhood in multispectral images, have been achieved

Robust Modular Feature-Based Terrain-Aided Visual Navigation and Mapping

The visual feature-based Terrain-Aided Navigation (TAN) system presented in this thesis addresses the problem of constraining inertial drift introduced into the location estimate of Unmanned Aerial Vehicles (UAVs) in GPS-denied environment. The presented TAN system utilises salient visual features representing semantic or human-interpretable objects (roads, forest and water boundaries) from onboard aerial imagery and associates them to a database of reference features created a-priori, through application of the same feature detection algorithms to satellite imagery. Correlation of the detected features with the reference features via a series of the robust data association steps allows a localisation solution to be achieved with a finite absolute bound precision defined by the certainty of the reference dataset. The feature-based Visual Navigation System (VNS) presented in this thesis was originally developed for a navigation application using simulated multi-year satellite image datasets. The extension of the system application into the mapping domain, in turn, has been based on the real (not simulated) flight data and imagery. In the mapping study the full potential of the system, being a versatile tool for enhancing the accuracy of the information derived from the aerial imagery has been demonstrated. Not only have the visual features, such as road networks, shorelines and water bodies, been used to obtain a position ’fix’, they have also been used in reverse for accurate mapping of vehicles detected on the roads into an inertial space with improved precision. Combined correction of the geo-coding errors and improved aircraft localisation formed a robust solution to the defense mapping application. A system of the proposed design will provide a complete independent navigation solution to an autonomous UAV and additionally give it object tracking capability