Hyperspectral Imaging for Real-Time Unmanned Aerial Vehicle Maritime Target Detection

The hyperspectral cameras use has been increasing over the past years, driven by the exponential growth of the computational systems power. The capability of acquiring multiple spectre wavelengths benefits the increase of the hyperspectral systems range of applications. However, until now, most hyperspectral systems are used in posprocessing and do not allow to take full advantage of the system capabilities. There is a recent trend to be able to use hyperspectral systems in real-time. Given the recent problems in European Union borders due to irregular immigration and drug smuggling, there is the need to develop novel autonomous surveillance systems that can work on these scenarios. This thesis addresses the scenario of using hyperspectral imaging systems for maritime target detection using unmanned aerial vehicles. Specifically, by working in the creation of a hyperspectral real-time data processing system pipeline. In our work, we develop a boresight calibration method that allows to calibrate the position of the navigation sensor related to the camera imaging sensor, and improve substantially the accuracy of the target geo-reference. We also develop a novel method of distinguish targets (boats) from their dominant background. With this application our system is able to only select relevant information to send to a remote station on the ground, thus making it suitable to be installed in an actual unmanned maritime surveillance system.A utilização de câmaras hiperespectrais tem vindo a aumentar nos últimos anos, motivada pelo crescimento exponencial da capacidade de processamento dos mais recentes sistemas computacionais. A sua aptidão para observar múltiplos comprimentos de onda beneficia aplicações em diferentes campos de atividade. No entanto, a maior parte das aplicações com câmaras hiperespectrais são realizadas em pós-processamento, não aproveitando totalmente as capacidades destes sistemas. Existe uma necessidade emergente de detetar mais características sobre o cenário que está a ser observado, incentivando o desenvolvimento de sistemas hiperespectrais capazes de adquirir e processar informação em tempo-real. Face aos mais recentes problemas de emigração e contrabando ilegal na União Europeia, surge a necessidade da realização de vigilância autónoma capaz de adquirir o máximo de informação possível sobre os meios envolventes presentes num dado percurso. E neste contexto que se insere a dissertação que visa a criação é implementação de um sistema hiperespectral em tempo-real. Para construir o sistema, foi necessário dividir o problema em diferentes etapas. Iniciou-se por um estudo detalhado dos sistemas hiperespectrais, desenvolvendo um método de calibração dos ângulos de boresight, que permitiu calibrar a relação entre o sistema de posicionamento e navegação da câmara hiperespectral e o sensor imagem. Esta calibração, permite numa fase posterior geo-referenciar os alvos com maior precisão. Posteriormente, foi criada uma pipeline de processamento, que permite analisar os espectros obtidos, distinguindo os alvos do cenário onde estão inseridos. Após a deteção dos alvos, procede-se `a sua geo-referenciação, de forma a obter as coordenadas UTM do alvo. Toda a informação obtida sobre o alvo e a sua posição é enviada para uma estacão em terra, de forma a ser validada por um humano. Para tal, foi também desenvolvida a metodologia de envio, para selecionar a informação a enviar apenas à mais relevante

Using QR Factorization for Real-Time Anomaly Detection of Hyperspectral Images

Anomaly detection has been used successfully on hyperspectral images for over a decade. However, there is an ever increasing need for real-time anomaly detectors. Historically, anomaly detection methods have focused on analysis after the entire image has been collected. As useful as post-collection anomaly detection is, there is a great advantage to detecting an anomaly as it is being collected. This research is focused on speeding up the process of detection for a pre-existing method, Linear RX, which is a variation on the traditional Reed-Xiaoli detector. By speeding up the process of detection, it is possible to create a real-time anomaly detector. The window covariance matrix is our main area focus for speed improvement. Several methods were investigated, including QR factorization and tracking the change in the window covariance matrix as it moves through the image. Finally, performance comparisons are made to the original Linear RX detector

A comprehensive approach for the efficient acquisition and processing of hyperspectral images and sequence

Programa Oficial de Doctorado en Computación. 5009P01[Abstract] Despite the scientific and technological developments achieved during the last two decades in the hyperspectral field, some methodological, operational and conceptual issues have restricted the progress, promotion and popular dissemination of this technology. These shortcomings include the specialized knowledge required for the acquisition of hyperspectral images, the shortage of publicly accessible hyperspectral image repositories with reliable ground truth images or the lack of methodologies that allow for the adaptation of algorithms to particular user or application processing needs. The work presented here has the objective of contributing to the hyperspectral field with procedures for the automatic acquisition of hyperspectral scenes, including the hardware adaptation of our own imagers and the development of methods for the calibration and correction of the hyperspectral datacubes, the creation of a publicly available hyperspectral repository of well categorized and labeled images and the design and implementation of novel computational intelligence based processing techniques that solve typical issues related to the segmentation and denoising of hyperspectral images as well as sequences of them taking into account their temporal evolution.[Resumen] A pesar de los desarrollos tecnológicos y científicos logrados en el campo hiperespectral durante las dos últimas décadas, alg\mas limitaciones de tipo metodológico, operacional y conceptual han restringido el progreso, difusión y popularización de esta tecnología, entre ellas, el conocimiento especializado requerido en la adquisición de imágenes hiperespectrales, la carencia de repositorios de imágenes hiperespectrales con etiquetados fiables y de acceso público o la falta de metodologías que posibiliten la adaptación de algoritmos a usuarios o necesidades de procesamiento concretas. Este trabajo doctoral tiene el objetivo de contribuir al campo hiperespectral con procedimientos para la adquisición automática de escenas hiperespectrales, incluyendo la adaptación hardware de cámaras hiperespectrales propias y el desarrollo de métodos para la calibración y corrección de cubos de datos hiperespectrales; la creación de un repositorio hiperespectral de acceso público con imágenes categorizadas y con verdades de terreno fiables; y el diseño e implementación de técnicas de procesamiento basadas en inteligencia computacional para la resolución de problemas típicamente relacionados con las tareas de segmentación y eliminación de ruido en imágenes estáticas y secuencias de imágenes hiperespectrales teniendo en consideración su evolución temporal.[Resumo] A pesar dos desenvolvementos tecnolóxicos e científicos logrados no campo hiperespectral durante as dúas últimas décadas, algunhas lirrútacións de tipo metodolóxico¡ operacional e conceptual restrinxiron o progreso) difusión e popularización desta tecnoloxía, entre elas, o coñecemento especializado requirido na adquisición de imaxes hiperespectrales¡ a carencia de repositorios de irnaxes hiperespectrales con etiquetaxes fiables e de acceso público ou a falta de metodoloxías que posibiliten a adaptación de algoritmos a usuarios ou necesidades de procesamento concretas. Este traballo doutoral ten o obxectívo de contribuir ao campo hiperespectral con procedementos para a adquisición automática de eicenas hiperespectrais, incluíndo a adaptación hardware de cámaras hiperespectrales propias e o desenvolvemento de métodos para a calibración e corrección de cubos de datos hiperespectrais; a creación dun repositorio hiperespectral de acceso público con imaxes categorizadas e con verdades de terreo fiables; e o deseño e implementación de técnicas de procesamento baseadas en intelixencia computacional para a resolución de problemas tipicamente relacionado~ coas tarefas de segmentación e eliminación de ruído en imaxes estáticas e secuencias de imaxes hiperespectrai~ tendo en consideración a súa evolución temporal

Reconstruction Error and Principal Component Based Anomaly Detection in Hyperspectral imagery

The rapid expansion of remote sensing and information collection capabilities demands methods to highlight interesting or anomalous patterns within an overabundance of data. This research addresses this issue for hyperspectral imagery (HSI). Two new reconstruction based HSI anomaly detectors are outlined: one using principal component analysis (PCA), and the other a form of non-linear PCA called logistic principal component analysis. Two very effective, yet relatively simple, modifications to the autonomous global anomaly detector are also presented, improving algorithm performance and enabling receiver operating characteristic analysis. A novel technique for HSI anomaly detection dubbed multiple PCA is introduced and found to perform as well or better than existing detectors on HYDICE data while using only linear deterministic methods. Finally, a response surface based optimization is performed on algorithm parameters such as to affect consistent desired algorithm performance

Assessing the impact of illumination on UAV pushbroom hyperspectral imagery collected under various cloud cover conditions

The recent development of small form-factor (<6 kg), full range (400–2500 nm) pushbroom hyperspectral imaging systems (HSI) for unmanned aerial vehicles (UAV) poses a new range of opportunities for passive remote sensing applications. The flexible deployment of these UAV-HSI systems have the potential to expand the data acquisition window to acceptable (though non-ideal) atmospheric conditions. This is an important consideration for time-sensitive applications (e.g. phenology) in areas with persistent cloud cover. Since the majority of UAV studies have focused on applications with ideal illumination conditions (e.g. minimal or non-cloud cover), little is known to what extent UAV-HSI data are affected by changes in illumination conditions due to variable cloud cover. In this study, we acquired UAV pushbroom HSI (400–2500 nm) over three consecutive days with various illumination conditions (i.e. cloud cover), which were complemented with downwelling irradiance data to characterize illumination conditions and in-situ and laboratory reference panel measurements across a range of reflectivity (i.e. 2%, 10%, 18% and 50%) used to evaluate reflectance products. Using these data we address four fundamental aspects for UAV-HSI acquired under various conditions ranging from high (624.6 ± 16.63 W·m2) to low (2.5 ± 0.9 W·m2) direct irradiance: atmospheric compensation, signal-to-noise ratio (SNR), spectral vegetation indices and endmembers extraction. For instance, two atmospheric compensation methods were applied, a radiative transfer model suitable for high direct irradiance, and an Empirical Line Model (ELM) for diffuse irradiance conditions. SNR results for two distinctive vegetation classes (i.e. tree canopy vs herbaceous vegetation) reveal wavelength dependent attenuation by cloud cover, with higher SNR under high direct irradiance for canopy vegetation. Spectral vegetation index (SVIs) results revealed high variability and index dependent effects. For example, NDVI had significant differences (p < 0.05) across illumination conditions, while NDWI appeared insensitive at the canopy level. Finally, often neglected diffuse illumination conditions may be beneficial for revealing spectral features in vegetation that are obscured by the predominantly non-Lambertian reflectance encountered under high direct illumination. To our knowledge, our study is the first to use a full range pushbroom UAV sensor (400–2500 nm) for assessing illumination effects on the aforementioned variables. Our findings pave the way for understanding the advantages and limitations of ultra-high spatial resolution full range high fidelity UAV-HSI for ecological and other applications

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

Modern hyperspectral imaging systems produce huge datasets potentially conveying a great abundance of information; such a resource, however, poses many challenges in the analysis and interpretation of these data. Deep learning approaches certainly offer a great variety of opportunities for solving classical imaging tasks and also for approaching new stimulating problems in the spatial–spectral domain. This is fundamental in the driving sector of Remote Sensing where hyperspectral technology was born and has mostly developed, but it is perhaps even more true in the multitude of current and evolving application sectors that involve these imaging technologies. The present review develops on two fronts: on the one hand, it is aimed at domain professionals who want to have an updated overview on how hyperspectral acquisition techniques can combine with deep learning architectures to solve specific tasks in different application fields. On the other hand, we want to target the machine learning and computer vision experts by giving them a picture of how deep learning technologies are applied to hyperspectral data from a multidisciplinary perspective. The presence of these two viewpoints and the inclusion of application fields other than Remote Sensing are the original contributions of this review, which also highlights some potentialities and critical issues related to the observed development trends

Geo-correction of high-resolution imagery using fast template matching on a GPU in emergency mapping contexts

The increasing availability of satellite imagery acquired from existing and new sensors allow a wide variety of new applications that depend on the use of diverse spectral and spatial resolution data sets. One of the pre-conditions for the use of hybrid image data sets is a consistent geo-correction capacity. We demonstrate how a novel fast template matching approach implemented on a Graphics Processing Unit (GPU) allows us to accurately and rapidly geo-correct imagery in an automated way. The key difference with existing geo-correction approaches, which do not use a GPU, is the possibility to match large source image segments (8192 by 8192 pixels) with relatively large templates (512 by 512 pixels). Our approach is sufficiently robust to allow for the use of various reference data sources. The need for accelerated processing is relevant in our application context, which relates to mapping activities in the European Copernicus emergency management service. Our new method is demonstrated over an area North-West of Valencia (Spain) for a large forest fire event in July 2012. We use DEIMOS-1 and RapidEye imagery for the delineation of burnt fire scar extent. Automated geo-correction of each full resolution image sets takes approximately 1 minute. The reference templates are taken from the TerraColor data set and the Spanish national ortho-imagery data base, through the use of dedicate web map services (WMS). Geo-correction results are compared to the vector sets derived in the related Copernicus emergency service activation request.JRC.G.2-Global security and crisis managemen

Hyperspectral Imaging from Ground Based Mobile Platforms and Applications in Precision Agriculture

This thesis focuses on the use of line scanning hyperspectral sensors on mobile ground based platforms and applying them to agricultural applications. First this work deals with the geometric and radiometric calibration and correction of acquired hyperspectral data. When operating at low altitudes, changing lighting conditions are common and inevitable, complicating the retrieval of a surface's reflectance, which is solely a function of its physical structure and chemical composition. Therefore, this thesis contributes the evaluation of an approach to compensate for changes in illumination and obtain reflectance that is less labour intensive than traditional empirical methods. Convenient field protocols are produced that only require a representative set of illumination and reflectance spectral samples. In addition, a method for determining a line scanning camera's rigid 6 degree of freedom (DOF) offset and uncertainty with respect to a navigation system is developed, enabling accurate georegistration and sensor fusion. The thesis then applies the data captured from the platform to two different agricultural applications. The first is a self-supervised weed detection framework that allows training of a per-pixel classifier using hyperspectral data without manual labelling. The experiments support the effectiveness of the framework, rivalling classifiers trained on hand labelled training data. Then the thesis demonstrates the mapping of mango maturity using hyperspectral data on an orchard wide scale using efficient image scanning techniques, which is a world first result. A novel classification, regression and mapping pipeline is proposed to generate per tree mango maturity averages. The results confirm that maturity prediction in mango orchards is possible in natural daylight using a hyperspectral camera, despite complex micro-illumination-climates under the canopy

A novel spectral-spatial singular spectrum analysis technique for near real-time in-situ feature extraction in hyperspectral imaging.

As a cutting-edge technique for denoising and feature extraction, singular spectrum analysis (SSA) has been applied successfully for feature mining in hyperspectral images (HSI). However, when applying SSA for in situ feature extraction in HSI, conventional pixel-based 1-D SSA fails to produce satisfactory results, while the band-image-based 2D-SSA is also infeasible especially for the popularly used line-scan mode. To tackle these challenges, in this article, a novel 1.5D-SSA approach is proposed for in situ spectral-spatial feature extraction in HSI, where pixels from a small window are used as spatial information. For each sequentially acquired pixel, similar pixels are located from a window centered at the pixel to form an extended trajectory matrix for feature extraction. Classification results on two well-known benchmark HSI datasets and an actual urban scene dataset have demonstrated that the proposed 1.5D-SSA achieves the superior performance compared with several state-of-the-art spectral and spatial methods. In addition, the near real-time implementation in aligning to the HSI acquisition process can meet the requirement of online image analysis for more efficient feature extraction than the conventional offline workflow

Remote Sensing

This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas