Application of Multi-Sensor Fusion Technology in Target Detection and Recognition

Application of multi-sensor fusion technology has drawn a lot of industrial and academic interest in recent years. The multi-sensor fusion methods are widely used in many applications, such as autonomous systems, remote sensing, video surveillance, and the military. These methods can obtain the complementary properties of targets by considering multiple sensors. On the other hand, they can achieve a detailed environment description and accurate detection of interest targets based on the information from different sensors.This book collects novel developments in the field of multi-sensor, multi-source, and multi-process information fusion. Articles are expected to emphasize one or more of the three facets: architectures, algorithms, and applications. Published papers dealing with fundamental theoretical analyses, as well as those demonstrating their application to real-world problems

Cloud removal from optical remote sensing images

Optical remote sensing images used for Earth surface observations are constantly contaminated by cloud cover. Clouds dynamically affect the applications of optical data and increase the difficulty of image analysis. Therefore, cloud is considered as one of the sources of noise in optical image data, and its detection and removal need to be operated as a pre-processing step in most remote sensing image processing applications. This thesis investigates the current cloud detection and removal algorithms and develops three new cloud removal methods to improve the accuracy of the results. A thin cloud removal method based on signal transmission principles and spectral mixture analysis (ST-SMA) for pixel correction is developed in the first contribution. This method considers not only the additive reflectance from the clouds but also the energy absorption when solar radiation passes through them. Data correction is achieved by subtracting the product of the cloud endmember signature and the cloud abundance and rescaling according to the cloud thickness. The proposed method has no requirement for meteorological data and does not rely on reference images. The experimental results indicate that the proposed approach is able to perform effective removal of thin clouds in different scenarios. In the second study, an effective cloud removal method is proposed by taking advantage of the noise-adjusted principal components transform (CR-NAPCT). It is found that the signal-to-noise ratio (S/N) of cloud data is higher than data without cloud contamination, when spatial correlation is considered and are shown in the first NAPCT component (NAPC1) in the NAPCT data. An inverse transformation with a modified first component is then applied to generate the cloud free image. The effectiveness of the proposed method is assessed by performing experiments on simulated and real data to compare the quantitative and qualitative performance of the proposed approach. The third study of this thesis deals with both cloud and cloud shadow problems with the aid of an auxiliary image in a clear sky condition. A new cloud removal approach called multitemporal dictionary learning (MDL) is proposed. Dictionaries of the cloudy areas (target data) and the cloud free areas (reference data) are learned separately in the spectral domain. An online dictionary learning method is then applied to obtain the two dictionaries in this method. The removal process is conducted by using the coefficients from the reference image and the dictionary learned from the target image. This method is able to recover the data contaminated by thin and thick clouds or cloud shadows. The experimental results show that the MDL method is effective from both quantitative and qualitative viewpoints

On the generation of high dynamic range images: theory and practice from a statistical perspective

This dissertation studies the problem of high dynamic range (HDR) image generation from a statistical perspective. A thorough analysis of the camera acquisition process leads to a simplified yet realistic statistical model describing raw pixel values. The analysis and methods then proposed are based on this model. First, the theoretical performance bound of the problem is computed for the static case, where the acquisition conditions are controlled. Furthermore, a new method is proposed that, unlike previous methods, improves the reconstructed HDR image by taking into account the information carried by saturated samples. From a more practical perspective, two methods are proposed to generate HDR images in the more realistic and complex case where both objects and camera may exhibit motion. The first one is a multi-image, patch-based method, that simultaneously estimates and denoises the HDR image. The other is a single image approach that makes use of a general restoration method to generate the HDR image. This general restoration method, applicable to a wide range of problems, constitutes the last contribution of this dissertation

Book of short Abstracts of the 11th International Symposium on Digital Earth

The Booklet is a collection of accepted short abstracts of the ISDE11 Symposium

Exploring the Internal Statistics: Single Image Super-Resolution, Completion and Captioning

Image enhancement has drawn increasingly attention in improving image quality or interpretability. It aims to modify images to achieve a better perception for human visual system or a more suitable representation for further analysis in a variety of applications such as medical imaging, remote sensing, and video surveillance. Based on different attributes of the given input images, enhancement tasks vary, e.g., noise removal, deblurring, resolution enhancement, prediction of missing pixels, etc. The latter two are usually referred to as image super-resolution and image inpainting (or completion). Image super-resolution and completion are numerically ill-posed problems. Multi-frame-based approaches make use of the presence of aliasing in multiple frames of the same scene. For cases where only one input image is available, it is extremely challenging to estimate the unknown pixel values. In this dissertation, we target at single image super-resolution and completion by exploring the internal statistics within the input image and across scales. An internal gradient similarity-based single image super-resolution algorithm is first presented. Then we demonstrate that the proposed framework could be naturally extended to accomplish super-resolution and completion simultaneously. Afterwards, a hybrid learning-based single image super-resolution approach is proposed to benefit from both external and internal statistics. This framework hinges on image-level hallucination from externally learned regression models as well as gradient level pyramid self-awareness for edges and textures refinement. The framework is then employed to break the resolution limitation of the passive microwave imagery and to boost the tracking accuracy of the sea ice movements. To extend our research to the quality enhancement of the depth maps, a novel system is presented to handle circumstances where only one pair of registered low-resolution intensity and depth images are available. High quality RGB and depth images are generated after the system. Extensive experimental results have demonstrated the effectiveness of all the proposed frameworks both quantitatively and qualitatively. Different from image super-resolution and completion which belong to low-level vision research, image captioning is a high-level vision task related to the semantic understanding of an input image. It is a natural task for human beings. However, image captioning remains challenging from a computer vision point of view especially due to the fact that the task itself is ambiguous. In principle, descriptions of an image can talk about any visual aspects in it varying from object attributes to scene features, or even refer to objects that are not depicted and the hidden interaction or connection that requires common sense knowledge to analyze. Therefore, learning-based image captioning is in general a data-driven task, which relies on the training dataset. Descriptions in the majority of the existing image-sentence datasets are generated by humans under specific instructions. Real-world sentence data is rarely directly utilized for training since it is sometimes noisy and unbalanced, which makes it ‘imperfect’ for the training of the image captioning task. In this dissertation, we present a novel image captioning framework to deal with the uncontrolled image-sentence dataset where descriptions could be strongly or weakly correlated to the image content and in arbitrary lengths. A self-guiding learning process is proposed to fully reveal the internal statistics of the training dataset and to look into the learning process in a global way and generate descriptions that are syntactically correct and semantically sound

Sur la génération d'images à grande gamme dynamique. Théorie et pratique : une perspective statistique

This dissertation studies the problem of high dynamic range (HDR) image generation from a statistical perspective. A thorough analysis of the camera acquisition process leads to a simplified yet realistic statistical model describing raw pixel values. The analysis and methods then proposed are based on this model. First, the theoretical performance bound of the problem is computed for the static case, where the acquisition conditions are controlled. Furthermore, a new method is proposed that, unlike previous methods, improves the reconstructed HDR image by taking into account the information carried by saturated samples. From a more practical perspective, two methods are proposed to generate HDR images in the more realistic and complex case where both objects and camera may exhibit motion. The first one is a multi-image, patch-based method, that simultaneously estimates and denoises the HDR image. The other is a single image approach that makes use of a general restoration method to generate the HDR image. This general restoration method, applicable to a wide range of problems, constitutes the last contribution of this dissertation.Cette thèse porte sur le problème de la génération d'images à grande gamme dynamique (HDR pour l'anglais High Dynamic Range). Une analyse approfondie du processus d'acquisition de la caméra conduit tout d'abord à un modèle statistique simplifié mais réaliste décrivant les valeurs brutes des pixels. Les analyses et méthodes proposées par la suite sont fondées sur ce modèle.Nous posons le problème de l'estimation de l'irradiance comme un problème d'estimation statistique et en calculons la borne de performance. Les performances des estimateurs d'irradiance classiques sont comparées à cette borne. Les résultats obtenus justifient l'introduction d'un nouvel estimateur qui, au contraire des méthodes de la littérature, prend en compte les échantillons saturés.D'un point de vue plus pratique, deux méthodes sont proposées pour générer des images HDR dans le cas plus réaliste et complexe de scènes dynamiques. Nous proposons tout d'abord une méthode multi-image qui utilise des voisinages (patches) pour estimer et débruiter l'image HDR de façon simultanée. Nous proposons également une approche qui repose sur l'acquisition d'une seule image. Cette approche repose sur une méthode générique, par patches, de résolution des problèmes inverses pour génerer l'image HDR. Cette méthode de restauration, d'un point de vue plus général et pour une large gamme d'applications, constitue la dernière contribution de cette thèse

Alligatoring: An investigation into paint failure and loss of image integrity in 19th century oil paintings

"Alligatoring" or "Bitumen cracking" are terms used to describe extreme paint defects found in 19th century oil paintings. This paint failure in the form of severely disfiguring cracking and surface distor- tions often results in a loss of image integrity. This problem has been associated with the use of as- phalt/bitumen paint, with no clear understanding of the materials and mechanisms which contribute to the phenomenon. This thesis investigates this phenomenon from a multi-disciplinary approach that aims to con- tribute knowledge to the study of oil painting suffering from alligatoring. Part 1 focuses on the perception of the problem through a literature survey, and introduces the 19th century painting used in the case study. A thorough review of the literature showed the existing bias created by the perceived connection between appearance and cause, resulting in an association of paint film defects in brown paint with the use of asphalt/bitumen. Because of this, the overall focus of previous research has been the detection of asphalt, considering it in isolation and as the primary factor in the paint defect. This singular view has had significant implications in the study and analyses of paintings with alligatoring. It inhibited wider investigations and overlooked other materials present in the paintings that may be acting in combination or be more predominant in the deterioration mech- anism. The visual and chemical study of the oil painting O Cardeal D. Henrique recebendo a notícia da morte de D. Sebastião, by Marciano Henriques da Silva (1831-1873), painted in Rome in 1861, which exibits extreme alligatoring, offered specific challenges due to its complex and highly disrupted paint layer stratigraphy coupled with the uncertainty introduced by analytical detection limits. For that rea- son, a multi-analytical approach was carried out using Optical Microscopy, micro Raman Spectroscopy, Scanning Electron Microscopy with Energy Dispersive X-ray Spectrometry, X-ray Fluorescence, micro Fourier Transform Infrared Spectroscopy, Fourier Transform Raman Spectroscopy, Attenuated Total Reflection Fourier Transform Infrared Spectroscopy Microscopy, Pyrolysis-Tetramethylammonium Hy- droxide-Gas Chromatography/Mass Spectrometry and Time-of-Flight Secondary Ion Mass Spectrome- try. This research indicates that the deterioration of the painting might be related to the oil binder which has a high degree of oxidation, rich in polar fatty acids and diacids, combined with a lack of the xiv pigments that are capable of stabilizing the paint. The presence of the translucent lead azelate layer above the ground underlines the complexity of the paint layering, materials present, and possible mechanisms for degradation. Part 2 reports on new research carried out using the British colourman Winsor & Newton's 19th- century Archive Database to analyse their production records for “Bitumen” brown for oil tube paints. While there are a substantial number of published recipes in 19th century artist’s manuals for the use of asphalt/bitumen brown, for the first time W&N’s records offer a unique source of detailed infor- mation on the commercially prepared product which differs substantially in ingredients used and method of preparation. The critical analysis of W&N's production records for "Bitumen" revealed that their formulation became standardised in the 1850s and that their product was prepared in two sep- arate steps at two separate locations. Despite standarisation of the ingredients this research revealed that the company still found it necessary to make adjustments for each production run in order to achieve a uniform product. A production record from 1858 was selected and reconstructed, using where possible, histori- cally appropriate materials. W&N’s formulation for bitumen brown oil paint involved a complex mix- ture of ingredients, some of which were other proprietary products sold by the company (these were reconstructed individually using their production records and included drying oil, double mastic var- nish, lead acetate, purple lake and the gelled painter's Medium, megilp). The reconstructions were analysed using thermally assisted methylation with tetrame- thylammonium hydroxide, and pyrolysis comprehensive two-dimensional gas chromatography and compared with the starting material, Trinidad Lake asphalt, to determine how detectable this asphalt is after heat processing in lead treated linseed oil. Results show that asphalt markers identified in the Trinidad Lake asphalt disappear in the first stage of reconstructing W&N's "Bitumen" oil paint. This important finding offers an explanation for the paucity of analytical evidence in previous attempts to identify asphalt/bitumen in paintings where this material was believed to have been used. In addition to clarifying the analytical results obtained from the investigation of the painting, O Cardeal D. Henrique…, the reference samples produced from the W&N reconstruction illustrate the strengths and weaknesses of organic analysis of highly processed complex mixtures."Alligatoring" ou "Bitumen cracking" são termos utilizados para descrever defeitos extremos encon- trados em pinturas a óleo do século XIX. Esta patologia sob a forma de fissuras e distorções severas da superfície pictórica resulta frequentemente numa perda de integridade da imagem. Este problema tem sido associado à utilização de tintas de asfalto/betume, sem no entanto existir uma compreensão clara dos materiais e mecanismos que contribuem para este fenómeno. Esta tese investiga este fenómeno a partir de uma abordagem multidisciplinar que visa contri- buir para o conhecimento de pinturas a óleo afetadas por este defeito extremo. A parte 1 centra-se na perceção que existe sobre este problema através de uma pesquisa bibli- ográfica, e introduz a pintura do século XIX utilizada como caso de estudo. A revisão rigorosa da lite- ratura revelou o preconceito existente criado pela perceção da ligação entre a aparência e a causa, resultando numa associação de defeitos em filmes de tinta castanha com a utilização de tintas de as- falto/betume. Devido a isto, o foco geral de estudos anteriores foi a deteção do asfalto, considerando- o isoladamente e como a principal causa nos defeitos de tinta. Esta visão simplista tem tido implicações significativas no estudo e análise de pinturas com “alligatoring” (efeito de pele de crocodilo), inibindo investigações mais abrangentes e subvalorizando outros materiais presentes nas pinturas que podem estar a atuar em combinação ou ser mais predominantes no mecanismo de deterioração. O estudo visual e químico da pintura a óleo O Cardeal D. Henrique recebendo a notícia da morte de D. Sebastião, de Marciano Henriques da Silva (1831-1873), pintado em Roma em 1861, que exibe “alligatoring” extremo, ofereceu desafios específicos devido à sua estratigrafia complexa e muito alte- rada, aliada à incerteza introduzida pelos limites de deteção analítica. Por essa razão, foi realizada uma abordagem multi-analítica utilizando Microscopia Óptica, Micro-espectroscopia de Raman, Microsco- pia Electrónica de Varrimento com Espectrometria Dispersiva de Raios-X, Micro-espectrometria por Fluorescência de Raios X Dispersiva de Energias, Micro-espectroscopia de Infravermelho com Trans- formada de Fourier, Espectroscopia Raman com Transformada de Fourier, Espectroscopia de Infraver- melho com Transformada de Fourier por Reflexão Total Atenuada acoplada a Microscopia Óptica, Pi- rólise-Cromatografia Gasosa acoplada à Espectrometria de Massa com derivatização usando Hidróxido de Tetrametilamónio e Espectrometria de Massa de iões secundários. Esta investigação indica que a deterioração da pintura pode estar relacionada com o aglutinante de óleo altamente hidrolisado, rico em ácidos gordos polares e diácidos, combinado com a falta de pigmentos capazes de estabilizar a tinta. A presença da camada translúcida de azelato de chumbo sobre a camada de preparação sublinha a complexidade da estratigrafia, dos materiais presentes, e dos possíveis mecanismos de degradação. A Parte 2 descreve a analise dos registos de produção de "Betume" castanho para tintas em tubos de óleo do arquivo e base de dados do século XIX da Winsor & Newton (fabricante de materiais para artistas britânico). Embora exista um número substancial de receitas de tintas castanhas de as- falto/betume publicadas em manuais para artistas do século XIX, os registos da W&N são uma fonte de informação detalhada e única sobre o produto preparado comercialmente que difere substancial- mente nos ingredientes utilizados e no método de preparação. A análise crítica dos registos de produ- ção da W&N para "Betume" revelou que a sua formulação foi normalizada na década de 1850 e que era preparado em duas etapas distintas e em dois locais distintos. Apesar da padronização dos ingre- dientes, esta investigação revelou que a empresa comsiderava necessário fazer ajustes em cada etapa de produção, a fim de obter um produto uniforme. Um registo de produção de 1858 foi selecionado e reconstruído, utilizando, sempre que possí- vel, materiais historicamente apropriados. A formulação da W&N para a tinta castanha de betume a óleo envolveu uma mistura complexa de ingredientes, alguns dos quais eram produtos produzidos e vendidos pela W&N. Estes foram reconstruídos individualmente, utilizando os seus próprios registos de produção, e incluíram óleo de secagem, verniz mástique duplo, acetato de chumbo, pigmento laca roxo, e o gel tixotrópico Megilp). As reconstruções foram analisadas utilizando Pirólise-Cromatografia Gasosa acoplada à Espec- trometria de Massa com derivatização usando Hidróxido de Tetrametilamónio, e Pirólise por Croma- tografia Gasosa Bidimensional acoplada à Espectrometria de Massa. As reconstruções foram compa- radas com a materia prima utilizada, o asfalto do Lago Trinidad, para determinar o grau de deteção deste asfalto após o processamento térmico em óleo de linhaça tratado com chumbo. Os resultados mostram que os marcadores asfálticos identificados no asfalto do Lago de Trinidad desaparecem na primeira fase da reconstrução da tinta a óleo "Betume" da W&N. Esta descoberta importante oferece uma explicação para a escassez de provas analíticas em tentativas anteriores de identificação do asfalto/betume em pinturas em que se acreditava ter sido utilizado este material. Para além de clarificar os resultados analíticos obtidos a partir da investigação da pintura O Cardeal D. Henrique..., as amostras de referência produzidas a partir da reconstrução da W&N ilustram os pontos fortes e fracos da análise orgânica de misturas complexas altamente proces- sadas

Ocean Remote Sensing with Synthetic Aperture Radar

The ocean covers approximately 71% of the Earth’s surface, 90% of the biosphere and contains 97% of Earth’s water. The Synthetic Aperture Radar (SAR) can image the ocean surface in all weather conditions and day or night. SAR remote sensing on ocean and coastal monitoring has become a research hotspot in geoscience and remote sensing. This book—Progress in SAR Oceanography—provides an update of the current state of the science on ocean remote sensing with SAR. Overall, the book presents a variety of marine applications, such as, oceanic surface and internal waves, wind, bathymetry, oil spill, coastline and intertidal zone classification, ship and other man-made objects’ detection, as well as remotely sensed data assimilation. The book is aimed at a wide audience, ranging from graduate students, university teachers and working scientists to policy makers and managers. Efforts have been made to highlight general principles as well as the state-of-the-art technologies in the field of SAR Oceanography

Machine Learning in Sensors and Imaging

Machine learning is extending its applications in various fields, such as image processing, the Internet of Things, user interface, big data, manufacturing, management, etc. As data are required to build machine learning networks, sensors are one of the most important technologies. In addition, machine learning networks can contribute to the improvement in sensor performance and the creation of new sensor applications. This Special Issue addresses all types of machine learning applications related to sensors and imaging. It covers computer vision-based control, activity recognition, fuzzy label classification, failure classification, motor temperature estimation, the camera calibration of intelligent vehicles, error detection, color prior model, compressive sensing, wildfire risk assessment, shelf auditing, forest-growing stem volume estimation, road management, image denoising, and touchscreens