Multisensory Imagery Cues for Object Separation, Specularity Detection and Deep Learning based Inpainting

Multisensory imagery cues have been actively investigated in diverse applications in the computer vision community to provide additional geometric information that is either absent or difficult to capture from mainstream two-dimensional imaging. The inherent features of multispectral polarimetric light field imagery (MSPLFI) include object distribution over spectra, surface properties, shape, shading and pixel flow in light space. The aim of this dissertation is to explore these inherent properties to exploit new structures and methodologies for the tasks of object separation, specularity detection and deep learning-based inpainting in MSPLFI. In the first part of this research, an application to separate foreground objects from the background in both outdoor and indoor scenes using multispectral polarimetric imagery (MSPI) cues is examined. Based on the pixel neighbourhood relationship, an on-demand clustering technique is proposed and implemented to separate artificial objects from natural background in a complex outdoor scene. However, due to indoor scenes only containing artificial objects, with vast variations in energy levels among spectra, a multiband fusion technique followed by a background segmentation algorithm is proposed to separate the foreground from the background. In this regard, first, each spectrum is decomposed into low and high frequencies using the fast Fourier transform (FFT) method. Second, principal component analysis (PCA) is applied on both frequency images of the individual spectrum and then combined with the first principal components as a fused image. Finally, a polarimetric background segmentation (BS) algorithm based on the Stokes vector is proposed and implemented on the fused image. The performance of the proposed approaches are evaluated and compared using publicly available MSPI datasets and the dice similarity coefficient (DSC). The proposed multiband fusion and BS methods demonstrate better fusion quality and higher segmentation accuracy compared with other studies for several metrics, including mean absolute percentage error (MAPE), peak signal-to-noise ratio (PSNR), Pearson correlation coefficient (PCOR) mutual information (MI), accuracy, Geometric Mean (G-mean), precision, recall and F1-score. In the second part of this work, a twofold framework for specular reflection detection (SRD) and specular reflection inpainting (SRI) in transparent objects is proposed. The SRD algorithm is based on the mean, the covariance and the Mahalanobis distance for predicting anomalous pixels in MSPLFI. The SRI algorithm first selects four-connected neighbouring pixels from sub-aperture images and then replaces the SRD pixel with the closest matched pixel. For both algorithms, a 6D MSPLFI transparent object dataset is captured from multisensory imagery cues due to the unavailability of this kind of dataset. The experimental results demonstrate that the proposed algorithms predict higher SRD accuracy and better SRI quality than the existing approaches reported in this part in terms of F1-score, G-mean, accuracy, the structural similarity index (SSIM), the PSNR, the mean squared error (IMMSE) and the mean absolute deviation (MAD). However, due to synthesising SRD pixels based on the pixel neighbourhood relationship, the proposed inpainting method in this research produces artefacts and errors when inpainting large specularity areas with irregular holes. Therefore, in the last part of this research, the emphasis is on inpainting large specularity areas with irregular holes based on the deep feature extraction from multisensory imagery cues. The proposed six-stage deep learning inpainting (DLI) framework is based on the generative adversarial network (GAN) architecture and consists of a generator network and a discriminator network. First, pixels’ global flow in the sub-aperture images is calculated by applying the large displacement optical flow (LDOF) method. The proposed training algorithm combines global flow with local flow and coarse inpainting results predicted from the baseline method. The generator attempts to generate best-matched features, while the discriminator seeks to predict the maximum difference between the predicted results and the actual results. The experimental results demonstrate that in terms of the PSNR, MSSIM, IMMSE and MAD, the proposed DLI framework predicts superior inpainting quality to the baseline method and the previous part of this research

Synthetic Aperture Radar (SAR) Meets Deep Learning

This reprint focuses on the application of the combination of synthetic aperture radars and depth learning technology. It aims to further promote the development of SAR image intelligent interpretation technology. A synthetic aperture radar (SAR) is an important active microwave imaging sensor, whose all-day and all-weather working capacity give it an important place in the remote sensing community. Since the United States launched the first SAR satellite, SAR has received much attention in the remote sensing community, e.g., in geological exploration, topographic mapping, disaster forecast, and traffic monitoring. It is valuable and meaningful, therefore, to study SAR-based remote sensing applications. In recent years, deep learning represented by convolution neural networks has promoted significant progress in the computer vision community, e.g., in face recognition, the driverless field and Internet of things (IoT). Deep learning can enable computational models with multiple processing layers to learn data representations with multiple-level abstractions. This can greatly improve the performance of various applications. This reprint provides a platform for researchers to handle the above significant challenges and present their innovative and cutting-edge research results when applying deep learning to SAR in various manuscript types, e.g., articles, letters, reviews and technical reports

Proceedings of the 2020 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

In 2020 fand der jährliche Workshop des Faunhofer IOSB und the Lehrstuhls für interaktive Echtzeitsysteme statt. Vom 27. bis zum 31. Juli trugen die Doktorranden der beiden Institute über den Stand ihrer Forschung vor in Themen wie KI, maschinellen Lernen, computer vision, usage control, Metrologie vor. Die Ergebnisse dieser Vorträge sind in diesem Band als technische Berichte gesammelt

Outdoor Scenes Pixel-Wise Semantic Segmentation using Polarimetry and Fully Convolutional Network

International audienceIn this paper, we propose a novel method for pixel-wise scene segmentation application using polarimetry. To address the difficulty of detecting highly reflective areas such as water and windows, we use the angle and degree of polarization of these areas, obtained by processing images from a polarimetric camera. A deep learning framework, based on encoder-decoder architecture, is used for the segmentation of regions of interest. Different methods of augmentation have been developed to obtain a sufficient amount of data, while preserving the physical properties of the polarimetric images. Moreover, we introduce a new dataset comprising both RGB and polarimetric images with manual ground truth annotations for seven different classes. Experimental results on this dataset, show that deep learning can benefit from polarimetry and obtain better segmentation results compared to RGB modality. In particular, we obtain an improvement of 38.35% and 22.92% in the accuracy for segmenting windows and cars respectively

Shortest Route at Dynamic Location with Node Combination-Dijkstra Algorithm

Abstract— Online transportation has become a basic requirement of the general public in support of all activities to go to work, school or vacation to the sights. Public transportation services compete to provide the best service so that consumers feel comfortable using the services offered, so that all activities are noticed, one of them is the search for the shortest route in picking the buyer or delivering to the destination. Node Combination method can minimize memory usage and this methode is more optimal when compared to A* and Ant Colony in the shortest route search like Dijkstra algorithm, but can’t store the history node that has been passed. Therefore, using node combination algorithm is very good in searching the shortest distance is not the shortest route. This paper is structured to modify the node combination algorithm to solve the problem of finding the shortest route at the dynamic location obtained from the transport fleet by displaying the nodes that have the shortest distance and will be implemented in the geographic information system in the form of map to facilitate the use of the system. Keywords— Shortest Path, Algorithm Dijkstra, Node Combination, Dynamic Location (key words

Remote Sensing of Plant Biodiversity

This Open Access volume aims to methodologically improve our understanding of biodiversity by linking disciplines that incorporate remote sensing, and uniting data and perspectives in the fields of biology, landscape ecology, and geography. The book provides a framework for how biodiversity can be detected and evaluated—focusing particularly on plants—using proximal and remotely sensed hyperspectral data and other tools such as LiDAR. The volume, whose chapters bring together a large cross-section of the biodiversity community engaged in these methods, attempts to establish a common language across disciplines for understanding and implementing remote sensing of biodiversity across scales. The first part of the book offers a potential basis for remote detection of biodiversity. An overview of the nature of biodiversity is described, along with ways for determining traits of plant biodiversity through spectral analyses across spatial scales and linking spectral data to the tree of life. The second part details what can be detected spectrally and remotely. Specific instrumentation and technologies are described, as well as the technical challenges of detection and data synthesis, collection and processing. The third part discusses spatial resolution and integration across scales and ends with a vision for developing a global biodiversity monitoring system. Topics include spectral and functional variation across habitats and biomes, biodiversity variables for global scale assessment, and the prospects and pitfalls in remote sensing of biodiversity at the global scale

Remote Sensing of Plant Biodiversity

At last, here it is. For some time now, the world has needed a text providing both a new theoretical foundation and practical guidance on how to approach the challenge of biodiversity decline in the Anthropocene. This is a global challenge demanding global approaches to understand its scope and implications. Until recently, we have simply lacked the tools to do so. We are now entering an era in which we can realistically begin to understand and monitor the multidimensional phenomenon of biodiversity at a planetary scale. This era builds upon three centuries of scientific research on biodiversity at site to landscape levels, augmented over the past two decades by airborne research platforms carrying spectrometers, lidars, and radars for larger-scale observations. Emerging international networks of fine-grain in-situ biodiversity observations complemented by space-based sensors offering coarser-grain imagery—but global coverage—of ecosystem composition, function, and structure together provide the information necessary to monitor and track change in biodiversity globally. This book is a road map on how to observe and interpret terrestrial biodiversity across scales through plants—primary producers and the foundation of the trophic pyramid. It honors the fact that biodiversity exists across different dimensions, including both phylogenetic and functional. Then, it relates these aspects of biodiversity to another dimension, the spectral diversity captured by remote sensing instruments operating at scales from leaf to canopy to biome. The biodiversity community has needed a Rosetta Stone to translate between the language of satellite remote sensing and its resulting spectral diversity and the languages of those exploring the phylogenetic diversity and functional trait diversity of life on Earth. By assembling the vital translation, this volume has globalized our ability to track biodiversity state and change. Thus, a global problem meets a key component of the global solution. The editors have cleverly built the book in three parts. Part 1 addresses the theory behind the remote sensing of terrestrial plant biodiversity: why spectral diversity relates to plant functional traits and phylogenetic diversity. Starting with first principles, it connects plant biochemistry, physiology, and macroecology to remotely sensed spectra and explores the processes behind the patterns we observe. Examples from the field demonstrate the rising synthesis of multiple disciplines to create a new cross-spatial and spectral science of biodiversity. Part 2 discusses how to implement this evolving science. It focuses on the plethora of novel in-situ, airborne, and spaceborne Earth observation tools currently and soon to be available while also incorporating the ways of actually making biodiversity measurements with these tools. It includes instructions for organizing and conducting a field campaign. Throughout, there is a focus on the burgeoning field of imaging spectroscopy, which is revolutionizing our ability to characterize life remotely. Part 3 takes on an overarching issue for any effort to globalize biodiversity observations, the issue of scale. It addresses scale from two perspectives. The first is that of combining observations across varying spatial, temporal, and spectral resolutions for better understanding—that is, what scales and how. This is an area of ongoing research driven by a confluence of innovations in observation systems and rising computational capacity. The second is the organizational side of the scaling challenge. It explores existing frameworks for integrating multi-scale observations within global networks. The focus here is on what practical steps can be taken to organize multi-scale data and what is already happening in this regard. These frameworks include essential biodiversity variables and the Group on Earth Observations Biodiversity Observation Network (GEO BON). This book constitutes an end-to-end guide uniting the latest in research and techniques to cover the theory and practice of the remote sensing of plant biodiversity. In putting it together, the editors and their coauthors, all preeminent in their fields, have done a great service for those seeking to understand and conserve life on Earth—just when we need it most. For if the world is ever to construct a coordinated response to the planetwide crisis of biodiversity loss, it must first assemble adequate—and global—measures of what we are losing

Cumulative index to NASA Tech Briefs, 1986-1990, volumes 10-14

Tech Briefs are short announcements of new technology derived from the R&D activities of the National Aeronautics and Space Administration. These briefs emphasize information considered likely to be transferrable across industrial, regional, or disciplinary lines and are issued to encourage commercial application. This cumulative index of Tech Briefs contains abstracts and four indexes (subject, personal author, originating center, and Tech Brief number) and covers the period 1986 to 1990. The abstract section is organized by the following subject categories: electronic components and circuits, electronic systems, physical sciences, materials, computer programs, life sciences, mechanics, machinery, fabrication technology, and mathematics and information sciences