Machine Learning for Robust Understanding of Scene Materials in Hyperspectral Images

The major challenges in hyperspectral (HS) imaging and data analysis are expensive sensors, high dimensionality of the signal, limited ground truth, and spectral variability. This dissertation develops and analyzes machine learning based methods to address these problems. In the first part, we examine one of the most important HS data analysis tasks-vegetation parameter estimation. We present two Gaussian processes based approaches for improving the accuracy of vegetation parameter retrieval when ground truth is limited and/or spectral variability is high. The first is the adoption of covariance functions based on well-established metrics, such as, spectral angle and spectral correlation, which are known to be better measures of similarity for spectral data. The second is the joint modeling of related vegetation parameters by multitask Gaussian processes so that the prediction accuracy of the vegetation parameter of interest can be improved with the aid of related vegetation parameters for which a larger set of ground truth is available. The efficacy of the proposed methods is demonstrated by comparing them against state-of-the art approaches on three real-world HS datasets and one synthetic dataset. In the second part, we demonstrate how Bayesian optimization can be applied to jointly tune the different components of hyperspectral data analysis frameworks for better performance. Experimental validation on the spatial-spectral classification framework consisting of a classifier and a Markov random field is provided. In the third part, we investigate whether high dimensional HS spectra can be reconstructed from low dimensional multispectral (MS) signals, that can be obtained from much cheaper, lower spectral resolution sensors. A novel end-to-end convolutional residual neural network architecture is proposed that can simultaneously optimize both the MS bands and the transformation to reconstruct HS spectra from MS signals by analyzing a large quantity of HS data. The learned band can be implemented in sensor hardware and the learned transformation can be incorporated in the data processing pipeline to build a low-cost hyperspectral data collection system. Using a diverse set of real-world datasets, we show how the proposed approach of optimizing MS bands along with the transformation rather than just optimizing the transformation with fixed bands, as proposed by previous studies, can drastically increase the reconstruction accuracy. Additionally, we also investigate the prospects of using reconstructed HS spectra for land cover classification

Linking Canopy Reflectance and Plant Functioning through Radiative Transfer Models

Von den Tropen bis zur Tundra hat sich die Pflanzenwelt durch Anpassungen an lokale Umwelteinflüsse diversifiziert. Diese Anpassungen sind in der Funktionsweise der Pflanzen manifestiert, welche unter anderem Wachstum, Fortpflanzung, Konkurrenzfähigkeit oder Ausdauer beinhalten. Pflanzenfunktionen haben nicht nur direkten Einfluss auf die Artenzusammensetzung, sondern auch auf großräumige Prozesse wie Bio- und Atmossphäreninteraktionen oder Stoffkreisläufe. Folglich wurden viele Forschungsanstrengungen unternommen um Pflanzenfunktionen weiter zu verstehen und zu erfassen, z.B. darauf abzielend generalisierende Modelle von Pflanzenfunktionen zu entwickeln oder individuelle Pflanzenmerkmale als Indikatoren für Pflanzenfunktion zu identifizieren. Trotz der wissenschaftlichen Fortschritte fehlt ein vollständiges Bild der Funktionsvielfalt der Pflanzenwelt, sowohl in geographischer als auch funktioneller Hinsicht. Dies ist im Wesentlichen auf die Komplexität und die logistischen Einschränkungen bei der Messung von Pflanzenfunktionen im Feld zurückzuführen. Um dieses Bild zu vervollständigen wird insbesondere optischen Erdbeobachtungsdaten ein hohes Potenzial zugeschrieben. Optische Erdbeobachtungssensoren erfassen das vom Kronendach reflektierte Sonnenlicht. Letzteres wird durch verschiedene biochemische und strukturelle Pflanzenmerkmale (im Folgenden optische Merkmale) beeinträchtigt (z.B. Blattchlorophyllgehalt oder Blattwinkel). Das Abfangen und Absorbieren von Sonnenlicht ist die Grundlage des pflanzeneigenen Metabolismus und folglich liegt es Nahe, dass diese optischen Merkmale direkt mit Pflanzenfunktionen zusammenhängen. Der Zusammenhang dieser optische Merkmale mit Pflanzenfunktionen wurde jedoch noch nicht systematisch untersucht, und ebenso ist der Zusammenhang zwischen Pflanzenfunktion und Kronendachreflektion noch nicht vollständig untersucht. Die physikalischen Interaktionen von Licht und optischen Pflanzenmerkmalen sind bereits hinreichend verstanden und in Strahlungstransfermodellen (RTM) für Vegetationskronendächer formuliert. RTM können als prozessbasierte Modelle betrachtet werden, die die Reflektion des Kronendachs in Abhängigkeit von optische Merkmalen, dem Bodenhintergrund und der Sonnen-Sensorgeometrie modellieren. Das Ziel und die Innovation dieser Dissertation war die kausalen Zusammenhänge zwischen Kronendachreflektion und Pflanzenfunktion mittels RTM zu verstehen und zu nutzen. Es wurde gezeigt, dass für die Fernerkundung von Pflanzenfunktionen die Kopplung von Kronendachreflektion und Pflanzenfunktionen durch RTM mehrere Potentiale bietet: Erstens, ermöglichen RTM die Kartierung von Pflanzenmerkmalen. Innerhalb einer Fallstudie wurde gezeigt, dass eine Inversion von RTM mit hyperspektralen Daten eine Kartierung von optischen Merkmalen erlaubt, für die keine Felddaten zur Modellkalibrierung benötigt werden. Die kartierten Merkmale zeigten eine hohe Übereinstimmung mit Merkmalsausprägungen aus unabhängigen Datenbanken und spiegelten die im Feld gemessenen ökologischen Gradienten wider. Dies deutet darauf hin, dass RTM-Inversion als äußerst übertragbare Methode betrachtet werden kann, um räumliche Karten von Pflanzenmerkmalen zu erstellen, die als Proxies für Pflanzenfunktionen dienen können. Allerdings erfordert die Implementierung von RTM Inversionen fundierte Kenntnisse über die Prinzipien der Strahlentransfermodellierung und der zu untersuchenden Vegetationscharakteristiken. Zweitens, ermöglichen RTM die Untersuchung von Zusammenhängen zwischen Pflanzenfunktion und der Kronendachreflektion. In der vorliegenden Thesis wurden simulierte Kronendachspektren aus einem RTM verwendet, um den Beitrag der optischen Merkmale zu den spektralen Unterschieden zwischen Pflanzenfunktionstypen zu erfassen. Die Ergebnisse zeigten die dominanten Pflanzenmerkmale und die entsprechenden spektralen Charakteristiken die für eine fernerkundliche Unterscheidung der Pflanzenfunktion von großer Relevanz sind. Darüber hinaus wurde gezeigt, dass RTM-basierte Simulationen Einschränkungen von Fallstudien kompensieren und Kenntnisse über die Zusammenhänge von Pflanzenfunktionen, Pflanzeneigenschaften und Kronendachtreflektion erweitern können. Diese Kenntnisse bilden die Grundlage für die Entwicklung und Verbesserung von Sensoren und Algorithmen zur Fernerkundung von Pflanzenfunktionen. Drittens, erweitern RTM und die darin enthaltenen optischen Merkmale unsere Möglichkeiten Unterschiede in der Pflanzenfunktion zu verstehen und zu quantifizieren. Mit Hilfe von in-situ gemessenen Merkmalsausprägungen konnte gezeigt werden, dass die in RTM enthaltenen optischen Merkmale kausal mit primären Pflanzenfunktionen zusammenhängen. Dies wiederum bedeutet, dass die Reflexion des Kronendachs unmittelbar mit den primären Funktionen der Pflanze zusammenhängt (‘Reflektion folgt Funktion’). Darüber hinaus wurde festgestellt, dass optische Merkmale vergleichbare oder sogar höhere Korrelationen mit den verwendeten pflanzlichen Funktionsgradienten aufweisen als die in der Pflanzenökologie üblich verwendeten Merkmale. Entsprechend bieten RTM sowohl eine alternative Perspektive als auch ein Set von Pflanzenmerkmalen mit denen Unterschiede der Pflanzenfunktion charakterisiert und quantifiziert werden können. Diese Merkmale können somit als wertvolle Ergänzung oder Alternative zu den in der Pflanzenökologie üblichen Merkmalen dienen. Zusammengefasst zeigt diese Thesis, dass RTM unsere Möglichkeiten erweiterten können die funktionelle Vielfalt der globalen Vegetationsbedeckung weiter zu verstehen und zu erfassen und führt zukunftsrelevante Forschungspotentiale auf

High-throughput phenotyping of plant leaf morphological, physiological, and biochemical traits on multiple scales using optical sensing

Acquisition of plant phenotypic information facilitates plant breeding, sheds light on gene action, and can be applied to optimize the quality of agricultural and forestry products. Because leaves often show the fastest responses to external environmental stimuli, leaf phenotypic traits are indicators of plant growth, health, and stress levels. Combination of new imaging sensors, image processing, and data analytics permits measurement over the full life span of plants at high temporal resolution and at several organizational levels from organs to individual plants to field populations of plants. We review the optical sensors and associated data analytics used for measuring morphological, physiological, and biochemical traits of plant leaves on multiple scales. We summarize the characteristics, advantages and limitations of optical sensing and data-processing methods applied in various plant phenotyping scenarios. Finally, we discuss the future prospects of plant leaf phenotyping research. This review aims to help researchers choose appropriate optical sensors and data processing methods to acquire plant leaf phenotypes rapidly, accurately, and cost-effectively

Quantitative estimation of vegetation traits and temporal dynamics using 3-D radiative transfer models, high-resolution hyperspectral images and satellite imagery

Large-scale monitoring of vegetation dynamics by remote sensing is key to detecting early signs of vegetation decline. Spectral-based indicators of phys-iological plant traits (PTs) have the potential to quantify variations in pho-tosynthetic pigments, chlorophyll fluorescence emission, and structural changes of vegetation as a function of stress. However, the specific response of PTs to disease-induced decline in heterogeneous canopies remains largely unknown, which is critical for the early detection of irreversible damage at different scales. Four specific objectives were defined in this research: i) to assess the feasibility of modelling the incidence and severity of Phytophthora cinnamomi and Xylella fastidiosa based on PTs and biophysical properties of vegetation; ii) to assess non-visual early indicators, iii) to retrieve PT using radiative transfer models (RTM), high-resolution imagery and satellite observations; and iv) to establish the basis for scaling up PTs at different spatial resolutions using RTM for their retrieval in different vegetation co-vers. This thesis integrates different approaches combining field data, air- and space-borne imagery, and physical and empirical models that allow the retrieval of indicators and the evaluation of each component’s contribution to understanding temporal variations of disease-induced symptoms in heter-ogeneous canopies. Furthermore, the effects associated with the understory are introduced, showing not only their impact but also providing a compre-hensive model to account for it. Consequently, a new methodology has been established to detect vegetation health processes and the influence of biotic and abiotic factors, considering different components of the canopy and their impact on the aggregated signal. It is expected that, using the presented methods, existing remote sensors and future developments, the ability to detect and assess vegetation health globally will have a substantial impact not only on socio-economic factors, but also on the preservation of our eco-system as a whole

Generation of 360 Degree Point Cloud for Characterization of Morphological and Chemical Properties of Maize and Sorghum

Recently, imaged-based high-throughput phenotyping methods have gained popularity in plant phenotyping. Imaging projects the 3D space into a 2D grid causing the loss of depth information and thus causes the retrieval of plant morphological traits challenging. In this study, LiDAR was used along with a turntable to generate a 360-degree point cloud of single plants. A LABVIEW program was developed to control and synchronize both the devices. A data processing pipeline was built to recover the digital surface models of the plants. The system was tested with maize and sorghum plants to derive the morphological properties including leaf area, leaf angle and leaf angular distribution. The results showed a high correlation between the manual measurement and the LiDAR measurements of the leaf area (R2\u3e0.91). Also, Structure from Motion (SFM) was used to generate 3D spectral point clouds of single plants at different narrow spectral bands using 2D images acquired by moving the camera completely around the plants. Seven narrow band (band width of 10 nm) optical filters, with center wavelengths at 530 nm, 570 nm, 660 nm, 680 nm, 720 nm, 770 nm and 970 nm were used to obtain the images for generating a spectral point cloud. The possibility of deriving the biochemical properties of the plants: nitrogen, phosphorous, potassium and moisture content using the multispectral information from the 3D point cloud was tested through statistical modeling techniques. The results were optimistic and thus indicated the possibility of generating a 3D spectral point cloud for deriving both the morphological and biochemical properties of the plants in the future. Advisor: Yufeng G

The data concept behind the data: From metadata models and labelling schemes towards a generic spectral library

Spectral libraries play a major role in imaging spectroscopy. They are commonly used to store end-member and spectrally pure material spectra, which are primarily used for mapping or unmixing purposes. However, the development of spectral libraries is time consuming and usually sensor and site dependent. Spectral libraries are therefore often developed, used and tailored only for a specific case study and only for one sensor. Multi-sensor and multi-site use of spectral libraries is difficult and requires technical effort for adaptation, transformation, and data harmonization steps. Especially the huge amount of urban material specifications and its spectral variations hamper the setup of a complete spectral library consisting of all available urban material spectra. By a combined use of different urban spectral libraries, besides the improvement of spectral inter- and intra-class variability, missing material spectra could be considered with respect to a multi-sensor/ -site use. Publicly available spectral libraries mostly lack the metadata information that is essential for describing spectra acquisition and sampling background, and can serve to some extent as a measure of quality and reliability of the spectra and the entire library itself. In the GenLib project, a concept for a generic, multi-site and multi-sensor usable spectral library for image spectra on the urban focus was developed. This presentation will introduce a 1) unified, easy-to-understand hierarchical labeling scheme combined with 2) a comprehensive metadata concept that is 3) implemented in the SPECCHIO spectral information system to promote the setup and usability of a generic urban spectral library (GUSL). The labelling scheme was developed to ensure the translation of individual spectral libraries with their own labelling schemes and their usually varying level of details into the GUSL framework. It is based on a modified version of the EAGLE classification concept by combining land use, land cover, land characteristics and spectral characteristics. The metadata concept consists of 59 mandatory and optional attributes that are intended to specify the spatial context, spectral library information, references, accessibility, calibration, preprocessing steps, and spectra specific information describing library spectra implemented in the GUSL. It was developed on the basis of existing metadata concepts and was subject of an expert survey. The metadata concept and the labelling scheme are implemented in the spectral information system SPECCHIO, which is used for sharing and holding GUSL spectra. It allows easy implementation of spectra as well as their specification with the proposed metadata information to extend the GUSL. Therefore, the proposed data model represents a first fundamental step towards a generic usable and continuously expandable spectral library for urban areas. The metadata concept and the labelling scheme also build the basis for the necessary adaptation and transformation steps of the GUSL in order to use it entirely or in excerpts for further multi-site and multi-sensor applications

Remote Sensing of Plant Biodiversity

This Open Access volume aims to methodologically improve our understanding of biodiversity by linking disciplines that incorporate remote sensing, and uniting data and perspectives in the fields of biology, landscape ecology, and geography. The book provides a framework for how biodiversity can be detected and evaluated—focusing particularly on plants—using proximal and remotely sensed hyperspectral data and other tools such as LiDAR. The volume, whose chapters bring together a large cross-section of the biodiversity community engaged in these methods, attempts to establish a common language across disciplines for understanding and implementing remote sensing of biodiversity across scales. The first part of the book offers a potential basis for remote detection of biodiversity. An overview of the nature of biodiversity is described, along with ways for determining traits of plant biodiversity through spectral analyses across spatial scales and linking spectral data to the tree of life. The second part details what can be detected spectrally and remotely. Specific instrumentation and technologies are described, as well as the technical challenges of detection and data synthesis, collection and processing. The third part discusses spatial resolution and integration across scales and ends with a vision for developing a global biodiversity monitoring system. Topics include spectral and functional variation across habitats and biomes, biodiversity variables for global scale assessment, and the prospects and pitfalls in remote sensing of biodiversity at the global scale

Remote Sensing of Plant Biodiversity

At last, here it is. For some time now, the world has needed a text providing both a new theoretical foundation and practical guidance on how to approach the challenge of biodiversity decline in the Anthropocene. This is a global challenge demanding global approaches to understand its scope and implications. Until recently, we have simply lacked the tools to do so. We are now entering an era in which we can realistically begin to understand and monitor the multidimensional phenomenon of biodiversity at a planetary scale. This era builds upon three centuries of scientific research on biodiversity at site to landscape levels, augmented over the past two decades by airborne research platforms carrying spectrometers, lidars, and radars for larger-scale observations. Emerging international networks of fine-grain in-situ biodiversity observations complemented by space-based sensors offering coarser-grain imagery—but global coverage—of ecosystem composition, function, and structure together provide the information necessary to monitor and track change in biodiversity globally. This book is a road map on how to observe and interpret terrestrial biodiversity across scales through plants—primary producers and the foundation of the trophic pyramid. It honors the fact that biodiversity exists across different dimensions, including both phylogenetic and functional. Then, it relates these aspects of biodiversity to another dimension, the spectral diversity captured by remote sensing instruments operating at scales from leaf to canopy to biome. The biodiversity community has needed a Rosetta Stone to translate between the language of satellite remote sensing and its resulting spectral diversity and the languages of those exploring the phylogenetic diversity and functional trait diversity of life on Earth. By assembling the vital translation, this volume has globalized our ability to track biodiversity state and change. Thus, a global problem meets a key component of the global solution. The editors have cleverly built the book in three parts. Part 1 addresses the theory behind the remote sensing of terrestrial plant biodiversity: why spectral diversity relates to plant functional traits and phylogenetic diversity. Starting with first principles, it connects plant biochemistry, physiology, and macroecology to remotely sensed spectra and explores the processes behind the patterns we observe. Examples from the field demonstrate the rising synthesis of multiple disciplines to create a new cross-spatial and spectral science of biodiversity. Part 2 discusses how to implement this evolving science. It focuses on the plethora of novel in-situ, airborne, and spaceborne Earth observation tools currently and soon to be available while also incorporating the ways of actually making biodiversity measurements with these tools. It includes instructions for organizing and conducting a field campaign. Throughout, there is a focus on the burgeoning field of imaging spectroscopy, which is revolutionizing our ability to characterize life remotely. Part 3 takes on an overarching issue for any effort to globalize biodiversity observations, the issue of scale. It addresses scale from two perspectives. The first is that of combining observations across varying spatial, temporal, and spectral resolutions for better understanding—that is, what scales and how. This is an area of ongoing research driven by a confluence of innovations in observation systems and rising computational capacity. The second is the organizational side of the scaling challenge. It explores existing frameworks for integrating multi-scale observations within global networks. The focus here is on what practical steps can be taken to organize multi-scale data and what is already happening in this regard. These frameworks include essential biodiversity variables and the Group on Earth Observations Biodiversity Observation Network (GEO BON). This book constitutes an end-to-end guide uniting the latest in research and techniques to cover the theory and practice of the remote sensing of plant biodiversity. In putting it together, the editors and their coauthors, all preeminent in their fields, have done a great service for those seeking to understand and conserve life on Earth—just when we need it most. For if the world is ever to construct a coordinated response to the planetwide crisis of biodiversity loss, it must first assemble adequate—and global—measures of what we are losing