Testing the Potential Application of Simulated Multispectral Data in Discriminating Tree Species in Taita Hills

Peer reviewe

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Spatiotemporal dynamics of stress factors in wheat analysed by multisensoral remote sensing and geostatistics

Plant stresses, in particular fungal diseases, basically show a high variability in space and time with respect to their impact on the host. Recent ‘Precision Agriculture’ techniques allow for a spatially and temporally adjusted pest control that might reduce the amount of cost-intensive and ecologically harmful agrochemicals. Conventional stress detection techniques such as random monitoring do not meet demands of such optimally placed management actions. The prerequisite is a profound knowledge about the controlled phenomena as well as their accurate sensor-based detection. Therefore, the present study focused on spatiotemporal dynamics of stress factors in wheat, Europe’s main crop. Primarily, the spatiotemporal characteristics of the fungal diseases, powdery mildew (Blumeria graminis) and leaf rust (Puccinia recondita), were analysed by remote sensing techniques and geo-statistics on leaf and field scale. Basically, there are two different approaches to sensor-based detection of crop stresses: near-range sensors and airborne-/satellite-borne sensors. In order to assess the potential of both approaches, various experiments in field and laboratory were carried out with the use of multiple sensors operated at different scales. Besides the spatial dimension of crop stresses, all studies focussed on the temporal dimension of these phenomena, since this is the key question for an operational use of these techniques. In addition, a comparison between multispectral and hyperspectral data gave an indication of their suitability for this purpose. The results exhibit very high spatiotemporal dynamics for both fungal diseases. However, powdery mildew and leaf rust showed different characteristics, with leaf rust showing a more systematic temporal progress. The physiological behaviours of the phenomena, which are strongly influenced by various environmental factors, define the optimal disease detection date as well as the temporal resolution required for sensor-based disease detection. Due to the high spatiotemporal dynamics of the investigated diseases, a general recommendation of optimal detection periods can not be given, but critical periods are highlighted for each pathogen. The results indicate that multispectral remote sensing data with high spatial resolution shows a high potential for quantifying crop vigour by using spectral mixture analyses. Simulated endmembers for the identification of stressed wheat areas were utilized, whereby promising results could be achieved. However, due to the low spectral resolution of these data, a discrimination of stress factors or early disease detection is not possible. Hyperspectral data was therefore used to point out the potential of early detection of crop diseases, which is a crucial and restrictive factor for Precision Agriculture applications. In a laboratory experiment, leaf rust infections could be detected by hyperspectral data five days after inoculation. In a field experiment with respect to early stress detection, it could be demonstrated that hyperspectral data outperformed multispectral data. High accuracy for the detection of powdery mildew infections in the field was thereby achieved. Due to the fact that typical spatiotemporal characteristics for each pathogen were found, there is a high potential for decision support systems, considering all variables that affect the disease progress. Besides the further analysis of hyperspectral data for disease detection, the development of a decision support system is the subject of the upcoming last period of the Research Training Group 722

FINCH: A Blueprint for Accessible and Scientifically Valuable Remote Sensing Satellite Missions

Satellite remote sensing missions have grown in popularity over the past fifteen years due to their ability to cover large swaths of land at regular time intervals, making them suitable for monitoring environmental trends such as greenhouse gas emissions and agricultural practices. As environmental monitoring becomes central in global efforts to combat climate change, accessible platforms for contributing to this research are critical. Many remote sensing missions demand high performance of payloads, restricting research and development to organizations with sufficient resources to address these challenges. Atmospheric remote sensing missions, for example, require extremely high spatial and spectral resolutions to generate scientifically useful results. As an undergraduate-led design team, the University of Toronto Aerospace Team’s Space Systems Division has performed an extensive mission selection process to find a feasible and impactful mission focusing on crop residue mapping. This mission profile provides the data needed to improve crop residue retention practices and reduce greenhouse gas emissions from soil, while relaxing performance requirements relative to many active atmospheric sensing missions. This is accompanied by the design of FINCH, a 3U CubeSat with a hyperspectral camera composed of custom and commercial off-the-shelf components. The team’s custom composite payload, the FINCH Eye, strives to advance performance achieved at this form factor by leveraging novel technologies while keeping design feasibility for a student team a priority. Optical and mechanical design decisions and performance are detailed, as well as assembly, integration, and testing considerations. Beyond its design, the FINCH Eye is examined from operational, timeline, and financial perspectives, and a discussion of the supporting firmware, data processing, and attitude control systems is included. Insight is provided into open-source tools that the team has developed to aid in the design process, including a linear error analysis tool for assessing scientific performance, an optical system tradeoff analysis tool, and data processing algorithms. Ultimately, the team presents a comprehensive case study of an accessible and impactful satellite optical payload design process, in hopes of serving as a blueprint for future design teams seeking to contribute to remote sensing research

SIMULATIONS-GUIDED DESIGN OF PROCESS ANALYTICAL SENSOR USING MOLECULAR FACTOR COMPUTING

Many areas of science now generate huge volumes of data that present visualization, modeling, and interpretation challenges. Methods for effectively representing the original data in a reduced coordinate space are therefore receiving much attention. The purpose of this research is to test the hypothesis that molecular computing of vectors for transformation matrices enables spectra to be represented in any arbitrary coordinate system. New coordinate systems are selected to reduce the dimensionality of the spectral hyperspace and simplify the mechanical/electrical/computational construction of a spectrometer. A novel integrated sensing and processing system, termed Molecular Factor Computing (MFC) based near infrared (NIR) spectrometer, is proposed in this dissertation. In an MFC -based NIR spectrometer, spectral features are encoded by the transmission spectrum of MFC filters which effectively compute the calibration function or the discriminant functions by weighing the signals received from a broad wavelength band. Compared with the conventional spectrometers, the novel NIR analyzer proposed in this work is orders of magnitude faster and more rugged than traditional spectroscopy instruments without sacrificing the accuracy that makes it an ideal analytical tool for process analysis. Two different MFC filter-generating algorithms are developed and tested for searching a near-infrared spectral library to select molecular filters for MFC-based spectroscopy. One using genetic algorithms coupled with predictive modeling methods to select MFC filters from a spectral library for quantitative prediction is firstly described. The second filter-generating algorithm designed to select MFC filters for qualitative classification purpose is then presented. The concept of molecular factor computing (MFC)-based predictive spectroscopy is demonstrated with quantitative analysis of ethanol-in-water mixtures in a MFC-based prototype instrument

Large Area Land Cover Mapping Using Deep Neural Networks and Landsat Time-Series Observations

This dissertation focuses on analysis and implementation of deep learning methodologies in the field of remote sensing to enhance land cover classification accuracy, which has important applications in many areas of environmental planning and natural resources management. The first manuscript conducted a land cover analysis on 26 Landsat scenes in the United States by considering six classifier variants. An extensive grid search was conducted to optimize classifier parameters using only the spectral components of each pixel. Results showed no gain in using deep networks by using only spectral components over conventional classifiers, possibly due to the small reference sample size and richness of features. The effect of changing training data size, class distribution, or scene heterogeneity were also studied and we found all of them having significant effect on classifier accuracy. The second manuscript reviewed 103 research papers on the application of deep learning methodologies in remote sensing, with emphasis on per-pixel classification of mono-temporal data and utilizing spectral and spatial data dimensions. A meta-analysis quantified deep network architecture improvement over selected convolutional classifiers. The effect of network size, learning methodology, input data dimensionality and training data size were also studied, with deep models providing enhanced performance over conventional one using spectral and spatial data. The analysis found that input dataset was a major limitation and available datasets have already been utilized to their maximum capacity. The third manuscript described the steps to build the full environment for dataset generation based on Landsat time-series data using spectral, spatial, and temporal information available for each pixel. A large dataset containing one sample block from each of 84 ecoregions in the conterminous United States (CONUS) was created and then processed by a hybrid convolutional+recurrent deep network, and the network structure was optimized with thousands of simulations. The developed model achieved an overall accuracy of 98% on the test dataset. Also, the model was evaluated for its overall and per-class performance under different conditions, including individual blocks, individual or combined Landsat sensors, and different sequence lengths. The analysis found that although the deep model performance per each block is superior to other candidates, the per block performance still varies considerably from block to block. This suggests extending the work by model fine-tuning for local areas. The analysis also found that including more time stamps or combining different Landsat sensor observations in the model input significantly enhances the model performance

Drone-based Integration of Hyperspectral Imaging and Magnetics for Mineral Exploration

The advent of unoccupied aerial systems (UAS) as disruptive technology has a lasting impact on remote sensing, geophysics and most geosciences. Small, lightweight, and low-cost UAS enable researchers and surveyors to acquire earth observation data in higher spatial and spectral resolution as compared to airborne and satellite data. UAS-based applications range from rapid topographic mapping using photogrammetric techniques to hyperspectral and geophysical measurements of surface and subsurface geology. UAS surveys contribute to identifying metal deposits, monitoring of mine sites and can reveal arising environmental issues associated with mining. Further, affordable UAS technology will boost exploration data availability and expertise in the global south. This thesis investigates the application of UAS-based multi-sensor data for mineral exploration, in particular the integration of hyperspectral imagers, magnetometers and digital cameras (covering the visible red, green, blue light spectrum). UAS-based research is maturing, however the aforementioned methods are not unified effectively. RGB-based photogrammetry is used to investigate topography and surface texture. Image spectrometers measure mineral-specific surface signatures. Magnetometers detect geomagnetic field changes caused by magnetic minerals at surface and depth. The integration of such UAS sensor-based methods in this thesis augments exploration potential with non-invasive, high-resolution, safe, rapid and practical survey methods. UAS-based surveying acquired, processed and integrated data from three distinct test sites. The sites are located in Finland (Fe-Ti-V at Otanmäki; apatite at Siilinjärvi) and Greenland (Ni-Cu-PGE at Qullissat, Disko Island) and were chosen as geologically diverse areas in subarctic to arctic environments. Restricted accessibility, unfavourable atmospheric conditions, dark rocks, debris and vegetation cover and low solar illumination were common features. While the topography in Finland was moderately flat, a steep landscape challenged the Greenland field work. These restraints meant that acquisitions varied from site to site and how data was integrated and interpreted is dependent on the commodity of interest. Iron-based spectral absorption and magnetic mineral response were detected using hyperspectral and magnetic surveying in Otanmäki. Multi-sensor-based image feature detection and classification combined with magnetic forward modelling enabled seamless geologic mapping in Siilinjärvi. Detailed magnetic inversion and multispectral photogrammetry led to the construction of a comprehensive 3D model of magmatic exploration targets in Greenland. Ground truth at different intensity was employed to verify UAS-based data interpretations during all case studies. Laboratory analysis was applied when deemed necessary to acquire geologic-mineralogic validation (e.g., X-ray diffraction and optical microscopy for mineral identification to establish lithologic domains, magnetic susceptibility measurements for subsurface modelling), for example for trace amounts of magnetite in carbonatite (Siilinjärvi) and native iron occurrence in basalt (Qullissat). Technical achievements were the integration of a multicopter-based prototype fluxgate-magnetometer data from different survey altitudes with ground truth, and a feasibility study with a high-speed multispectral image system for fixed-wing UAS. The employed case studies transfer the experiences made towards general recommendations for UAS application-based multi-sensor integration. This thesis highlights the feasibility of UAS-based surveying at target scale (1–50 km2) and solidifies versatile survey approaches for multi-sensor integration.Ziel dieser Arbeit war es, das Potenzial einer Drohnen-basierten Mineralexploration mit Multisensor-Datenintegration unter Verwendung optisch-spektroskopischer und magnetischer Methoden zu untersuchen, um u. a. übertragbare Arbeitsabläufe zu erstellen. Die untersuchte Literatur legt nahe, dass Drohnen-basierte Bildspektroskopie und magnetische Sensoren ein ausgereiftes technologisches Niveau erreichen und erhebliches Potenzial für die Anwendungsentwicklung bieten, aber es noch keine ausreichende Synergie von hyperspektralen und magnetischen Methoden gibt. Diese Arbeit umfasste drei Fallstudien, bei denen die Drohnengestützte Vermessung von geologischen Zielen in subarktischen bis arktischen Regionen angewendet wurde. Eine Kombination von Drohnen-Technologie mit RGB, Multi- und Hyperspektralkameras und Magnetometern ist vorteilhaft und schuf die Grundlage für eine integrierte Modellierung in den Fallstudien. Die Untersuchungen wurden in einem Gelände mit flacher und zerklüfteter Topografie, verdeckten Zielen und unter oft schlechten Lichtverhältnissen durchgeführt. Unter diesen Bedingungen war es das Ziel, die Anwendbarkeit von Drohnen-basierten Multisensordaten in verschiedenen Explorationsumgebungen zu bewerten. Hochauflösende Oberflächenbilder und Untergrundinformationen aus der Magnetik wurden fusioniert und gemeinsam interpretiert, dabei war eine selektive Gesteinsprobennahme und Analyse ein wesentlicher Bestandteil dieser Arbeit und für die Validierung notwendig. Für eine Eisenerzlagerstätte wurde eine einfache Ressourcenschätzung durchgeführt, indem Magnetik, bildspektroskopisch-basierte Indizes und 2D-Strukturinterpretation integriert wurden. Fotogrammetrische 3D-Modellierung, magnetisches forward-modelling und hyperspektrale Klassifizierungen wurden für eine Karbonatit-Intrusion angewendet, um einen kompletten Explorationsabschnitt zu erfassen. Eine Vektorinversion von magnetischen Daten von Disko Island, Grönland, wurden genutzt, um großräumige 3D-Modelle von undifferenzierten Erdrutschblöcken zu erstellen, sowie diese zu identifizieren und zu vermessen. Die integrierte spektrale und magnetische Kartierung in komplexen Gebieten verbesserte die Erkennungsrate und räumliche Auflösung von Erkundungszielen und reduzierte Zeit, Aufwand und benötigtes Probenmaterial für eine komplexe Interpretation. Der Prototyp einer Multispektralkamera, gebaut für eine Starrflügler-Drohne für die schnelle Vermessung, wurde entwickelt, erfolgreich getestet und zum Teil ausgewertet. Die vorgelegte Arbeit zeigt die Vorteile und Potenziale von Multisensor-Drohnen als praktisches, leichtes, sicheres, schnelles und komfortabel einsetzbares geowissenschaftliches Werkzeug, um digitale Modelle für präzise Rohstofferkundung und geologische Kartierung zu erstellen

Landscape-scale prediction of forest productivity by hyperspectral remote sensing of canopy nitrogen

Foliar nitrogen concentration represents a direct and primary link between carbon and nitrogen cycling in terrestrial ecosystems. Although foliar N is used by many ecosystem models to predict leaf-level photosynthetic rates, it has rarely been examined as a direct scalar to stand-level carbon gain. Significant improvements in remote sensing detector technology in the list decade now allow for improved landscape-level estimation of the biochemical attributes of forest ecosystems. In this study, relationships among forest growth (aboveground net primary productivity (ANPP) and aboveground woody biomass production (AWBP)), canopy chemistry and structure, and high resolution imaging spectrometry were examined for 88 long-term forest growth inventory plots maintained by the USDA Forest Service within the 300,000 ha White Mountain National Forest, New Hampshire. Analysis of plot-level data demonstrates a highly predictive relationship between whole canopy nitrogen concentration (g/100 g) and aboveground forest productivity (ANPP: R2 = 0.81, p \u3c 0.000; AWBP: R 2 = 0.86, p \u3c 0.000) within and among forest types. Forest productivity was more strongly related to mass-based foliar nitrogen concentration than with either total canopy N or canopy leaf area. Empirical relationships were developed among spectral data from the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and field-measured canopy nitrogen concentration (mass basis). Results of this analysis suggest that hyperspectral remote sensing can be used to accurately predict foliar nitrogen concentration, by mean of a full-spectrum partial least squares calibration method, both within a single scene (R2 = 0.84, SECV = 0.23) and across a large number of contiguous images (R2 = 82, SECV = 0.25), as well as between image dates (R2 = 0.69, SECV = 0.25). Forest productivity coverages for the White Mountain National Forest were developed by estimating whole canopy foliar N concentration from AVIRIS spectral response. Image spatial patterns broadly reflect the distribution of functional types, while fine scale spatial variation results from a variety of natural and anthropogenic factors. This approach provides the potential to increase the accuracy of forest growth and carbon gain estimates at the landscape level by providing information at the fine spatial scale over which environmental characteristics and human land use vary