Fusion of multi-scale hyperspectral and LiDAR features for tree species mapping

The added value of multiple data sources on tree species mapping has been widely analyzed. In particular, fusion of hyper-spectral (HS) and LiDAR sensors for forest applications is a very hot topic. In this paper, we exploit the use of multi-scale features to fuse HS and LiDAR data for tree species mapping. Hyperspectral data is obtained from the APEX sensor with 286 spectral bands. LiDAR data has been acquired with a TopoSys sensor Harrier 56 at full waveform. We generate multi-scale features on both HS and LiDAR data, by considering the diameter and the height layer of different tree species. Experimental results on a forested area in Belgium demonstrate the effectiveness of using multi-scale features for fusion of HS image and LiDAR data both visually and quantitatively

Assessing the role of EO in biodiversity monitoring: options for integrating in-situ observations with EO within the context of the EBONE concept

The European Biodiversity Observation Network (EBONE) is a European contribution on terrestrial monitoring to GEO BON, the Group on Earth Observations Biodiversity Observation Network. EBONE’s aims are to develop a system of biodiversity observation at regional, national and European levels by assessing existing approaches in terms of their validity and applicability starting in Europe, then expanding to regions in Africa. The objective of EBONE is to deliver: 1. A sound scientific basis for the production of statistical estimates of stock and change of key indicators; 2. The development of a system for estimating past changes and forecasting and testing policy options and management strategies for threatened ecosystems and species; 3. A proposal for a cost-effective biodiversity monitoring system. There is a consensus that Earth Observation (EO) has a role to play in monitoring biodiversity. With its capacity to observe detailed spatial patterns and variability across large areas at regular intervals, our instinct suggests that EO could deliver the type of spatial and temporal coverage that is beyond reach with in-situ efforts. Furthermore, when considering the emerging networks of in-situ observations, the prospect of enhancing the quality of the information whilst reducing cost through integration is compelling. This report gives a realistic assessment of the role of EO in biodiversity monitoring and the options for integrating in-situ observations with EO within the context of the EBONE concept (cfr. EBONE-ID1.4). The assessment is mainly based on a set of targeted pilot studies. Building on this assessment, the report then presents a series of recommendations on the best options for using EO in an effective, consistent and sustainable biodiversity monitoring scheme. The issues that we faced were many: 1. Integration can be interpreted in different ways. One possible interpretation is: the combined use of independent data sets to deliver a different but improved data set; another is: the use of one data set to complement another dataset. 2. The targeted improvement will vary with stakeholder group: some will seek for more efficiency, others for more reliable estimates (accuracy and/or precision); others for more detail in space and/or time or more of everything. 3. Integration requires a link between the datasets (EO and in-situ). The strength of the link between reflected electromagnetic radiation and the habitats and their biodiversity observed in-situ is function of many variables, for example: the spatial scale of the observations; timing of the observations; the adopted nomenclature for classification; the complexity of the landscape in terms of composition, spatial structure and the physical environment; the habitat and land cover types under consideration. 4. The type of the EO data available varies (function of e.g. budget, size and location of region, cloudiness, national and/or international investment in airborne campaigns or space technology) which determines its capability to deliver the required output. EO and in-situ could be combined in different ways, depending on the type of integration we wanted to achieve and the targeted improvement. We aimed for an improvement in accuracy (i.e. the reduction in error of our indicator estimate calculated for an environmental zone). Furthermore, EO would also provide the spatial patterns for correlated in-situ data. EBONE in its initial development, focused on three main indicators covering: (i) the extent and change of habitats of European interest in the context of a general habitat assessment; (ii) abundance and distribution of selected species (birds, butterflies and plants); and (iii) fragmentation of natural and semi-natural areas. For habitat extent, we decided that it did not matter how in-situ was integrated with EO as long as we could demonstrate that acceptable accuracies could be achieved and the precision could consistently be improved. The nomenclature used to map habitats in-situ was the General Habitat Classification. We considered the following options where the EO and in-situ play different roles: using in-situ samples to re-calibrate a habitat map independently derived from EO; improving the accuracy of in-situ sampled habitat statistics, by post-stratification with correlated EO data; and using in-situ samples to train the classification of EO data into habitat types where the EO data delivers full coverage or a larger number of samples. For some of the above cases we also considered the impact that the sampling strategy employed to deliver the samples would have on the accuracy and precision achieved. Restricted access to European wide species data prevented work on the indicator ‘abundance and distribution of species’. With respect to the indicator ‘fragmentation’, we investigated ways of delivering EO derived measures of habitat patterns that are meaningful to sampled in-situ observations

Ash Tree Identification Based on the Integration of Hyperspectral Imagery and High-density Lidar Data

Monitoring and management of ash trees has become particularly important in recent years due to the heightened risk of attack from the invasive pest, the emerald ash borer (EAB). However, distinguishing ash from other deciduous trees can be challenging. Both hyperspectral imagery and Light detection and ranging (LiDAR) data are two valuable data sources that are often used for tree species classification. Hyperspectral imagery measures detailed spectral reflectance related to the biochemical properties of vegetation, while LiDAR data measures the three-dimensional structure of tree crowns related to morphological characteristics. Thus, the accuracy of vegetation classification may be improved by combining both techniques. Therefore, the objective of this research is to integrate hyperspectral imagery and LiDAR data for improving ash tree identification. Specifically, the research aims include: 1) using LiDAR data for individual tree crowns segmentation; 2) using hyperspectral imagery for extraction of relative pure crown spectra; 3) fusing hyperspectral and LiDAR data for ash tree identification. It is expected that the classification accuracy of ash trees will be significantly improved with the integration of hyperspectral and LiDAR techniques. Analysis results suggest that, first, 3D crown structures of individual trees can be reconstructed using a set of generalized geometric models which optimally matched LiDAR-derived raster image, and crown widths can be further estimated using tree height and shape-related parameters as independent variables and ground measurement of crown widths as dependent variables. Second, with constrained linear spectral mixture analysis method, the fractions of all materials within a pixel can be extracted, and relative pure crown-scale spectra can be further calculated using illuminated-leaf fraction as weighting factors for tree species classification. Third, both crown shape index (SI) and coefficient of variation (CV) can be extracted from LiDAR data as invariant variables in tree’s life cycle, and improve ash tree identification by integrating with pixel-weighted crown spectra. Therefore, three major contributions of this research have been made in the field of tree species classification:1) the automatic estimation of individual tree crown width from LiDAR data by combining a generalized geometric model and a regression model, 2) the computation of relative pure crown-scale spectral reflectance using a pixel-weighting algorithm for tree species classification, 3) the fusion of shape-related structural features and pixel-weighted crown-scale spectral features for improving of ash tree identification

Tree Classification with Fused Mobile Laser Scanning and Hyperspectral Data

Mobile Laser Scanning data were collected simultaneously with hyperspectral data using the Finnish Geodetic Institute Sensei system. The data were tested for tree species classification. The test area was an urban garden in the City of Espoo, Finland. Point clouds representing 168 individual tree specimens of 23 tree species were determined manually. The classification of the trees was done using first only the spatial data from point clouds, then with only the spectral data obtained with a spectrometer, and finally with the combined spatial and hyperspectral data from both sensors. Two classification tests were performed: the separation of coniferous and deciduous trees, and the identification of individual tree species. All determined tree specimens were used in distinguishing coniferous and deciduous trees. A subset of 133 trees and 10 tree species was used in the tree species classification. The best classification results for the fused data were 95.8% for the separation of the coniferous and deciduous classes. The best overall tree species classification succeeded with 83.5% accuracy for the best tested fused data feature combination. The respective results for paired structural features derived from the laser point cloud were 90.5% for the separation of the coniferous and deciduous classes and 65.4% for the species classification. Classification accuracies with paired hyperspectral reflectance value data were 90.5% for the separation of coniferous and deciduous classes and 62.4% for different species. The results are among the first of their kind and they show that mobile collected fused data outperformed single-sensor data in both classification tests and by a significant margin

Remote sensing image fusion on 3D scenarios: A review of applications for agriculture and forestry

Three-dimensional (3D) image mapping of real-world scenarios has a great potential to provide the user with a more accurate scene understanding. This will enable, among others, unsupervised automatic sampling of meaningful material classes from the target area for adaptive semi-supervised deep learning techniques. This path is already being taken by the recent and fast-developing research in computational fields, however, some issues related to computationally expensive processes in the integration of multi-source sensing data remain. Recent studies focused on Earth observation and characterization are enhanced by the proliferation of Unmanned Aerial Vehicles (UAV) and sensors able to capture massive datasets with a high spatial resolution. In this scope, many approaches have been presented for 3D modeling, remote sensing, image processing and mapping, and multi-source data fusion. This survey aims to present a summary of previous work according to the most relevant contributions for the reconstruction and analysis of 3D models of real scenarios using multispectral, thermal and hyperspectral imagery. Surveyed applications are focused on agriculture and forestry since these fields concentrate most applications and are widely studied. Many challenges are currently being overcome by recent methods based on the reconstruction of multi-sensorial 3D scenarios. In parallel, the processing of large image datasets has recently been accelerated by General-Purpose Graphics Processing Unit (GPGPU) approaches that are also summarized in this work. Finally, as a conclusion, some open issues and future research directions are presented.European Commission 1381202-GEU PYC20-RE-005-UJA IEG-2021Junta de Andalucia 1381202-GEU PYC20-RE-005-UJA IEG-2021Instituto de Estudios GiennesesEuropean CommissionSpanish Government UIDB/04033/2020DATI-Digital Agriculture TechnologiesPortuguese Foundation for Science and Technology 1381202-GEU FPU19/0010

MAPPING FOREST STRUCTURE AND HABITAT CHARACTERISTICS USING LIDAR AND MULTI-SENSOR FUSION

This dissertation explored the combined use of lidar and other remote sensing data for improved forest structure and habitat mapping. The objectives were to quantify aboveground biomass and canopy dynamics and map habitat characteristics with lidar and /or fusion approaches. Structural metrics from lidar and spectral characteristics from hyperspectral data were combined for improving biomass estimates in the Sierra Nevada, California. Addition of hyperspectral metrics only marginally improved biomass estimates from lidar, however, predictions from lidar after species stratification of field data improved by 12%. Spatial predictions from lidar after species stratification of hyperspectral data also had lower errors suggesting this could be viable method for mapping biomass at landscape level. A combined analysis of the two datasets further showed that fusion could have considerably more value in understanding ecosystem and habitat characteristics. The second objective was to quantify canopy height and biomass changes in in the Sierra Nevada using lidar data acquired in 1999 and 2008. Direct change detection showed overall statistically significant positive height change at footprint level (ΔRH100 = 0.69 m, +/- 7.94 m). Across the landscape, ~20 % of height and biomass changes were significant with more than 60% being positive, suggesting regeneration from past disturbances and a small net carbon sink. This study added further evidence to the capabilities of waveform lidar in mapping canopy dynamics while highlighting the need for error analysis and rigorous field validation Lastly, fusion applications for habitat mapping were tested with radar, lidar and multispectral data in the Hubbard Brook Experimental Forest, New Hampshire. A suite of metrics from each dataset was used to predict multi-year presence for eight migratory songbirds with data mining methods. Results showed that fusion improved predictions for all datasets, with more than 25% improvement from radar alone. Spatial predictions from fusion were also consistent with known habitat preferences for the birds demonstrating the potential of multi- sensor fusion in mapping habitat characteristics. The main contribution of this research was an improved understanding of lidar and multi-sensor fusion approaches for applications in carbon science and habitat studies

Individual Tree Species Classification from Airborne Multisensor Imagery Using Robust PCA

Remote sensing of individual tree species has many applications in resource management, biodiversity assessment, and conservation. Airborne remote sensing using light detection and ranging (LiDAR) and hyperspectral sensors has been used extensively to extract biophysical traits of vegetation and to detect species. However, its application for individual tree mapping remains limited due to the technical challenges of precise coalignment of images acquired from different sensors and accurately delineating individual tree crowns (ITCs). In this study, we developed a generic workflow to map tree species at ITC level from hyperspectral imagery and LiDAR data using a combination of well established and recently developed techniques. The workflow uses a nonparametric image registration approach to coalign images, a multiclass normalized graph cut method for ITC delineation, robust principal component analysis for feature extraction, and support vector machine for species classification. This workflow allows us to automatically map tree species at both pixel- and ITC-level. Experimental tests of the technique were conducted using ground data collected from a fully mapped temperate woodland in the UK. The overall accuracy of pixel-level classification was 91%, while that of ITC-level classification was 61%. The test results demonstrate the effectiveness of the approach, and in particular the use of robust principal component analysis to prune the hyperspectral dataset and reveal subtle difference among species.Department for Environment, Food and Rural AffairsThis is the author accepted manuscript. The final version is available from IEEE via http://dx.doi.org/10.1109/JSTARS.2016.256940

Multisource and Multitemporal Data Fusion in Remote Sensing

The sharp and recent increase in the availability of data captured by different sensors combined with their considerably heterogeneous natures poses a serious challenge for the effective and efficient processing of remotely sensed data. Such an increase in remote sensing and ancillary datasets, however, opens up the possibility of utilizing multimodal datasets in a joint manner to further improve the performance of the processing approaches with respect to the application at hand. Multisource data fusion has, therefore, received enormous attention from researchers worldwide for a wide variety of applications. Moreover, thanks to the revisit capability of several spaceborne sensors, the integration of the temporal information with the spatial and/or spectral/backscattering information of the remotely sensed data is possible and helps to move from a representation of 2D/3D data to 4D data structures, where the time variable adds new information as well as challenges for the information extraction algorithms. There are a huge number of research works dedicated to multisource and multitemporal data fusion, but the methods for the fusion of different modalities have expanded in different paths according to each research community. This paper brings together the advances of multisource and multitemporal data fusion approaches with respect to different research communities and provides a thorough and discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to conduct novel investigations on this challenging topic by supplying sufficient detail and references

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin