Deep learning in plant phenological research: A systematic literature review

Climate change represents one of the most critical threats to biodiversity with far-reaching consequences for species interactions, the functioning of ecosystems, or the assembly of biotic communities. Plant phenology research has gained increasing attention as the timing of periodic events in plants is strongly affected by seasonal and interannual climate variation. Recent technological development allowed us to gather invaluable data at a variety of spatial and ecological scales. The feasibility of phenological monitoring today and in the future depends heavily on developing tools capable of efficiently analyzing these enormous amounts of data. Deep Neural Networks learn representations from data with impressive accuracy and lead to significant breakthroughs in, e.g., image processing. This article is the first systematic literature review aiming to thoroughly analyze all primary studies on deep learning approaches in plant phenology research. In a multi-stage process, we selected 24 peer-reviewed studies published in the last five years (2016–2021). After carefully analyzing these studies, we describe the applied methods categorized according to the studied phenological stages, vegetation type, spatial scale, data acquisition- and deep learning methods. Furthermore, we identify and discuss research trends and highlight promising future directions. We present a systematic overview of previously applied methods on different tasks that can guide this emerging complex research field

Monitoring Fine-Scale Forest Health Using Unmanned Aerial Systems (UAS) Multispectral Models

Forest disturbances—driven by pests, pathogens, and discrete events—have led to billions of dollars in lost ecosystem services and management costs. To understand the patterns and severity of these stressors across complex landscapes, there must be an increase in reliable data at scales compatible with management actions. Unmanned aerial systems (UAS or UAV) offer a capable platform for collecting local scale (e.g., individual tree) forestry data. In this study, we evaluate the capability of UAS multispectral imagery and freely available National Agricultural Imagery Program (NAIP) imagery for differentiating coniferous healthy, coniferous stressed, deciduous healthy, deciduous stressed, and degraded individual trees throughout a complex, mixed-species forests. These methods are first compared to assessments of crown vigor in the field, to evaluate the potential in supplementing this resource intensive practice. This investigation uses the random forest and support vector machine (SVM) machine learning algorithms to classify the imagery into the five forest health classes. Using the random forest classifier, the UAS imagery correctly classified five forest Health classes with an overall accuracy of 65.43%. Using similar methods, the high-resolution airborne NAIP imagery achieved an overall accuracy of 50.50% for the five health classes, a reduction of 14.93%. When these classes were generalized to healthy, stressed, and degraded trees, the accuracy improved to 71.19%, using UAS imagery, and 70.62%, using airborne imagery. Further analysis into the precise calibration of UAS multispectral imagery, a refinement of image segmentation methods, and the fusion of these data with more widely distributed remotely sensed imagery would further enhance the potential of these methods to more effectively and efficiently collect forest health information from the UAS instead of using field methods

Fernerkundung der Vegetationsphänologie über MODIS NDVI Daten - Herausforderungen bei der Datenverarbeitung und -validierung mittels Bodenbeobachtungen zahlreicher Arten und LiDAR

Phenology, the cyclic events in living organisms is triggered by climatic conditions and indicators of climate change. They are important factors influencing species interactions and ecosystem functioning. This thesis deals with the estimation of phenological metrics (Land Surface Phenology or LSP) from MODIS based time series NDVI data. Results of data analysis emphasises the role of ground observations, topography and LiDAR characteristics of forest stand in describing the variability in LSP.Phänologie, die zyklischen Stadien von lebenden Organismen werden über klimatische Verhältnisse gesteuert und dienen als Indikatoren des Klimawandels. Diese Faktoren beeinflussen maßgeblich die Interaktionen zwischen Arten und sind für das Funktionieren von Ökosystemen ausschlaggebend. Diese Arbeit behandelt die Bestimmung von phänologischen Metriken (Phänologie der Landoberfläche oder LSP) unter Verwendung von MODIS basierten NDVI Zeitreihen. Die Ergebnisse der Datenanalyse hebt die Wichtigkeit von Bodenbeobachtungen, Topographie und LiDAR Merkmalen von Waldbeständen bei der Beschreibung der LSP Variabilität hervor

UNMANNED AERIAL SYSTEMS (UAS) AS A TOOL FOR INVESTIGATING EDGE INFLUENCES IN NEW HAMPSHIRE FORESTS

The continued decline in forest cover across New England becomes more concerning when faced with the fact that these same forests may be playing an important role in the fight against climate change. New Hampshire, in particular, is experiencing a 0.27% annual net loss in forest cover as of 2018. Increased population growth and accompanied development has resulted in the removal of forest cover and the fragmentation of once continuous forest blocks. Fragmentation can lead to further degradation of the remaining forest stands via alterations of the biotic and abiotic process at their edges. The use of unmanned aerial systems (UAS) is becoming an important tool to ensure the sustainable management of current forests stands and may help to better understand the effects of fragmentation at forest edges. Because of the relatively recent arrival of this technology, effective and appropriate testing for accurate and efficient data collection is necessary. Furthermore, UAS have not been employed yet to detect edge effects.This research investigated the impacts of UAS flight parameters on the accuracy of canopy height estimates made from UAS data by comparing UAS estimates across twelve combinations of flying height and image overlap to ground measured canopy height. A multi-temporal approach to species level mapping with UAS imagery was tested by collecting multiple dates of UAS imagery from early spring to late summer and assessing whether the inclusion of one or more dates improved classification accuracy. Additional comparisons between RGB and multi-spectral cameras were carried out. Finally, UAS imagery was used to measure and assess the changes in canopy cover with increased distance from the edge. This trend was compared to trends in canopy cover measured on the ground. The results show that flying height had no impact of the accuracy of the height estimates made from UAS data and increasing forward image overlap resulted in a significant but minor increase in accuracy. Classification accuracy was improved with the use of multi-temporal data collection but no more than three dates of optimally timed imagery was necessary. Additionally, the RGB imagery produced maps with consistently higher accuracy than the multi-spectral sensor employed in this study. Finally, we were able to detect and measure a significant trend in canopy cover that mimicked the trends found on the ground. The results of the first two parts of this dissertation will go on to provide guidance to forestry practitioners on how to collect UAS that balances accuracy and efficiency, thus reducing project costs. The final result serves as an initial demonstration of utilizing UAS for understanding edge effects and opens the door to better understanding the impacts of fragmentation over larger areas

UNMANNED AERIAL SYSTEMS (UAS) AS A TOOL FOR INVESTIGATING EDGE INFLUENCES IN NEW HAMPSHIRE FORESTS

The continued decline in forest cover across New England becomes more concerning when faced with the fact that these same forests may be playing an important role in the fight against climate change. New Hampshire, in particular, is experiencing a 0.27% annual net loss in forest cover as of 2018. Increased population growth and accompanied development has resulted in the removal of forest cover and the fragmentation of once continuous forest blocks. Fragmentation can lead to further degradation of the remaining forest stands via alterations of the biotic and abiotic process at their edges. The use of unmanned aerial systems (UAS) is becoming an important tool to ensure the sustainable management of current forests stands and may help to better understand the effects of fragmentation at forest edges. Because of the relatively recent arrival of this technology, effective and appropriate testing for accurate and efficient data collection is necessary. Furthermore, UAS have not been employed yet to detect edge effects.This research investigated the impacts of UAS flight parameters on the accuracy of canopy height estimates made from UAS data by comparing UAS estimates across twelve combinations of flying height and image overlap to ground measured canopy height. A multi-temporal approach to species level mapping with UAS imagery was tested by collecting multiple dates of UAS imagery from early spring to late summer and assessing whether the inclusion of one or more dates improved classification accuracy. Additional comparisons between RGB and multi-spectral cameras were carried out. Finally, UAS imagery was used to measure and assess the changes in canopy cover with increased distance from the edge. This trend was compared to trends in canopy cover measured on the ground. The results show that flying height had no impact of the accuracy of the height estimates made from UAS data and increasing forward image overlap resulted in a significant but minor increase in accuracy. Classification accuracy was improved with the use of multi-temporal data collection but no more than three dates of optimally timed imagery was necessary. Additionally, the RGB imagery produced maps with consistently higher accuracy than the multi-spectral sensor employed in this study. Finally, we were able to detect and measure a significant trend in canopy cover that mimicked the trends found on the ground. The results of the first two parts of this dissertation will go on to provide guidance to forestry practitioners on how to collect UAS that balances accuracy and efficiency, thus reducing project costs. The final result serves as an initial demonstration of utilizing UAS for understanding edge effects and opens the door to better understanding the impacts of fragmentation over larger areas

Assessing spring phenology of a temperate woodland : a multiscale comparison of ground, unmanned aerial vehicle and Landsat satellite observations

PhD ThesisVegetation phenology is the study of plant natural life cycle stages. Plant phenological events are related to carbon, energy and water cycles within terrestrial ecosystems, operating from local to global scales. As plant phenology events are highly sensitive to climate fluctuations, the timing of these events has been used as an independent indicator of climate change. The monitoring of forest phenology in a cost-effective manner, at a fine spatial scale and over relatively large areas remains a significant challenge. To address this issue, unmanned aerial vehicles (UAVs) appear to be a potential new platform for forest phenology monitoring. The aim of this research is to assess the potential of UAV data to track the temporal dynamics of spring phenology, from the individual tree to woodland scale, and to cross-compare UAV results against ground and satellite observations, in order to better understand characteristics of UAV data and assess potential for use in validation of satellite-derived phenology. A time series of UAV data were acquired in tandem with an intensive ground campaign during the spring season of 2015, over Hanging Leaves Wood, Northumberland, UK. The radiometric quality of the UAV imagery acquired by two consumer-grade cameras was assessed, in terms of the ability to retrieve reflectance and Normalised Difference Vegetation Index (NDVI), and successfully validated against ground (0.84≤R2≥0.96) and Landsat (0.73≤R2≥0.89) measurements, but only NDVI resulted in stable time series. The start (SOS), middle (MOS) and end (EOS) of spring season dates were estimated at an individual tree-level using UAV time series of NDVI and Green Chromatic Coordinate (GCC), with GCC resulting in a clearer and stronger seasonal signal at a tree crown scale. UAV-derived SOS could be predicted more accurately than MOS and EOS, with an accuracy of less than 1 week for deciduous woodland and within 2 weeks for evergreen. The UAV data were used to map phenological events for individual trees across the whole woodland, demonstrating that contrasting canopy phenological events can occur within the extent of a single Landsat pixel. This accounted for the poor relationships found between UAV- and Landsat-derived phenometrics (R2<0.45) in this study. An opportunity is now available to track very fine scale land surface changes over contiguous vegetation communities, information which could improve characterization of vegetation phenology at multiple scales.The Science without Borders program, managed by CAPES-Brazil (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior)

Empirical Approach for Modelling Tree Phenology in Mixed Forests Using Remote Sensing

Phenological events are good indicators of the effects of climate change, since phenological phases are sensitive to changes in environmental conditions. Although several national phenological networks monitor the phenology of different plant species, direct observations can only be conducted on individual trees, which cannot be easily extended over large and continuous areas. Remote sensing has often been applied to model phenology for large areas, focusing mostly on pure forests in which it is relatively easier to match vegetation indices with ground observations. In mixed forests, phenology modelling from remote sensing is often limited to land surface phenology, which consists of an overall phenology of all tree species present in a pixel. The potential of remote sensing for modelling the phenology of individual tree species in mixed forests remains underexplored. In this study, we applied the seasonal midpoint (SM) method with MODIS GPP to model the start of season (SOS) and the end of season (EOS) of six different tree species in Slovenian mixed forests. First, substitute locations were identified for each combination of observation station and plant species based on similar environmental conditions (aspect, slope, and altitude) and tree species of interest, and used to retrieve the remote sensing information used in the SM method after fitting the best of a Gaussian and two double logistic functions to each year of GPP time series. Then, the best thresholds were identified for SOS and EOS, and the results were validated using cross-validation. The results show clearly that the usual threshold of 0.5 is not best in most cases, especially for estimating the EOS. Despite the difficulty in modelling the phenology of different tree species in a mixed forest using remote sensing, it was possible to estimate SOS and EOS with moderate errors as low as <8 days (Fagus sylvatica and Tilia sp.) and <10 days (Fagus sylvatica and Populus tremula), respectively

Knowledge-Based Classification of Grassland Ecosystem Based on Multi-Temporal WorldView-2 Data and FAO-LCCS Taxonomy

Grassland ecosystems can provide a variety of services for humans, such as carbon storage, food production, crop pollination and pest regulation. However, grasslands are today one of the most endangered ecosystems due to land use change, agricultural intensification, land abandonment as well as climate change. The present study explores the performance of a knowledge-driven GEOgraphic-Object—based Image Analysis (GEOBIA) learning scheme to classify Very High Resolution(VHR)imagesfornaturalgrasslandecosystemmapping. Theclassificationwasappliedto a Natura 2000 protected area in Southern Italy. The Food and Agricultural Organization Land Cover Classification System (FAO-LCCS) hierarchical scheme was instantiated in the learning phase of the algorithm. Four multi-temporal WorldView-2 (WV-2) images were classified by combining plant phenology and agricultural practices rules with prior-image spectral knowledge. Drawing on this knowledge, spectral bands and entropy features from one single date (Post Peak of Biomass) were firstly used for multiple-scale image segmentation into Small Objects (SO) and Large Objects (LO). Thereafter, SO were labelled by considering spectral and context-sensitive features from the whole multi-seasonal data set available together with ancillary data. Lastly, the labelled SO were overlaid to LO segments and, in turn, the latter were labelled by adopting FAO-LCCS criteria about the SOs presence dominance in each LO. Ground reference samples were used only for validating the SO and LO output maps. The knowledge driven GEOBIA classifier for SO classification obtained an OA value of 97.35% with an error of 0.04. For LO classification the value was 75.09% with an error of 0.70. At SO scale, grasslands ecosystem was classified with 92.6%, 99.9% and 96.1% of User’s, Producer’s Accuracy and F1-score, respectively. The findings reported indicate that the knowledge-driven approach not only can be applied for (semi)natural grasslands ecosystem mapping in vast and not accessible areas but can also reduce the costs of ground truth data acquisition. The approach used may provide different level of details (small and large objects in the scene) but also indicates how to design and validate local conservation policies

Remote detection of invasive alien species

The spread of invasive alien species (IAS) is recognized as the most severe threat to biodiversity outside of climate change and anthropogenic habitat destruction. IAS negatively impact ecosystems, local economies, and residents. They are especially problematic because once established, they give rise to positive feedbacks, increasing the likelihood of further invasions and spread. The integration of remote sensing (RS) to the study of invasion, in addition to contributing to our understanding of invasion processes and impacts to biodiversity, has enabled managers to monitor invasions and predict the spread of IAS, thus supporting biodiversity conservation and management action. This chapter focuses on RS capabilities to detect and monitor invasive plant species across terrestrial, riparian, aquatic, and human-modified ecosystems. All of these environments have unique species assemblages and their own optimal methodology for effective detection and mapping, which we discuss in detail

Arctic tundra plant phenology and greenness across space and time

The Arctic is warming at twice the rate of the rest of the planet with dramatic consequences for Northern ecosystems. The rapid warming is predicted to cause shifts in plant phenology and increases in tundra vegetation productivity. Changes in phenology and productivity can have knock-on effects on key ecosystem functions. They directly influence plant-herbivore and plant-pollinator interactions creating the potential for mismatches and changes in food web structure, and they alter carbon and nutrient cycling, which in turn influence feedback mechanisms that couple the tundra biome with the global climate system. Improving our understanding of changes in tundra phenology and productivity is therefore critical to projecting not only the future state of Arctic ecosystems, but also the magnitude of potential feedbacks to global climate change. In this thesis, I combine observations from ground-based ecological monitoring, satellites and drones (also known as unmanned aerial vehicles or remotely piloted aircraft systems) to investigate how tundra plant phenology and productivity are changing across space and time, and to test how observational scales influences our ability to detect these changes. Spring plant phenology is tightly linked to temperatures, and advances in spring phenology are one of the most well documented effects of climate change on global biological systems. With rapid and near-ubiquitous Arctic warming, the absence of consistent trends in tundra spring phenology among sites suggests that additional environmental factors may exert important controls on tundra plant phenology. Indeed, further to temperature, snowmelt and sea-ice have been reported to strongly influence tundra phenology. Yet, the relative influence of these three factors has yet to be evaluated in a single cross-site analysis. In Chapter 2, I tested the importance of local average spring temperatures, local snowmelt and the timing of the drop in regional spring sea-ice extent as controls on variation in spring leaf out and flowering of 14 plant species from long-term records at four coastal sites in Arctic Alaska, Canada and Greenland. I found that spring phenology was best explained by snowmelt and spring temperature. In contrast to previous studies, sea-ice did not predict spring plant phenology at these study sites. This contrasting finding is likely explained by differences in the scale of the sea-ice measures employed. While many previous studies used descriptors of circum-polar sea-ice conditions that serve as aggregate measures for global weather conditions, I tested for the indirect effects of sea-ice conditions at a regional scale. My findings (re)emphasize the importance of snowmelt timing for tundra spring plant phenology and therefore highlight the localised nature of some of the key drivers of tundra vegetation change. Discrepancies between conventional scales of observation and underlying ecological processes could limit our ability to explain variation in tundra plant phenology and vegetation productivity. In the remote biome, ground-based monitoring is logistically challenging and restricted to comparably few sites and small plot sizes. Multispectral satellite observations cover the whole biome but are coarse in scale (tens of meters to kilometres) and uncertainties persist in how trends in vegetation indices like the Normalised Differential Vegetation Index (NDVI) relate to in situ ecological processes. Recent advances in drone technologies allow for the collection of multispectral fine-grain imagery at landscape level and have the potential to bridge the gap in observational scales. However, collecting high-quality multispectral drone imagery that is comparable across sensors, space and time remains challenging particularly when operating in extreme environments such as the tundra. In Chapter 3 of this thesis, I discuss the key error sources associated with solar angle, weather conditions, geolocation and radiometric calibration and estimate their relative contributions to the uncertainty of landscape level NDVI measurements at Qikiqtaruk in the Yukon Territory of Canada. My findings show that these errors can lead to uncertainties of greater than ± 10% in peak season NDVI, but also demonstrate they can be accounted for by improved flight planning, meta-data collection, ground control point deployment, use of reflectance targets and quality control. Satellite data suggest that vegetation productivity in the Arctic tundra has been increasing in recent decades: the tundra is greening. However, the observed trends show a lot of variation: although many parts of the tundra are greening, others show reductions in vegetation productivity (sometimes known as browning), and the satellite-based trends do not always match in situ records of change. Our ability to explain this variation has been limited by the coarse grain sizes of the satellite observations. In Chapter 4, I combined time-series of multispectral drone and satellite imagery (Sentinel 2 and MODIS) of coastal tundra plots at my focal study site Qikiqtaruk to quantify the correspondence among satellite and drone observations of vegetation productivity change across spatial scales. My findings show that NDVI estimates of tundra productivity collected with both platform types correspond well at landscape scales (10 m – 100 m) but demonstrate that the majority of spatial variation in NDVI at the study sites occurs at distances below 10 m and is therefore not captured by the latest generation of publicly available satellite products, like those of the Sentinel 2 satellites. I observed strong differences in mean estimates and variation of vegetation productivity between the dominant vegetation types at the field site. When comparing greening observations over two years, I detected differences in the amount of variation amongst years and a within-season decline in variation towards peak growing season for both years. These results suggest that not only the timing, but also the heterogeneity of tundra landscape phenology can vary within and among years, and if lowered by warming could alter trophic interactions between species. The findings presented in this thesis highlight the importance of the localised processes that influence large-scale patterns and trends in tundra vegetation phenology and productivity. Localised snowmelt timing best explained variation in tundra plant phenology and drone imagery revealed meter-scale heterogeneity in tundra productivity. Research that identifies the most relevant scales at which key biological processes occur is therefore critical to improving our forecasts of ecosystem change in the tundra and resulting feedbacks on the global climate system