Estimating Biomass and Nitrogen Amount of Barley and Grass Using UAV and Aircraft Based Spectral and Photogrammetric 3D Features

201

Drone-based spectral and 3D remote sensing applications for forestry and agriculture

Practising sustainable agriculture and forestry requires information on the state of forests and crops to support management. In precision agriculture, crops are observed in order to treat them precisely in the right place and at the right time, saving both production costs and the environment. Similarly, in forests, information on the composition and state of forest health are crucial to enable their sustainable management. In particular, climate-change-driven insect pests have increased, but economic and ecological losses can be reduced by the right actions if up-to-date and precise information on the health of forests is available. In recent years, drones with cameras have evolved into a flexible way to collect remote sensing data locally. Spectral cameras provide accurate information about the reflection properties of objects, and photogrammetric methods also provide a cost-effective way to collect three-dimensional (3D) data from an object. The objective of this work was to develop and assess drone-based 3D and spectral remote sensing techniques to classify the health status of individual trees and to estimate crop biomass, various biochemical parameters such as nitrogen content, and grass-feeding quality. The work developed a processing chain in which spectral and 3D features were extracted from remote sensing data. Then, combining the features with observations and reference measurements collected from plants, machine learning models were developed for tree health classification and estimation of crop-related parameters. The effects of different factors related to data collection and processing on classification and estimation accuracies were studied in order to generate knowledge on optimal sensors and methods. In general, radiometric corrections, spectral resolution, and the combined use of spectral and 3D features improved classification and estimation accuracies. However, the optimal sensors as well as the data collection and processing methods depend on the different applications and their accuracy requirements. This work was the first to demonstrate the ability of drone hyperspectral data to map the health status of a forest by classifying individual trees infested by bark beetles. The results of the work also showed that drone-based mapping offers a great tool to estimate agricultural crop parameters which can be applied to the optimization of various precision agriculture tasks.Kestävän maa- ja metsätalouden harjoittaminen vaatii tietoa metsien ja viljelykasvien tilasta päätöksenteon tueksi. Täsmämaataloudessa viljelykasveja havainnoidaan, jotta viljelytoimenpiteet voidaan kohdistaa oikeaan paikkaan ja oikea-aikaisesti säästäen sekä tuotantokustannuksia että ympäristöä. Metsissä tieto metsien terveydentilasta on tärkeää, jotta voidaan hillitä metsätuhojen leviämistä. Erityisesti hyönteistuhot ovat lisääntyneet voimakkaasti ilmastonmuutoksen vauhdittamana, mutta taloudellisia ja ekologisia tappiota voidaan vähentää oikeilla toimenpiteillä, jos on olemassa ajantasaisesta tietoa metsien terveydentilasta. Dronet ja niihin asennettavat kamerat ovat kehittyneet viime vuosina joustavaksi tavaksi kerätä kaukokartoitusaineistoa paikallisesti. Spektrikameroilla saadaan tarkkaa tietoa kohteen heijastusominaisuuksista, ja fotogrammetriset menetelmät mahdollistavat myös kustannustehokkaan tavan kerätä kohteesta kolmiulotteista (3D) tietoa. Tämän työn tavoitteena oli kehittää näihin aineistoihin nojautuen kaukokartoitusmenetelmiä yksittäisten puiden terveydentilan luokitteluun sekä viljelykasvien biomassan, erilaisten biokemiallisten parametrien, kuten typpipitoisuuden sekä nurmen ruokintalaadun, kuten D-arvon estimointiin. Työssä kehitettiin prosessointiketju, jossa kaukokartoitusaineistoista irrotettiin spektri- ja 3D-piirteitä, yhdistettiin ne kasveista kerättyihin havaintoihin ja mittauksiin sekä muodostettiin koneoppimismalleja puiden luokittelua ja viljelykasveihin liittyvien parametrien estimointia varten. Työssä verrattiin useiden aineistonkeräykseen ja -prosessointiin liittyvien tekijöiden vaikutuksia luokittelu- ja estimointitulosten tarkkuuteen optimaalisten menetelmien löytämiseksi. Esimerkiksi spektri- ja 3D-piirteiden hyödyntäminen yhdessä sekä radiometriset korjaukset paransivat yleisesti luokittelu- ja estimointitarkkuuksia. Optimaaliset sensorit sekä aineistonkeräys- ja käsittelytavat riippuvat kuitenkin eri sovelluksista ja niiden tarkkuusvaatimuksista. Työssä osoitettiin ensimmäistä kertaa dronesta kerätyn hyperspektrisen aineiston kyvykkyys metsän terveydentilan havainnoinnissa luokittelemalla kuuset kolmeen luokkaan kirjanpainajan aiheuttaman tuhon perusteella. Työn tulokset myös osoittivat drone-pohjaisen kartoituksen kyvyn estimoida erilaisia viljelykasvien parametreja, joita voidaan edelleen soveltaa suunniteltaessa esimerkiksi lisälannoitusta tai säilörehun optimaalista korjuuaikaa

Investigating the Potential of a Newly Developed UAV-Mounted VNIR/SWIR Imaging System for Monitoring Crop Traits-A Case Study for Winter Wheat

UAV-based multispectral multi-camera systems are widely used in scientific research for non-destructive crop traits estimation to optimize agricultural management decisions. These systems typically provide data from the visible and near-infrared (VNIR) domain. However, several key absorption features related to biomass and nitrogen (N) are located in the short-wave infrared (SWIR) domain. Therefore, this study investigates a novel multi-camera system prototype that addresses this spectral gap with a sensitivity from 600 to 1700 nm by implementing dedicated bandpass filter combinations to derive application-specific vegetation indices (VIs). In this study, two VIs, GnyLi and NRI, were applied using data obtained on a single observation date at a winter wheat field experiment located in Germany. Ground truth data were destructively sampled for the entire growing season. Likewise, crop heights were derived from UAV-based RGB image data using an improved approach developed within this study. Based on these variables, regression models were derived to estimate fresh and dry biomass, crop moisture, N concentration, and N uptake. The relationships between the NIR/SWIR-based VIs and the estimated crop traits were successfully evaluated (R-2: 0.57 to 0.66). Both VIs were further validated against the sampled ground truth data (R-2: 0.75 to 0.84). These results indicate the imaging system's potential for monitoring crop traits in agricultural applications, but further multitemporal validations are needed.Peer reviewe

Investigating the Potential of a Newly Developed UAV-Mounted VNIR/SWIR Imaging System for Monitoring Crop Traits—A Case Study for Winter Wheat

UAV-based multispectral multi-camera systems are widely used in scientific research for non-destructive crop traits estimation to optimize agricultural management decisions. These systems typically provide data from the visible and near-infrared (VNIR) domain. However, several key absorption features related to biomass and nitrogen (N) are located in the short-wave infrared (SWIR) domain. Therefore, this study investigates a novel multi-camera system prototype that addresses this spectral gap with a sensitivity from 600 to 1700 nm by implementing dedicated bandpass filter combinations to derive application-specific vegetation indices (VIs). In this study, two VIs, GnyLi and NRI, were applied using data obtained on a single observation date at a winter wheat field experiment located in Germany. Ground truth data were destructively sampled for the entire growing season. Likewise, crop heights were derived from UAV-based RGB image data using an improved approach developed within this study. Based on these variables, regression models were derived to estimate fresh and dry biomass, crop moisture, N concentration, and N uptake. The relationships between the NIR/SWIR-based VIs and the estimated crop traits were successfully evaluated (R2: 0.57 to 0.66). Both VIs were further validated against the sampled ground truth data (R2: 0.75 to 0.84). These results indicate the imaging system’s potential for monitoring crop traits in agricultural applications, but further multitemporal validations are needed

Investigating the Potential of a Newly Developed UAV-Mounted VNIR/SWIR Imaging System for Monitoring Crop Traits—A Case Study for Winter Wheat

UAV-based multispectral multi-camera systems are widely used in scientific research for non-destructive crop traits estimation to optimize agricultural management decisions. These systems typically provide data from the visible and near-infrared (VNIR) domain. However, several key absorption features related to biomass and nitrogen (N) are located in the short-wave infrared (SWIR) domain. Therefore, this study investigates a novel multi-camera system prototype that addresses this spectral gap with a sensitivity from 600 to 1700 nm by implementing dedicated bandpass filter combinations to derive application-specific vegetation indices (VIs). In this study, two VIs, GnyLi and NRI, were applied using data obtained on a single observation date at a winter wheat field experiment located in Germany. Ground truth data were destructively sampled for the entire growing season. Likewise, crop heights were derived from UAV-based RGB image data using an improved approach developed within this study. Based on these variables, regression models were derived to estimate fresh and dry biomass, crop moisture, N concentration, and N uptake. The relationships between the NIR/SWIR-based VIs and the estimated crop traits were successfully evaluated (R2: 0.57 to 0.66). Both VIs were further validated against the sampled ground truth data (R2: 0.75 to 0.84). These results indicate the imaging system’s potential for monitoring crop traits in agricultural applications, but further multitemporal validations are needed

Mehitamata õhusõiduki rakendamine põllukultuuride saagikuse ja maa harimisviiside tuvastamisel

A Thesis for applying for the degree of Doctor of Philosophy in Environmental Protection.Väitekiri filosoofiadoktori kraadi taotlemiseks keskkonnakaitse erialal.This thesis aims to examine how machine learning (ML) technologies have aided significant advancements in image analysis in the area of precision agriculture. These multimodal computing technologies extend the use of machine learning to a broader spectrum of data collecting and selection for the advancement of agricultural practices (Nawar et al., 2017) These techniques will assist complicated cropping systems with more informed decisions with less human intervention, and provide a scalable framework for incorporating expert knowledge of the PA system. (Chlingaryan et al., 2018). Complexity, on the other hand, can be seen as a disadvantage in crop trials, as machine learning models require training/testing databases, limited areas with insignificant sampling sizes, time and space-specificity, and environmental factor interventions, all of which complicate parameter selection and make using a single empirical model for an entire region impractical. During the early stages of writing this thesis, we used a relatively traditional machine learning method to address the regression problem of crop yield and biomass prediction [(i.e., random forest regression (RFR), support vector regression (SVR), and artificial neural network (ANN)] to predicted dry matter (DM) yields of red clover. It obtained favourable results, however, the choosing of hyperparameters, the lengthy algorithms selection process, data cleaning, and redundant collinearity issues significantly limited the way of the machine learning application. We will further discuss the recent trend of automated machine learning (AutoML) that has been driving further significant technological innovation in the application of artificial intelligence from its automated algorithm selection and hyperparameter optimization of the deployable pipeline model for unravelling substance problems. However, a present knowledge gap exists in the integration of machine learning (ML) technology with unmanned aerial systems (UAS) and hyperspectral-based imaging data categorization and regression applications. In this thesis, we explored a state-of-the-art (SOTA) and entirely open-source AutoML framework, Auto-sklearn, which was built on one of the most frequently used machine learning systems, Scikit-learn. It was integrated with two unique AutoML visualization tools to examine the recognition and acceptance of multispectral vegetation indices (VI) data collected from UAS and hyperspectral narrow-band VIs across a varied spectrum of agricultural management practices (AMP). These procedures incorporate soil tillage method (STM), cultivation method (CM), and manure application (MA), and are classified as four-crop combination fields (i.e., red clover-grass mixture, spring wheat, pea-oat mixture, and spring barley). Additionally, they have not been thoroughly evaluated and lack characteristics that are accessible in agriculture remote sensing applications. This thesis further explores the existing gaps in the knowledge base for several critical crop categories and cultivation management methods referring to biomass and yield analysis, as well as to gain a better understanding of the potential for remotely sensed solutions to field-based and multifunctional platforms to meet precision agriculture demands. To overcome these knowledge gaps, this research introduces a rapid, non-destructive, and low-cost framework for field-based biomass and grain yield modelling, as well as the identification of agricultural management practices. The results may aid agronomists and farmers in establishing more accurate agricultural methods and in monitoring environmental conditions more effectively.Doktoritöö eesmärk oli uurida, kuidas masinõppe (MÕ) tehnoloogiad võimaldavad edusamme täppispõllumajanduse valdkonna pildianalüüsis. Multimodaalsed arvutustehnoloogiad laiendavad masinõppe kasutamist põllumajanduses andmete kogumisel ja valimisel (Nawar et al., 2017). Selline täpsemal informatsioonil põhinev tehnoloogia võimaldab keerukate viljelussüsteemide puhul teha otsuseid inimese vähema sekkumisega, ja loob skaleeritava raamistiku täppispõllumajanduse jaoks (Chlingaryan et al., 2018). Põllukultuuride katsete korral on komplekssete masinõppemudelite kasutamine keerukas, sest alad on piiratud ning valimi suurus ei ole piisav; vaja on testandmebaase, kindlaid aja- ja ruumitingimusi ning keskkonnategureid. See komplitseerib parameetrite valikut ning muudab ebapraktiliseks ühe empiirilise mudeli kasutamise terves piirkonnas. Siinse uurimuse algetapis rakendati suhteliselt traditsioonilist masinõppemeetodit, et lahendada saagikuse ja biomassi prognoosimise regressiooniprobleem (otsustusmetsa regression, tugivektori regressioon ja tehisnärvivõrk) punase ristiku prognoositava kuivaine saagikuse suhtes. Saadi sobivaid tulemusi, kuid hüperparameetrite valimine, pikk algoritmide valimisprotsess, andmete puhastamine ja kollineaarsusprobleemid takistasid masinõpet oluliselt. Automatiseeritud masinõppe (AMÕ) uusimate suundumustena rakendatakse tehisintellekti, et lahendada põhiprobleemid automatiseeritud algoritmi valiku ja rakendatava pipeline-mudeli hüperparameetrite optimeerimise abil. Seni napib teadmisi MÕ tehnoloogia integreerimiseks mehitamata õhusõidukite ning hüperspektripõhiste pildiandmete kategoriseerimise ja regressioonirakendustega. Väitekirjas uuriti nüüdisaegset ja avatud lähtekoodiga AMÕ tehnoloogiat Auto-sklearn, mis on ühe enimkasutatava masinõppesüsteemi Scikit-learn edasiarendus. Süsteemiga liideti kaks unikaalset AMÕ visualiseerimisrakendust, et uurida mehitamata õhusõidukiga kogutud andmete multispektraalsete taimkatteindeksite ja hüperspektraalsete kitsaribaandmete taimkatteindeksite tuvastamist ja rakendamist põllumajanduses. Neid võtteid kasutatakse mullaharimisel, kultiveerimisel ja sõnnikuga väetamisel nelja kultuuriga põldudel (punase ristiku rohusegu, suvinisu, herne-kaera segu, suvioder). Neid ei ole põhjalikult hinnatud, samuti ei hõlma need omadusi, mida kasutatatakse põllumajanduses kaugseire rakendustes. Uurimus käsitleb biomassi ja saagikuse seni uurimata analüüsivõimalusi oluliste põllukultuuride ja viljelusmeetodite näitel. Hinnatakse ka kaugseirelahenduste potentsiaali põllupõhiste ja multifunktsionaalsete platvormide kasutamisel täppispõllumajanduses. Uurimus tutvustab kiiret, keskkonna suhtes kahjutut ja mõõduka hinnaga tehnoloogiat põllupõhise biomassi ja teraviljasaagi modelleerimiseks, et leida sobiv viljelusviis. Töö tulemused võimaldavad põllumajandustootjatel ja agronoomidel tõhusamalt valida põllundustehnoloogiaid ning arvestada täpsemalt keskkonnatingimustega.Publication of this thesis is supported by the Estonian University of Life Scieces and by the Doctoral School of Earth Sciences and Ecology created under the auspices of the European Social Fund

Remote sensing image fusion on 3D scenarios: A review of applications for agriculture and forestry

Three-dimensional (3D) image mapping of real-world scenarios has a great potential to provide the user with a more accurate scene understanding. This will enable, among others, unsupervised automatic sampling of meaningful material classes from the target area for adaptive semi-supervised deep learning techniques. This path is already being taken by the recent and fast-developing research in computational fields, however, some issues related to computationally expensive processes in the integration of multi-source sensing data remain. Recent studies focused on Earth observation and characterization are enhanced by the proliferation of Unmanned Aerial Vehicles (UAV) and sensors able to capture massive datasets with a high spatial resolution. In this scope, many approaches have been presented for 3D modeling, remote sensing, image processing and mapping, and multi-source data fusion. This survey aims to present a summary of previous work according to the most relevant contributions for the reconstruction and analysis of 3D models of real scenarios using multispectral, thermal and hyperspectral imagery. Surveyed applications are focused on agriculture and forestry since these fields concentrate most applications and are widely studied. Many challenges are currently being overcome by recent methods based on the reconstruction of multi-sensorial 3D scenarios. In parallel, the processing of large image datasets has recently been accelerated by General-Purpose Graphics Processing Unit (GPGPU) approaches that are also summarized in this work. Finally, as a conclusion, some open issues and future research directions are presented.European Commission 1381202-GEU PYC20-RE-005-UJA IEG-2021Junta de Andalucia 1381202-GEU PYC20-RE-005-UJA IEG-2021Instituto de Estudios GiennesesEuropean CommissionSpanish Government UIDB/04033/2020DATI-Digital Agriculture TechnologiesPortuguese Foundation for Science and Technology 1381202-GEU FPU19/0010

Application of UAV multispectral imaging for determining the characteristics of maize vegetation

Received: February 1st, 2023 ; Accepted: April 25th, 2023 ; Published: May 10th, 2023 ; Correspondence: [email protected] in forage maize (Zea mays L.) cultivation for livestock feed has grown in northern conditions. In addition, it is important to develop methods and tools to monitor crop development and other characteristics of the crop. For these purposes UAVs are very efficient and versatile tools. UAVs can be equipped with a variety of sensors like lidar or different types of cameras. Several studies have been conducted where data collected by UAVs are used to estimate different crop properties like yield and biomass. In this research, a forage maize field experiment was studied to examine how well the aerial multispectral data correlated with the different properties of the vegetation. The field test site is located in Helsinki, Finland. A multispectral camera (MicaSense Rededge 3) was used to take images from five spectral bands (Red, Green, Blue, Rededge and NIR). All the images were processed with Pix4D software to generate orthomosaic images. Several vegetation indices were calculated from the five spectral bands. During the growing season, crop height, chlorophyll content, leaf area index (LAI), fresh and dry matter biomass were measured from the vegetation. From the five spectral bands, Rededge had the highest correlation with fresh biomass (R2 = 0.273). The highest correlation for a vegetation index was found between NDRE and chlorophyll content (R2 = 0.809). A multiple linear regression (MLR) model using selected spectral bands and vegetation indices as inputs showed high correlations with the field measurements

Unmanned aerial vehicles (UAVs) for multi-temporal crop surface modelling. A new method for plant height and biomass estimation based on RGB-imaging

Data collection with unmanned aerial vehicles (UAVs) fills a gap on the observational scale in re-mote sensing by delivering high spatial and temporal resolution data that is required in crop growth monitoring. The latter is part of precision agriculture that facilitates detection and quan-tification of within-field variability to support agricultural management decisions such as effective fertilizer application. Biophysical parameters such as plant height and biomass are monitored to describe crop growth and serve as an indicator for the final crop yield. Multi-temporal crop surface models (CSMs) provide spatial information on plant height and plant growth. This study aims to examine whether (1) UAV-based CSMs are suitable for plant height modelling, (2) the derived plant height can be used for biomass estimation, and (3) the combination of plant height and vegetation indices has an added value for biomass estimation. To achieve these objectives, UAV-flight campaigns were carried out with a red-green-blue (RGB) camera over controlled field experiments on three study sites, two for summer barley in Western Germany and one for rice in Northeast China. High-resolution, multi-temporal CSMs were derived from the images by using computer vision software following the structure from motion (SfM) approach. The results show that plant height and plant growth can be accurately modelled with UAV-based CSMs from RGB imaging. To maximise the CSMs’ quality, accurate flight planning and well-considered data collection is necessary. Furthermore, biomass is successfully estimated from the derived plant height, with the restriction that results are based on a single-year dataset and thus require further validation. Nevertheless, plant height shows robust estimates in comparison with various vegetation indices. As for biomass estimation in early growth stages additional po-tential is found in exploiting visible band vegetation indices from UAV-based red-green-blue (RGB) imaging. However, the results are limited due to the use of uncalibrated images. Combining visible band vegetation indices and plant height does not significantly improve the performance of the biomass models. This study demonstrates that UAV-based RGB imaging delivers valuable data for productive crop monitoring. The demonstrated results for plant height and biomass estimation open new possi-bilities in precision agriculture by capturing in-field variability

Multitemporaalisen hyper- ja multispektrisen UAV kuvauksen käyttö kuusen kaarnakuoriaistuhoissa

Various biotic and abiotic stresses are threatening forests. Modern remote sensing technologies provide powerful means for monitoring forest health, and provide a sustainable basis for forest management and protection. The objective of this study was to develop unmanned aerial vehicle (UAV) based spectral remote sensing technologies for tree health assessment, particularly, for detecting the European spruce bark beetle (Ips typographus L.) attacks. Our focus was to study the early detection of bark beetle attack, i.e. the “green attack” phase. This is a difficult remote sensing task as there does not exist distinct symptoms that can be observed by the human eye. A test site in a Norway spruce (Picea abies (L.) Karst.) dominated forest was established in Southern-Finland in summer 2019. It had an emergent bark beetle outbreak and it was also suffering from other stress factors, especially the root and butt rot (Heterobasidion annosum (Fr.) Bref. s. lato). Altogether seven multitemporal hyper- and multispectral UAV remote sensing datasets were captured from the area in August to October 2019. Firstly, we explored deterioration of tree health and development of spectral symptoms using a time series of UAV hyperspectral imagery. Secondly, we trained assessed a machine learning model for classification of spruce health into classes of “bark beetle green attack”, “root-rot”, and “healthy”. Finally, we demonstrated the use of the model in tree health mapping in a test area. Our preliminary results were promising and indicated that the green attack phase could be detected using the accurately calibrated spectral image data.Peer reviewe