High resolution thermal and multispectral UAV imagery for precision assessment of apple tree response to water stress

UMR AGAP - équipe AFEF - Architecture et fonctionnement des espèces fruitières(Edited by Pablo Gonzalez-de-Santos and Angela Ribeiro)This manuscript presents a comprehensive methodology to obtain Thermal, Visible and Near Infrared ortho-mosaics, as a previous step for the further image-based assessment of response to water stress of an experimental apple tree orchard. Using this methodology, multi-temporal ortho-mosaics of the field plot were created and accuracy of ortho-rectification and geo-location computed. Unmanned aerial vehicle (UAV) flights were performed on an irrigated apple tree orchard located in Southern France. The 6400 m² plot was composed of 520 apple trees which were disposed in 10 rows. In this field set-up, five well irrigated rows alternated with five rows submitted to progressive summer water constraints. For remote image acquisition, on 4th July, 19th July, 1st August and 6th September UAV flights with three cameras onboard (thermal, visible and near infrared) were performed at solar noon. On 1st August, five successive UAV flights were carried out at 8, 10, 12, 14 and 16 h (solar time). By using selfdeveloped software, frames were automatically extracted from the recorded thermal video and turned in the right image format. The temperature of four different targets (hot, cold, wet and dry bare soil) was continuously measured by the IR120 thermoradiometers during each flight, for radiometric calibration purpose. Based each on thirty images, all ortho-mosaics were successfully obtained. As high spatial resolution imagery requires high precision geo-location, and the root mean squared error (RMSE) of each ortho-mosaic positioning was calculated in order to assess its spatial accuracy. RMSE values were less than twice the pixel size in every case, which allowed a precise overlapping of the mosaics created. Canopy temperature data extracted from thermal images for showed significantly higher temperatures in water stressed trees compared to well irrigated, difference being related to severity of water stress. Thanks to the ultrahigh resolution of remote images obtained (<0.1m spatial resolution for thermal infrared images), and beyond its capacity to delineate efficiently each individual tree, the methodology presented here will also make it possible the analysis of intra-canopy variations and the accurate calculation of vegetation and water stress indices

Acquisition d'images thermiques par drone : corrections radiométriques à partir de données terrain

Thermal images have many applications in agronomy, including crop water stress status assessment. Nowadays, the miniaturization of thermal cameras allows installing them onboard the Unmanned Aerial Vehicles (UAV), but this miniaturization leads to some difficulties: the miniaturized thermal cameras have no temperature control system of their sensor. The instability of the miniaturized camera makes a high drift in the acquisition of temperature data so that acquired thermal images don't fit the real temperature of the studying object, so data have to be continuously corrected. We need to have stable reference on field in order to compute the actual temperature value. In this article we present a method for radiometric correction of UAV remote sensed thermal images. We have implemented a device in order to retrieve ground temperature measurements. This device is composed with four targets (cold, hot, dry soil, wet soil) which measured continuously the target temperature thanks to IR120 (Campbell ®) radio-thermometer. A meteorological station is included in this ground system and acquires air temperature and moisture, solar radiation, wind speed and direction every 10 seconds. The images are radiometrically corrected by linear regression from on ground thermal data collected. Corrected images have been compared with mean canopy surface temperature of a sample of 10 trees measured with radio-thermometers. The results showed a good link between data from on ground radio-thermometer and data from thermal camera after radiometric correction. We can conclude that images obtained by this method are of sufficient quality to be used in vegetation water stress studies. (Résumé d'auteur

Time-intensive geoelectrical monitoring under winter wheat

Several studies have explored the potential of electrical resistivity tomography to monitor changes in soil moisture associated with the root water uptake of different crops. Such studies usually use a set of limited below-ground measurements throughout the growth season but are often unable to get a complete picture of the dynamics of the processes. With the development of high-throughput phenotyping platforms, we now have the capability to collect more frequent above-ground measurements, such as canopy cover, enabling the comparison with below-ground data. In this study hourly DC resistivity data were collected under the Field Scanalyzer platform at Rothamsted Research with different winter wheat varieties and nitrogen treatments in 2018 and 2019. Results from both years demonstrate the importance of applying the temperature correction to interpret hourly electrical conductivity (EC) data. Crops which received larger amounts of nitrogen showed larger canopy cover and more rapid changes in EC, especially during large rainfall events. The varieties showed contrasted heights although this does not appear to have influenced EC dynamics. The daily cyclic component of the EC signal was extracted by decomposing the time series. A shift in this daily component was observed during the growth season. For crops with appreciable difference in canopy cover, high frequency DC resistivity monitoring was able to distinguish the different below-ground behaviors. The results also highlight how coarse temporal sampling may affect interpretation of resistivity data from crop monitoring studies

Field phenotyping for African crops: overview and perspectives

Improvements in crop productivity are required to meet the dietary demands of the rapidly-increasing African population. The development of key staple crop cultivars that are high-yielding and resilient to biotic and abiotic stresses is essential. To contribute to this objective, high-throughput plant phenotyping approaches are important enablers for the African plant science community to measure complex quantitative phenotypes and to establish the genetic basis of agriculturally relevant traits. These advances will facilitate the screening of germplasm for optimum performance and adaptation to low-input agriculture and resource-constrained environments. Increasing the capacity to investigate plant function and structure through non-invasive technologies is an effective strategy to aid plant breeding and additionally may contribute to precision agriculture. However, despite the significant global advances in basic knowledge and sensor technology for plant phenotyping, Africa still lags behind in the development and implementation of these systems due to several practical, financial, geographical and political barriers. Currently, field phenotyping is mostly carried out by manual methods that are prone to error, costly, labor-intensive and may come with adverse economic implications. Therefore, improvements in advanced field phenotyping capabilities and appropriate implementation are key factors for success in modern breeding and agricultural monitoring. In this review, we provide an overview of the current state of field phenotyping and the challenges limiting its implementation in some African countries. We suggest that the lack of appropriate field phenotyping infrastructures is impeding the development of improved crop cultivars and will have a detrimental impact on the agricultural sector and on food security. We highlight the prospects for integrating emerging and advanced low-cost phenotyping technologies into breeding protocols and characterizing crop responses to environmental challenges in field experimentation. Finally, we explore strategies for overcoming the barriers and maximizing the full potential of emerging field phenotyping technologies in African agriculture. This review paper will open new windows and provide new perspectives for breeders and the entire plant science community in Africa.BBSRC: BB/P016855/

Machine learning methods for automatic segmentation of images of field-and glasshouse-based plants for high-throughput phenotyping

Image segmentation is a fundamental but critical step for achieving automated high- throughput phenotyping. While conventional segmentation methods perform well in homogenous environments, the performance decreases when used in more complex environments. This study aimed to develop a fast and robust neural-network-based segmentation tool to phenotype plants in both field and glasshouse environments in a high-throughput manner. Digital images of cowpea (from glasshouse) and wheat (from field) with different nutrient supplies across their full growth cycle were acquired. Image patches from 20 randomly selected images from the acquired dataset were transformed from their original RGB format to multiple color spaces. The pixels in the patches were annotated as foreground and background with a pixel having a feature vector of 24 color properties. A feature selection technique was applied to choose the sensitive features, which were used to train a multilayer perceptron network (MLP) and two other traditional machine learning models: support vector machines (SVMs) and random forest (RF). The performance of these models, together with two standard color-index segmentation techniques (excess green (ExG) and excess green–red (ExGR)), was compared. The proposed method outperformed the other methods in producing quality segmented images with over 98%-pixel classification accuracy. Regression models developed from the different segmentation methods to predict Soil Plant Analysis Development (SPAD) values of cowpea and wheat showed that images from the proposed MLP method produced models with high predictive power and accuracy comparably. This method will be an essential tool for the development of a data analysis pipeline for high-throughput plant phenotyping. The proposed technique is capable of learning from different environmental conditions, with a high level of robustness

Modeling the spatial-spectral characteristics of plants for nutrient status identification using hyperspectral data and deep learning methods

Sustainable fertilizer management in precision agriculture is essential for both economic and environmental reasons. To effectively manage fertilizer input, various methods are employed to monitor and track plant nutrient status. One such method is hyperspectral imaging, which has been on the rise in recent times. It is a remote sensing tool used to monitor plant physiological changes in response to environmental conditions and nutrient availability. However, conventional hyperspectral processing mainly focuses on either the spectral or spatial information of plants. This study aims to develop a hybrid convolution neural network (CNN) capable of simultaneously extracting spatial and spectral information from quinoa and cowpea plants to identify their nutrient status at different growth stages. To achieve this, a nutrient experiment with four treatments (high and low levels of nitrogen and phosphorus) was conducted in a glasshouse. A hybrid CNN model comprising a 3D CNN (extracts joint spectral-spatial information) and a 2D CNN (for abstract spatial information extraction) was proposed. Three pre-processing techniques, including second-order derivative, standard normal variate, and linear discriminant analysis, were applied to selected regions of interest within the plant spectral hypercube. Together with the raw data, these datasets were used as inputs to train the proposed model. This was done to assess the impact of different pre-processing techniques on hyperspectral-based nutrient phenotyping. The performance of the proposed model was compared with a 3D CNN, a 2D CNN, and a Hybrid Spectral Network (HybridSN) model. Effective wavebands were selected from the best-performing dataset using a greedy stepwise-based correlation feature selection (CFS) technique. The selected wavebands were then used to retrain the models to identify the nutrient status at five selected plant growth stages. From the results, the proposed hybrid model achieved a classification accuracy of over 94% on the test dataset, demonstrating its potential for identifying nitrogen and phosphorus status in cowpea and quinoa at different growth stages

Phénotypage haut débit par imagerie multispectrale au verger: Etude du déterminisme génétique de la réponse à la contrainte hydrique d'une population d'hybrides de pommier (<em>Malus x domestica</em> Borkh.)

In the context of foreseen climate changes and higher evaporative demand in crops, this PhD work is aiming at assessing the apple tree variability in response to hydric constraints. This could be of great interest for future breeding programs in this species, either in respect to temporary abiotic stress tolerance, or looking at a better water use efficiency. Phenotyping methods presently available for evaluating the plant response to hydric constraints do not allow the high-throughput that could be compatible with simultaneous comparison of a high number of individuals. As a consequence, this thesis focused on the application of airborne multispectral imagery as a tool for high-throughput field phenotyping, and it assessed the relevancy of this method for quantitative analysis of response to drought and further dissection of associated genetic determinisms. Trials were realized on a segregating hybrid apple population (Starkrimson x Granny Smith cross), 4-year old at the beginning of study, installed in the South of France. Images acquired in visible, near infrared and thermal infrared, with a spatial resolution close to 30cm, were the calculation basis of various vegetation and stress indices, allowing estimation of individual tree vigor and transpiration. Linear mixed models taking account of six flight dates, covering two acquisition campaigns and including two contrasted irrigation regimes over the apple population, made it possible to highlight the heritability of phenotypic indices and to perform numerous QTLs mapping. Eighteen QTLs were revealed independently from the acquisition date. Out of this QTLs set, a further analysis on 4 of them, which were adaptative i.e. expressed in stress conditions, allowed first identification of putative candidate genes potentially involved in the early stomatal closure.Dans le contexte des changements climatiques prévisibles et d'une demande évaporative accrue vis à vis des cultures, ce travail de thèse vise à évaluer la variabilité génétique du pommier pour sa réponse à la contrainte hydrique. Cette évaluation est d'un intérêt certain pour l'amélioration future de l'espèce, qu'il s'agisse de mieux tolérer le stress pouvant résulter d'une sécheresse temporaire du sol, ou d'une efficience accrue d'utilisation de l'eau. Les méthodes de phénotypage actuellement disponibles pour l'étude de la réponse à la contrainte hydrique ne permettent pas un haut débit de mesure compatible avec les comparaisons simultanées entre de nombreux individus. En conséquence, ce travail de thèse s'est focalisé sur l'application de l'imagerie multi-spectrale aéroportée comme outil de phénotypage à haut débit en condition de plein champ et sur l'évaluation de sa pertinence pour l'analyse quantitative des réponses à la contrainte hydrique et la recherche des déterminismes génétiques associés. Les essais ont été réalisés sur une population hybride de pommiers (croisement Starkrimson x Granny Smith) âgée de 4 ans au début de l'étude, implantée dans le sud de la France. A partir d'images acquises dans les longueurs d'onde du visible, du proche infrarouge et de l'infrarouge thermique, à une résolution spatiale proche de 30cm, différents indices de végétation et de stress ont été calculés permettant d'apprécier la vigueur des arbres et leur transpiration. La prise en compte, dans des modèles linéaires mixtes, de six dates d'expérimentation, couvrant deux campagnes d'acquisitions, et de régimes d'irrigation différenciés sur la population a permis de mettre en évidence l'héritabilité des indices phénotypiques et conduit à la détection de nombreux QTLs. 18 QTLs se sont révélés indépendants de la date d'acquisition. Pour quatre de ces QTLs, dits adaptatifs, c'est-à-dire exprimés en conditions de stress, un certain nombre de gènes candidats potentiellement impliqués dans les réponses précoces responsables de la régulation stomatique ont été identifiés

High-throughput phenotyping by multispectral imagery at orchard : study of genetic determinism of response to water constraint of an apple (Malus x domestica Bork) hybrid population.

Dans le contexte des changements climatiques prévisibles et d'une demande évaporative accrue vis à vis des cultures, ce travail de thèse vise à évaluer la variabilité génétique du pommier pour sa réponse à la contrainte hydrique. Cette évaluation est d'un intérêt certain pour l'amélioration future de l'espèce, qu'il s'agisse de mieux tolérer le stress pouvant résulter d'une sécheresse temporaire du sol, ou d'une efficience accrue d'utilisation de l'eau. Les méthodes de phénotypage actuellement disponibles pour l'étude de la réponse à la contrainte hydrique ne permettent pas un haut débit de mesure compatible avec les comparaisons simultanées entre de nombreux individus. En conséquence, ce travail de thèse s'est focalisé sur l'application de l'imagerie multi-spectrale aéroportée comme outil de phénotypage à haut débit en condition de plein champ et sur l'évaluation de sa pertinence pour l'analyse quantitative des réponses à la contrainte hydrique et la recherche des déterminismes génétiques associés. Les essais ont été réalisés sur une population hybride de pommiers (croisement Starkrimson x Granny Smith) âgée de 4 ans au début de l'étude, implantée dans le sud de la France. A partir d'images acquises dans les longueurs d'onde du visible, du proche infrarouge et de l'infrarouge thermique, à une résolution spatiale proche de 30cm, différents indices de végétation et de stress ont été calculés permettant d'apprécier la vigueur des arbres et leur transpiration. La prise en compte, dans des modèles linéaires mixtes, de six dates d'expérimentation, couvrant deux campagnes d'acquisitions, et de régimes d'irrigation différenciés sur la population a permis de mettre en évidence l'héritabilité des indices phénotypiques et conduit à la détection de nombreux QTLs. 18 QTLs se sont révélés indépendants de la date d'acquisition. Pour quatre de ces QTLs, dits adaptatifs, c'est-à-dire exprimés en conditions de stress, un certain nombre de gènes candidats potentiellement impliqués dans les réponses précoces responsables de la régulation stomatique ont été identifiés.In the context of foreseen climate changes and higher evaporative demand in crops, this PhD work is aiming at assessing the apple tree variability in response to hydric constraints. This could be of great interest for future breeding programs in this species, either in respect to temporary abiotic stress tolerance, or looking at a better water use efficiency. Phenotyping methods presently available for evaluating the plant response to hydric constraints do not allow the high-throughput that could be compatible with simultaneous comparison of a high number of individuals. As a consequence, this thesis focused on the application of airborne multispectral imagery as a tool for high-throughput field phenotyping, and it assessed the relevancy of this method for quantitative analysis of response to drought and further dissection of associated genetic determinisms. Trials were realized on a segregating hybrid apple population (Starkrimson x Granny Smith cross), 4-year old at the beginning of study, installed in the South of France. Images acquired in visible, near infrared and thermal infrared, with a spatial resolution close to 30cm, were the calculation basis of various vegetation and stress indices, allowing estimation of individual tree vigor and transpiration. Linear mixed models taking account of six flight dates, covering two acquisition campaigns and including two contrasted irrigation regimes over the apple population, made it possible to highlight the heritability of phenotypic indices and to perform numerous QTLs mapping. Eighteen QTLs were revealed independently from the acquisition date. Out of this QTLs set, a further analysis on 4 of them, which were adaptative i.e. expressed in stress conditions, allowed first identification of putative candidate genes potentially involved in the early stomatal closure

Scalable Database Indexing and Fast Image Retrieval Based on Deep Learning and Hierarchically Nested Structure Applied to Remote Sensing and Plant Biology

Digitalisation has opened a wealth of new data opportunities by revolutionizing how images are captured. Although the cost of data generation is no longer a major concern, the data management and processing have become a bottleneck. Any successful visual trait system requires automated data structuring and a data retrieval model to manage, search, and retrieve unstructured and complex image data. This paper investigates a highly scalable and computationally efficient image retrieval system for real-time content-based searching through large-scale image repositories in the domain of remote sensing and plant biology. Images are processed independently without considering any relevant context between sub-sets of images. We utilize a deep Convolutional Neural Network (CNN) model as a feature extractor to derive deep feature representations from the imaging data. In addition, we propose an effective scheme to optimize data structure that can facilitate faster querying at search time based on the hierarchically nested structure and recursive similarity measurements. A thorough series of tests were carried out for plant identification and high-resolution remote sensing data to evaluate the accuracy and the computational efficiency of the proposed approach against other content-based image retrieval (CBIR) techniques, such as the bag of visual words (BOVW) and multiple feature fusion techniques. The results demonstrate that the proposed scheme is effective and considerably faster than conventional indexing structures