A Low-cost Depth Imaging Mobile Platform for Canola Phenotyping

To meet the high demand for supporting and accelerating progress in the breeding of novel traits, plant scientists and breeders have to measure a large number of plants and their characteristics accurately. A variety of imaging methodologies are being deployed to acquire data for quantitative studies of complex traits. When applied to a large number of plants such as canola plants, however, a complete three-dimensional (3D) model is time-consuming and expensive for high-throughput phenotyping with an enormous amount of data. In some contexts, a full rebuild of entire plants may not be necessary. In recent years, many 3D plan phenotyping techniques with high cost and large-scale facilities have been introduced to extract plant phenotypic traits, but these applications may be affected by limited research budgets and cross environments. This thesis proposed a low-cost depth and high-throughput phenotyping mobile platform to measure canola plant traits in cross environments. Methods included detecting and counting canola branches and seedpods, monitoring canola growth stages, and fusing color images to improve images resolution and achieve higher accuracy. Canola plant traits were examined in both controlled environment and field scenarios. These methodologies were enhanced by different imaging techniques. Results revealed that this phenotyping mobile platform can be used to investigate canola plant traits in cross environments with high accuracy. The results also show that algorithms for counting canola branches and seedpods enable crop researchers to analyze the relationship between canola genotypes and phenotypes and estimate crop yields. In addition to counting algorithms, fusing techniques can be helpful for plant breeders with more comfortable access plant characteristics by improving the definition and resolution of color images. These findings add value to the automation, low-cost depth and high-throughput phenotyping for canola plants. These findings also contribute a novel multi-focus image fusion that exhibits a competitive performance with outperforms some other state-of-the-art methods based on the visual saliency maps and gradient domain fast guided filter. This proposed platform and counting algorithms can be applied to not only canola plants but also other closely related species. The proposed fusing technique can be extended to other fields, such as remote sensing and medical image fusion

LiDARPheno: A Low-Cost LiDAR-based 3D Scanning System for Plant Morphological Trait Characterization

The ever-growing world population brings the challenge for food security in the current world. The gene modification tools have opened a new era for fast-paced research on new crop identification and development. However, the bottleneck in the plant phenotyping technology restricts the alignment in Geno-pheno development as phenotyping is the key for the identification of potential crop for improved yield and resistance to the changing environment. Various attempts to making the plant phenotyping a “high-throughput” have been made while utilizing the existing sensors and technology. However, the demand for ‘good’ phenotypic information for linkage to the genome in understanding the gene-environment interactions is still a bottleneck in the plant phenotyping technologies. Moreover, the available technologies and instruments are inaccessible, expensive and sometimes bulky. This thesis work attempts to address some of the critical problems, such as exploration and development of a low-cost LiDAR-based platform for phenotyping the plants in-lab and in-field. A low-cost LiDAR-based system design, LiDARPheno, is introduced in this thesis work to assess the feasibility of the inexpensive LiDAR sensor in the leaf trait (length, width, and area) extraction. A detailed design of the LiDARPheno, based on low-cost and off-the-shelf components and modules, is presented. Moreover, the design of the firmware to control the hardware setup of the system and the user-level python-based script for data acquisition is proposed. The software part of the system utilizes the publicly available libraries and Application Programming Interfaces (APIs), making it easy to implement the system by a non-technical user. The LiDAR data analysis methods are presented, and algorithms for processing the data and extracting the leaf traits are developed. The processing includes conversion, cleaning/filtering, segmentation and trait extraction from the LiDAR data. Experiments on indoor plants and canola plants were performed for the development and validation of the methods for estimation of the leaf traits. The results of the LiDARPheno based trait extraction are compared with the SICK LMS400 (a commercial 2D LiDAR) to assess the performance of the developed system. Experimental results show a fair agreement between the developed system and a commercial LiDAR system. Moreover, the results are compared with the acquired ground truth as well as the commercial LiDAR system. The LiDARPheno can provide access to the inexpensive LiDAR-based scanning and open the opportunities for future exploration

DEVELOPMENT OF A FIELD-BASED MOBILE PLATFORM FOR PLANT PHENOTYPING

Design, implementation and performance verification of an affordable field-based high-throughput plant phenotyping platform for monitoring Canola plants, including both data acquisition/visualization software and measurement system, was the main objective of this research. The primary motivation for this research is the fact that breeders need a well-organized approach and efficient tool to monitor and analyze a number of plant traits to achieve a higher yield. At the moment, manual measurement is a conventional approach to gather the required information for plant analysis. Nevertheless, manual measurement has many limitations especially to study a large-scale field. To address this bottleneck, a high-throughput plant phenotyping platform (HTPP) was developed which consists of a data acquisition system, a data storage unit, and a data visualization and analysis software. Such an HTPP will be an essential asset for breeders to conveniently gather a comprehensive database which contains various information such as a plant height, temperature, Normalized Difference Vegetation Index (NDVI), etc. To develop and implement such an HTPP, first, the overall system block diagram and required algorithms were drawn. Then to find the optimum set of equipment according to the requirement of this application, the performance of different sensors and devices were examined using literature search and experimental examinations in the laboratory setting. Then a mechanical boom was attached to the rear of a farm vehicle (a Swather) to carry different sensors, cameras and other measurement equipment (mechanical development of the boom structure was carried out by other members of the research team). A control box containing power supplies, safety fuses, and a data logger unit was attached to the farm vehicle, and a program was developed for data logger to read sensors signals as well as GPS data for data geo-referencing and future retrieval purposes. The efficiency of different system architecture including different data transmission networks was examined by conducting several field tests to minimize existing errors such as delays in synchronizing different steps. Three programs were developed in MATLAB GUI for image acquisition via webcam and DSLR cameras as well as a central program for data processing and interactive data visualization. The indoor tests were performed at the Robotics laboratory, University of Saskatchewan and outdoor experiments were performed on a Canola nursery at Cargill Canada, Aberdeen, SK, throughout spring-summer 2016 and 2017. Finally, the performance and effectiveness of the developed field-based phenotyping platform was validated by various measures such as conducting some manual measurements and comparing the results with the values given by the platform. According to the achieved results, both hardware and software components of the proposed system meet the requirements of a field-based plant phenotyping platform as an essential asset for breeders for comprehensive study of Canola plants or any other cultivars as a result of some minor design modifications. The main contributions of this study to plant phenotyping research are autonomous image acquisition capability, enhancement of the data acquisition cycle to minimize data geo-referencing error, development of a modular program for data visualization in MATLAB, and faster data collection in a high-throughput fashion (almost 125 times faster)

High-throughput estimation of crop traits: A review of ground and aerial phenotyping platforms

Crop yields need to be improved in a sustainable manner to meet the expected worldwide increase in population over the coming decades as well as the effects of anticipated climate change. Recently, genomics-assisted breeding has become a popular approach to food security; in this regard, the crop breeding community must better link the relationships between the phenotype and the genotype. While high-throughput genotyping is feasible at a low cost, highthroughput crop phenotyping methods and data analytical capacities need to be improved. High-throughput phenotyping offers a powerful way to assess particular phenotypes in large-scale experiments, using high-tech sensors, advanced robotics, and imageprocessing systems to monitor and quantify plants in breeding nurseries and field experiments at multiple scales. In addition, new bioinformatics platforms are able to embrace large-scale, multidimensional phenotypic datasets. Through the combined analysis of phenotyping and genotyping data, environmental responses and gene functions can now be dissected at unprecedented resolution. This will aid in finding solutions to currently limited and incremental improvements in crop yields

LodgeNet: an automated framework for precise detection and classification of wheat lodging severity levels in precision farming

Wheat lodging is a serious problem affecting grain yield, plant health, and grain quality. Addressing the lodging issue in wheat is a desirable task in breeding programs. Precise detection of lodging levels during wheat screening can aid in selecting lines with resistance to lodging. Traditional approaches to phenotype lodging rely on manual data collection from field plots, which are slow and laborious, and can introduce errors and bias. This paper presents a framework called ‘LodgeNet,’ that facilitates wheat lodging detection. Using Unmanned Aerial Vehicles (UAVs) and Deep Learning (DL), LodgeNet improves traditional methods of detecting lodging with more precision and efficiency. Using a dataset of 2000 multi-spectral images of wheat plots, we have developed a novel image registration technique that aligns the different bands of multi-spectral images. This approach allows the creation of comprehensive RGB images, enhancing the detection and classification of wheat lodging. We have employed advanced image enhancement techniques to improve image quality, highlighting the important features of wheat lodging detection. We combined three color enhancement transformations into two presets for image refinement. The first preset, ‘Haze & Gamma Adjustment,’ minimize atmospheric haze and adjusts the gamma, while the second, ‘Stretching Contrast Limits,’ extends the contrast of the RGB image by calculating and applying the upper and lower limits of each band. LodgeNet, which relies on the state-of-the-art YOLOv8 deep learning algorithm, could detect and classify wheat lodging severity levels ranging from no lodging (Class 1) to severe lodging (Class 9). The results show the mean Average Precision (mAP) of 0.952% @0.5 and 0.641% @0.50-0.95 in classifying wheat lodging severity levels. LodgeNet promises an efficient and automated high-throughput solution for real-time crop monitoring of wheat lodging severity levels in the field

Quantifying soybean phenotypes using UAV imagery and machine learning, deep learning methods

Crop breeding programs aim to introduce new cultivars to the world with improved traits to solve the food crisis. Food production should need to be twice of current growth rate to feed the increasing number of people by 2050. Soybean is one the major grain in the world and only US contributes around 35 percent of world soybean production. To increase soybean production, breeders still rely on conventional breeding strategy, which is mainly a 'trial and error' process. These constraints limit the expected progress of the crop breeding program. The goal was to quantify the soybean phenotypes of plant lodging and pubescence color using UAV-based imagery and advanced machine learning. Plant lodging and soybean pubescence color are two of the most important phenotypes for soybean breeding programs. Soybean lodging and pubescence color is conventionally evaluated visually by breeders, which is time-consuming and subjective to human errors. The goal of this study was to investigate the potential of unmanned aerial vehicle (UAV)-based imagery and machine learning in the assessment of lodging conditions and deep learning in the assessment pubescence color of soybean breeding lines. A UAV imaging system equipped with an RGB (red-green-blue) camera was used to collect the imagery data of 1,266 four-row plots in a soybean breeding field at the reproductive stage. Soybean lodging scores and pubescence scores were visually assessed by experienced breeders. Lodging scores were grouped into four classes, i.e., non-lodging, moderate lodging, high lodging, and severe lodging. In contrast, pubescence color scores were grouped into three classes, i.e., gray, tawny, and segregation. UAV images were stitched to build orthomosaics, and soybean plots were segmented using a grid method. Twelve image features were extracted from the collected images to assess the lodging scores of each breeding line. Four models, i.e., extreme gradient boosting (XGBoost), random forest (RF), K-nearest neighbor (KNN), and artificial neural network (ANN), were evaluated to classify soybean lodging classes. Five data pre-processing methods were used to treat the imbalanced dataset to improve the classification accuracy. Results indicate that the pre-processing method SMOTE-ENN consistently performs well for all four (XGBoost, RF, KNN, and ANN) classifiers, achieving the highest overall accuracy (OA), lowest misclassification, higher F1-score, and higher Kappa coefficient. This suggests that Synthetic Minority Over-sampling-Edited Nearest Neighbor (SMOTE-ENN) may be an excellent pre-processing method for using unbalanced datasets and classification tasks. Furthermore, an overall accuracy of 96 percent was obtained using the SMOTE-ENN dataset and ANN classifier. On the other hand, to classify the soybean pubescence color, seven pre-trained deep learning models, i.e., DenseNet121, DenseNet169, DenseNet201, ResNet50, InceptionResNet-V2, Inception-V3, and EfficientNet were used, and images of each plot were fed into the model. Data was enhanced using two rotational and two scaling factors to increase the datasets. Among the seven pre-trained deep learning models, ResNet50 and DenseNet121 classifiers showed a higher overall accuracy of 88 percent, along with higher precision, recall, and F1-score for all three classes of pubescence color. In conclusion, the developed UAV-based high-throughput phenotyping system can gather image features to estimate soybean crucial phenotypes and classify the phenotypes, which will help the breeders in phenotypic variations in breeding trials. Also, the RGB imagery-based classification could be a cost-effective choice for breeders and associated researchers for plant breeding programs in identifying superior genotypes.Includes bibliographical references

Remote Sensing in Agriculture: State-of-the-Art

The Special Issue on “Remote Sensing in Agriculture: State-of-the-Art” gives an exhaustive overview of the ongoing remote sensing technology transfer into the agricultural sector. It consists of 10 high-quality papers focusing on a wide range of remote sensing models and techniques to forecast crop production and yield, to map agricultural landscape and to evaluate plant and soil biophysical features. Satellite, RPAS, and SAR data were involved. This preface describes shortly each contribution published in such Special Issue

Optimization of AI models as the Main Component in Prospective Edge Intelligence Applications

Artificial Intelligence (AI) is a successful paradigm with application in many fields; however, there can be some challenging scenarios like the deployment of AI models in remote locations or with limited connectivity, possibly needing to perform inference closer to where data is collected. A potential solution is the study of ways to optimize AI models, for deployment of intelligent algorithms closer to the edge. This thesis focuses on applications of AI that need to have characteristics that make them suitable for implementation on portable devices (e.g., aeroponics container, drone, mobile robot); thus, the development and implementation of custom models, and their optimization (i.e., reduction in size and inference time) is of upmost importance and the main goal of this dissertation. For this task, a number of options have been explored, including developing techniques to select relevant features from the samples that the model analyzes, and pruning and quantization. Therefore, this thesis proposes a scheme for the development, implementation, and optimization of custom AI models used mainly in agriculture, so that they have the desired characteristics that are needed for their deployment in edge devices. This main goal is fulfilled by implementing a number of sequential steps that include the validation of the hypothesis that there is at least an AI model capable of generating useful predictions for the applications being studied, the exploration and implementation of an approach for their optimization, and their final implementation in hardware of limited resources. The main contributions of this thesis are on the general workflow for optimization of custom models, as well as in the proposed scheme for feature selection based on model interpretability approaches. This carries most of the novelty of the thesis, since we have not found previous implementations of these ideas, at least in the field under study. This optimization is mainly based on a feature selection approach, but finally complemented with pruning and quantization. The implementation of some of these models on an edge-like device, demonstrates the feasibility of this approach. Finally, although this thesis tries to be a self-contained work, encompassing all the aspects required to go from AI model design to implementation on an edge device, there are some aspects that could be further studied, analyzed, and improved. Furthermore, it is almost impossible to keep the pace with all the new developments in the fields of AI, edge computing, hardware and software tools, etc. which opens the field for new discussions and proposals. This work tries to fill some gaps and to propose some ideas that hopefully will be useful for future researchers in the development of new technologies and solutions

Signals in the Soil: Subsurface Sensing

In this chapter, novel subsurface soil sensing approaches are presented for monitoring and real-time decision support system applications. The methods, materials, and operational feasibility aspects of soil sensors are explored. The soil sensing techniques covered in this chapter include aerial sensing, in-situ, proximal sensing, and remote sensing. The underlying mechanism used for sensing is also examined as well. The sensor selection and calibration techniques are described in detail. The chapter concludes with discussion of soil sensing challenges

Design of an Ultra-wideband Frequency System for Non-destructive Root Imaging

This thesis outlines the design and implementation of an ultra-wideband imaging system for use in imaging potted plant root system architectures. Understanding the root system architecture as plants develop is critical for plant phenotyping and ultra-wideband imaging systems have shown potential as a portable, low-cost solution to non-destructively imaging root system architectures. The proposed system is separated into three main subsystems: a Data Acquisition module, a Data Processing module, and an Image Processing and Analysis module. For each module, essential parameters and variables which largely affect the quality of the produced images and measurements of the system are analyzed and discussed. The Data Acquisition module is responsible for collecting ultra-wideband signal reflections off the potted roots in dry soil. The most critical variables for performance of the entire system are the relative permittivities of the root and the soil. Insufficient contrast between root and soil relative permittivity results in poor performance of the imaging system. Both simulated (using finite-difference time-domain methods) and experimental trials were performed and designed for data collection. The Data Processing module receives the ultra-wideband reflection data from the Data Acquisition module and produces a 2D image using delay-and-sum beamforming. This method takes advantage of known physical and electrical parameters to generate an energy mapping of reflective objects in the soil medium to be imaged. Careful design of parameters such as the steering vector and window size are essential to optimizing the quality of the results. The Image Processing and Analysis module removes any artifacts present in the produced images from the Data Processing module by primarily using morphological transformations. A modified top-hat transformation is used and the size of the structuring elements help remove unwanted artifacts. The system performs reasonably well under controlled soil conditions, and there are large improvements to be made with increasing the bandwidth of the ultra-wideband device. However, since the performance of the device is extremely reliant on the soil conditions, it is recommended that further work on ultra-wideband imaging systems for roots to be focused on measuring and modeling the complex electromagnetic properties of soil at high frequencies