Task-driven active sensing framework applied to leaf probing

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This article presents a new method for actively exploring a 3D workspace with the aim of localizing relevant regions for a given task. Our method encodes the exploration route in a multi-layer occupancy grid map. This map, together with a multiple-view estimator and a maximum-information-gain gathering approach, incrementally provide a better understanding of the scene until reaching the task termination criterion. This approach is designed to be applicable to any task entailing 3D object exploration where some previous knowledge of its approximate shape is available. Its suitability is demonstrated here for a leaf probing task using an eye-in-hand arm configuration in the context of a phenotyping application (leaf probing).Peer ReviewedPostprint (author's final draft

Early Detection of Magnaporthe oryzae-Infected Barley Leaves and Lesion Visualization Based on Hyperspectral Imaging

Early detection of foliar diseases is vital to the management of plant disease, since these pathogens hinder crop productivity worldwide. This research applied hyperspectral imaging (HSI) technology to early detection of Magnaporthe oryzae-infected barley leaves at four consecutive infection periods. The averaged spectra were used to identify the infection periods of the samples. Additionally, principal component analysis (PCA), spectral unmixing analysis and spectral angle mapping (SAM) were adopted to locate the lesion sites. The results indicated that linear discriminant analysis (LDA) coupled with competitive adaptive reweighted sampling (CARS) achieved over 98% classification accuracy and successfully identified the infected samples 24 h after inoculation. Importantly, spectral unmixing analysis was able to reveal the lesion regions within 24 h after inoculation, and the resulting visualization of host–pathogen interactions was interpretable. Therefore, HSI combined with analysis by those methods would be a promising tool for both early infection period identification and lesion visualization, which would greatly improve plant disease management

Quantifying soybean phenotypes using UAV imagery and machine learning, deep learning methods

Crop breeding programs aim to introduce new cultivars to the world with improved traits to solve the food crisis. Food production should need to be twice of current growth rate to feed the increasing number of people by 2050. Soybean is one the major grain in the world and only US contributes around 35 percent of world soybean production. To increase soybean production, breeders still rely on conventional breeding strategy, which is mainly a 'trial and error' process. These constraints limit the expected progress of the crop breeding program. The goal was to quantify the soybean phenotypes of plant lodging and pubescence color using UAV-based imagery and advanced machine learning. Plant lodging and soybean pubescence color are two of the most important phenotypes for soybean breeding programs. Soybean lodging and pubescence color is conventionally evaluated visually by breeders, which is time-consuming and subjective to human errors. The goal of this study was to investigate the potential of unmanned aerial vehicle (UAV)-based imagery and machine learning in the assessment of lodging conditions and deep learning in the assessment pubescence color of soybean breeding lines. A UAV imaging system equipped with an RGB (red-green-blue) camera was used to collect the imagery data of 1,266 four-row plots in a soybean breeding field at the reproductive stage. Soybean lodging scores and pubescence scores were visually assessed by experienced breeders. Lodging scores were grouped into four classes, i.e., non-lodging, moderate lodging, high lodging, and severe lodging. In contrast, pubescence color scores were grouped into three classes, i.e., gray, tawny, and segregation. UAV images were stitched to build orthomosaics, and soybean plots were segmented using a grid method. Twelve image features were extracted from the collected images to assess the lodging scores of each breeding line. Four models, i.e., extreme gradient boosting (XGBoost), random forest (RF), K-nearest neighbor (KNN), and artificial neural network (ANN), were evaluated to classify soybean lodging classes. Five data pre-processing methods were used to treat the imbalanced dataset to improve the classification accuracy. Results indicate that the pre-processing method SMOTE-ENN consistently performs well for all four (XGBoost, RF, KNN, and ANN) classifiers, achieving the highest overall accuracy (OA), lowest misclassification, higher F1-score, and higher Kappa coefficient. This suggests that Synthetic Minority Over-sampling-Edited Nearest Neighbor (SMOTE-ENN) may be an excellent pre-processing method for using unbalanced datasets and classification tasks. Furthermore, an overall accuracy of 96 percent was obtained using the SMOTE-ENN dataset and ANN classifier. On the other hand, to classify the soybean pubescence color, seven pre-trained deep learning models, i.e., DenseNet121, DenseNet169, DenseNet201, ResNet50, InceptionResNet-V2, Inception-V3, and EfficientNet were used, and images of each plot were fed into the model. Data was enhanced using two rotational and two scaling factors to increase the datasets. Among the seven pre-trained deep learning models, ResNet50 and DenseNet121 classifiers showed a higher overall accuracy of 88 percent, along with higher precision, recall, and F1-score for all three classes of pubescence color. In conclusion, the developed UAV-based high-throughput phenotyping system can gather image features to estimate soybean crucial phenotypes and classify the phenotypes, which will help the breeders in phenotypic variations in breeding trials. Also, the RGB imagery-based classification could be a cost-effective choice for breeders and associated researchers for plant breeding programs in identifying superior genotypes.Includes bibliographical references

Application of deep learning and machine learning workflows for field-scale phenotyping

Tassel is the male inflorescences organ of the maize plant that develops atop the plant. Coarse features of tassels, including shape and size, can influence shedding pollen, fertilization, and subsequently grain yield. Therefore, understanding tassel dynamics and characteristics as well as how it evolves during the plant growth can help the plant scientist community to increase the grain yield as a final goal. To do so, first, tassels were investigated in one time points. The tassels were cut in the field and their images were captured in a lightbox. Coarse features were measured using novel image processing approaches. 351 tassels with different genotypes were used for the experiment. Tassel length, first lowest branch length, and angle as well as central spike length were measured by applying image processing and machine learning techniques. Tassels were also classified to open and close structures to obtain accurate predictions for the traits. The results show that R2 values for the tassel length and central spike length were 0.92 and 0.80, respectively. In addition, the R2 values for the first lowest branch length and angle were 0.63 and 0.91, respectively. The R2 values for the first lowest branch length was low compared to others because locating the first lowest branch point and its corresponding branch tip was hard due to branches occlusion. This study was done to create a robust algorithm for tassel phenotyping. Challenges were figured out for better tassel phenotyping in the field. Then, we looked at a diverse panel in the field, using stationary cameras to capture 6 tassels every 10 minutes for 8 hours per day during a month. Traditional approaches for phenotyping anthesis progression are time-consuming, subjective, and labor-intensive and are thus impractical for phenotyping large populations in multiple environments. In this work, we utilize a high throughput phenotyping approach that is based on extracting time-lapse information of anthesis progress from digital cameras. The major challenge is identifying the region of the interest (i.e. location of tassels in the imaging window) in the acquired images. Camera drift, different types of weather, including fog, rain, clouds, and sun and additionally, occlusion of tassels by other tassels or leaves complicated this problem. We discussed the associated challenges for object detection and localization under noisy conditions. In addition, a framework was developed to utilize Amazon Mechanical Turk to allow turkers to annotate the images and evaluate them to create an object detection dataset. Finally, we illustrated a promising deep-learning approach to tassel recognition and localization that is based on Faster-RCNN which has shown the strong capability for detection and localization. This method was improved using a boosting method to improve the dataset. This approach is able to reliably identify a diverse set of tassel morphologies with the mAP of 0.81. Tassel flowering pattern is the most important and complex trait. Tassel maize as a male structure is responsible to produce pollen for the silk as a female organ on the same plant. The amount of pollen and shedding time is important for the breeders as well as the biologists. This study introduced an automated end-to-end pipeline by coupling various deep learning, machine learning and image processing approaches. Inbred lines from both SAM and NAM panels were grown at Curtiss farm at Iowa State University, Ames, IA. A stationary camera was installed for every two plants. Tassels architecture, weather type, tassels and camera movements are the most important challenges of the research. To address these issues, deep learning algorithms were utilized. Tassel detection, classification, and segmentation. In addition, advanced image processing approaches were used to crop the tassel main spike and track the during tassel evolution. The results showed that deep learning is a powerful tool to detect, classify and segment the tassels. The mAP for the tassel detection was 0.91. The F1-score obtained for the tassel classification was 0.93. In addition, the accuracy of semantic segmentation for creating a binary image from the RGB tassel images was 0.95. The width of the flowering was obtained using graph theory in image processing and the time and location of the flowering can be obtained from the width data over the main spike branch. In addition to tassel structures, crop growth simulation models can help farmers and breeders predict crop performance, and in maize, Leaf Appearance Rate (LAR) is an important parameter used in crop performance simulation models such as APSIM. Since breeders and biologists would like to minimize human involvement in monitoring LAR, this trait can be monitored by applying a high-throughput phenotyping system. Engineers have entered the picture in collaboration with plant scientists to establish different and robust phenotyping methods, and in this study, maize leaf appearance rate was investigated using high-throughput phenotyping approaches. We developed an imaging system for automatically capturing a time-series of images of maize plants under field conditions, with 380 RGB cameras were used to capture images from 380 rows. There were 6 plants with the same genotype in each row that had different genotypes differed row-by-row, and the images were taken for 9 hours daily at 20- minute intervals for more than one month during a growing season. An end-to-end deep learning method was then used to count the numbers of leaves in the images. The dataset for the deep learning algorithm, obtained using the Amazon Mechanical Turk platform, was created by one expert turker along with a well-trained turker. Results demonstrated that an end-to-end model with training based on the expert turker dataset performed very well, handling variation in images that included leaf occlusions and weather type. The R2 between the ground truth obtained by the expert turker and predicted values was approximately 0.73, 0.74, and 0.95 for three testing cameras. The model\u27s prediction performance demonstrated that the number of leaves increase with different slopes for different genotypes. The data can be used for further genotypic analysis

Physiological and genetic characterization of sorghum exposed to early season chilling and terminal heat and drought stress

Doctor of PhilosophyDepartment of AgronomyS.V. Krishna JagadishSorghum (Sorghum bicolor (L.) Moench) is one of the hardiest crop to abiotic stresses compared with other grain crops. However early stage chilling, terminal heat and drought stress are three most damaging abiotic stresses that have limited sorghum productivity in the US Great plains and other locations having similar environmental conditions. Three studies were conducted with an overall goal aimed at increasing grain sorghum’s resilience to harsh climatic conditions. In the first study, four promising chilling stress tolerant sorghum advanced breeding lines, a known early stage chilling tolerant Chinese landrace (Shan Qui Red - SQR) and a susceptible US elite cultivar (RTx430) as checks were assessed for chilling tolerance during emergence and early growth under field and controlled environments. Aerial phenotyping using unmanned aircraft systems (UAS) fitted with multispectral camera was used to capture reflectance-based vegetation indices (NDVI and NDRE) in field experiments. Some advanced breeding lines with superior agronomic background also recorded significantly better emergence, seedling growth and vigor compared to SQR under chilling conditions. Aerial phenotyping indices from images taken between 30 and 60 days after emergence were consistently correlated with destructive measurements under early plantings, indicating their effectiveness in differentiating chilling responses. Second study was conducted to understand physiological mechanisms inducing heat stress resilience in sorghum during flowering. A diverse set of sorghum inbreds and selected hybrids were tested under greenhouse, growth chamber facilities and field conditions. A highly conserved early-morning-flowering mechanism was observed across all the inbreds and hybrids, with the peak anthesis wherein >90% of florets completed flowering within 30 min after dawn. The conserved response was consistent even under drought stress and heat stress exposure imposed at different times of the day. Our findings report a novel heat escaping early-morning-flowering mechanism effectively employed by sorghum to minimize heat stress impact at anthesis. Another experiment with sequential increase in daytime temperature treatments suggest heat stress induced loss in pollen viability to be a key factor resulting in reduced seed-set and grain yield. The findings suggest heat stress could have a greater impact on post-pollen germination processes such as fertilization, embryo formation and development. We identified a heat tolerant genotype “Macia” which appears to be a promising donor for developing improved heat tolerant sorghum hybrids. In the third study, a bi-parental recombinant inbred lines (RILs) mapping population developed from elite post flowering drought susceptible cultivar (RTx430) and a known drought tolerant cultivar (SC35) were evaluated under wide spectrum of environments and moisture conditions. Several novel and major QTL for grain yield, panicle neck diameter, effective quantum yield of photosystem II and chlorophyll content were identified. The genomic regions and the candidate genes within these regions can potentially help in improving source and sink dynamics in sorghum under diverse environments. The findings from these studies will complement ongoing efforts in developing future sorghum with enhanced resilience to different abiotic stresses that continue to limit sorghum productivity

Analysis of Argonaute-Small RNA-Transcription Factor Circuits Controlling Leaf Development

Experimental studies of plant development have yielded many insights into gene regulation, revealing interactions between core transcriptional and post-transcriptional regulatory pathways present in all land plants. This work describes a direct connection between the three main small RNA-transcription factor circuits controlling leaf shape dynamics in the reference plant Arabidopsis thaliana. We used a high-throughput yeast 1-hybrid platform to identify factors directly binding the promoter of the highly specialized ARGONAUTE7 silencing factor. Two groups of developmentally significant microRNA-targeted transcription factors were the clearest hits from these screens, but transgenic complementation analysis indicated that their binding sites make only a small contribution to ARGONAUTE7 function, possibly indicating a role in fine tuning. Timelapse imaging methodology developed to quantify these small differences may have broad utility for plant biologists. Our analysis also clarified requirements for polar transcription of ARGONAUTE7. This work has implications for our understanding of patterning in land plants