Methods for Detecting and Classifying Weeds, Diseases and Fruits Using AI to Improve the Sustainability of Agricultural Crops: A Review

The rapid growth of the world’s population has put significant pressure on agriculture to meet the increasing demand for food. In this context, agriculture faces multiple challenges, one of which is weed management. While herbicides have traditionally been used to control weed growth, their excessive and random use can lead to environmental pollution and herbicide resistance. To address these challenges, in the agricultural industry, deep learning models have become a possible tool for decision-making by using massive amounts of information collected from smart farm sensors. However, agriculture’s varied environments pose a challenge to testing and adopting new technology effectively. This study reviews recent advances in deep learning models and methods for detecting and classifying weeds to improve the sustainability of agricultural crops. The study compares performance metrics such as recall, accuracy, F1-Score, and precision, and highlights the adoption of novel techniques, such as attention mechanisms, single-stage detection models, and new lightweight models, which can enhance the model’s performance. The use of deep learning methods in weed detection and classification has shown great potential in improving crop yields and reducing adverse environmental impacts of agriculture. The reduction in herbicide use can prevent pollution of water, food, land, and the ecosystem and avoid the resistance of weeds to chemicals. This can help mitigate and adapt to climate change by minimizing agriculture’s environmental impact and improving the sustainability of the agricultural sector. In addition to discussing recent advances, this study also highlights the challenges faced in adopting new technology in agriculture and proposes novel techniques to enhance the performance of deep learning models. The study provides valuable insights into the latest advances and challenges in process systems engineering and technology for agricultural activities

Optimized Angles of the Swing Hyperspectral Imaging Tower for Single Corn Plant

During recent years, hyperspectral imaging systems have been widely applied in the greenhouses for plant phenotyping purposes. Current imaging systems are mostly designed as either top view or side view imaging mode. Top-view is an ideal imaging angle for top leaves which are often more flat with more uniform reflectance. However, most bottom leaves are either blocked or shaded from top view. From side view, most leaves are viewable, and the entire structure can be imaged. However, at this angle most of the leaves are not facing the camera, which will impact the measurement quality. At the same time, there could be advantages with certain tilted imaging angle between top view and side view. Therefore, it’s important to explore the impact of different imaging angles to the phenotyping quality. For this purpose, we designed a swing hyperspectral imaging tower which enables us to rotate the camera and lighting source to capture images at any angle from side view (0◦) to top view (90◦). 36 corn plants were grown and divided into 3 different treatments: high nitrogen (N) and well-watered (control group), high N and drought-stressed, and low N and well-watered. Each plant was imaged at 7 different angles from 0◦ to 90◦ with an interval of 15◦. According to different treatments applied on experimental samples, two comparative pairs were set up: drought-stressed group vs. control group (Pair 1); N-deficiency group vs. control group (Pair 2). In this study, normalized difference vegetation index (NDVI) and relative water content (RWC) were computed and compared to determine optimized imaging angle(s). For NDVI, the imaging angle near to top view is optimized to separate Pair 1, while, the imaging angle near to side view is optimized to distinguish Pair 2. For RWC, partial least square regression (PLSR) models were applied to predict pixel-level RWC distribution of each plant, and higher imaging angles (close to top view) are better to tell the RWC distribution difference in Pair 1. In conclusion, higher imaging angles (close to top view) are better to separate different water treatments, while, lower imaging angles (close to side view) are better to separate different N treatments

Rapeseed Seedling Stand Counting and Seeding Performance Evaluation at Two Early Growth Stages Based on Unmanned Aerial Vehicle Imagery

The development of unmanned aerial vehicles (UAVs) and image processing algorithms for field-based phenotyping offers a non-invasive and effective technology to obtain plant growth traits such as canopy cover and plant height in fields. Crop seedling stand count in early growth stages is important not only for determining plant emergence, but also for planning other related agronomic practices. The main objective of this research was to develop practical and rapid remote sensing methods for early growth stage stand counting to evaluate mechanically seeded rapeseed (Brassica napus L.) seedlings. Rapeseed was seeded in a field by three different seeding devices. A digital single-lens reflex camera was installed on an UAV platform to capture ultrahigh resolution RGB images at two growth stages when most rapeseed plants had at least two leaves. Rapeseed plant objects were segmented from images of vegetation indices using typical Otsu thresholding method. After segmentation, shape features such as area, length-width ratio and elliptic fit were extracted from the segmented rapeseed plant objects to establish regression models of seedling stand count. Three row characteristics (the coefficient of variation of row spacing uniformity, the error rate of the row spacing and the coefficient of variation of seedling uniformity) were further calculated for seeding performance evaluation after crop row detection. Results demonstrated that shape features had strong correlations with ground-measured seedling stand count. The regression models achieved R-squared values of 0.845 and 0.867, respectively, for the two growth stages. The mean absolute errors of total stand count were 9.79 and 5.11% for the two respective stages. A single model over these two stages had an R-squared value of 0.846, and the total number of rapeseed plants was also accurately estimated with an average relative error of 6.83%.Moreover, the calculated row characteristics were demonstrated to be useful in recognizing areas of failed germination possibly resulted from skipped or ineffective planting. In summary, this study developed practical UAV-based remote sensing methods and demonstrated the feasibility of using the methods for rapeseed seedling stand counting and mechanical seeding performance evaluation at early growth stages

Rapeseed Seedling Stand Counting and Seeding Performance Evaluation at Two Early Growth Stages Based on Unmanned Aerial Vehicle Imagery

The development of unmanned aerial vehicles (UAVs) and image processing algorithms for field-based phenotyping offers a non-invasive and effective technology to obtain plant growth traits such as canopy cover and plant height in fields. Crop seedling stand count in early growth stages is important not only for determining plant emergence, but also for planning other related agronomic practices. The main objective of this research was to develop practical and rapid remote sensing methods for early growth stage stand counting to evaluate mechanically seeded rapeseed (Brassica napus L.) seedlings. Rapeseed was seeded in a field by three different seeding devices. A digital single-lens reflex camera was installed on an UAV platform to capture ultrahigh resolution RGB images at two growth stages when most rapeseed plants had at least two leaves. Rapeseed plant objects were segmented from images of vegetation indices using typical Otsu thresholding method. After segmentation, shape features such as area, length-width ratio and elliptic fit were extracted from the segmented rapeseed plant objects to establish regression models of seedling stand count. Three row characteristics (the coefficient of variation of row spacing uniformity, the error rate of the row spacing and the coefficient of variation of seedling uniformity) were further calculated for seeding performance evaluation after crop row detection. Results demonstrated that shape features had strong correlations with ground-measured seedling stand count. The regression models achieved R-squared values of 0.845 and 0.867, respectively, for the two growth stages. The mean absolute errors of total stand count were 9.79 and 5.11% for the two respective stages. A single model over these two stages had an R-squared value of 0.846, and the total number of rapeseed plants was also accurately estimated with an average relative error of 6.83%. Moreover, the calculated row characteristics were demonstrated to be useful in recognizing areas of failed germination possibly resulted from skipped or ineffective planting. In summary, this study developed practical UAV-based remote sensing methods and demonstrated the feasibility of using the methods for rapeseed seedling stand counting and mechanical seeding performance evaluation at early growth stages

Leaf nitrogen determination using non-destructive techniques–A review

© 2017 Taylor & Francis Group, LLC. The optimisation of plant nitrogen-use-efficiency (NUE) has a direct impact on increasing crop production by optimising use of nitrogen fertiliser. Moreover, it protects environment from negative effects of nitrate leaching and nitrous oxide production. Accordingly, nitrogen (N) management in agriculture systems has been major focus of many researchers. Improvement of NUE can be achieved through several methods including more accurate measurement of foliar N contents of crops during different growth phases. There are two types of methods to diagnose foliar N status: destructive and non-destructive. Destructive methods are expensive and time-consuming, as they require tissue sampling and subsequent laboratory analysis. Thus, many farmers find destructive methods to be less attractive. Non-destructive methods are rapid and less expensive but are usually less accurate. Accordingly, improving the accuracy of non-destructive N estimations has become a common goal of many researchers, and various methods varying in complexity and optimality have been proposed for this purpose. This paper reviews various commonly used non-destructive methods for estimating foliar N status of plants

Monitorización 3D de cultivos y cartografía de malas hierbas mediante vehículos aéreos no tripulados para un uso sostenible de fitosanitarios

En esta Tesis Doctoral se han utilizado las imágenes procedentes de un UAV para abordar la sostenibilidad de la aplicación de productos fitosanitarios mediante la generación de mapas que permitan su aplicación localizada. Se han desarrollado dos formas diferentes y complementarias para lograr este objetivo: 1) la reducción de la aplicación de herbicidas en post-emergencia temprana mediante el diseño de tratamientos dirigidos a las zonas infestadas por malas hierbas en varios cultivos herbáceos; y 2) la caracterización tridimensional (arquitectura y volumen) de cultivos leñosos para el diseño de tratamientos de aplicación localizada de fitosanitarios dirigidos a la parte aérea de los mismos. Para afrontar el control localizado de herbicidas se han estudiado la configuración y las especificaciones técnicas de un UAV y de los sensores embarcados a bordo para su aplicación en la detección temprana de malas hierbas y contribuir a la generación de mapas para un control localizado en tres cultivos herbáceos: maíz, trigo y girasol. A continuación, se evaluaron los índices espectrales más precisos para su uso en la discriminación de suelo desnudo y vegetación (cultivo y malas hierbas) en imágenes-UAV tomadas sobre dichos cultivos en fase temprana. Con el fin de automatizar dicha discriminación se implementó en un entorno OBIA un método de cálculo de umbrales. Finalmente, se desarrolló una metodología OBIA automática y robusta para la discriminación de cultivo, suelo desnudo y malas hierbas en los tres cultivos estudiados, y se evaluó la influencia sobre su funcionamiento de distintos parámetros relacionados con la toma de imágenes UAV (solape, tipo de sensor, altitud de vuelo, momento de programación de los vuelos, entre otros). Por otra parte y para facilitar el diseño de tratamientos fitosanitarios ajustados a las necesidades de los cultivos leñosos se ha desarrollado una metodología OBIA automática y robusta para la caracterización tridimensional (arquitectura y volumen) de cultivos leñosos usando imágenes y modelos digitales de superficies generados a partir de imágenes procedentes de un UAV. Asimismo, se evaluó la influencia de distintos parámetros relacionados con la toma de las imágenes (solape, tipo de sensor, altitud de vuelo) sobre el funcionamiento del algoritmo OBIA diseñado

Automatic plant features recognition using stereo vision for crop monitoring

Machine vision and robotic technologies have potential to accurately monitor plant parameters which reflect plant stress and water requirements, for use in farm management decisions. However, autonomous identification of individual plant leaves on a growing plant under natural conditions is a challenging task for vision-guided agricultural robots, due to the complexity of data relating to various stage of growth and ambient environmental conditions. There are numerous machine vision studies that are concerned with describing the shape of leaves that are individually-presented to a camera. The purpose of these studies is to identify plant species, or for the autonomous detection of multiple leaves from small seedlings under greenhouse conditions. Machine vision-based detection of individual leaves and challenges presented by overlapping leaves on a developed plant canopy using depth perception properties under natural outdoor conditions is yet to be reported. Stereo vision has recently emerged for use in a variety of agricultural applications and is expected to provide an accurate method for plant segmentation and identification which can benefit from depth properties and robustness. This thesis presents a plant leaf extraction algorithm using a stereo vision sensor. This algorithm is used on multiple leaf segmentation and overlapping leaves separation using a combination of image features, specifically colour, shape and depth. The separation between the connected and the overlapping leaves relies on the measurement of the discontinuity in depth gradient for the disparity maps. Two techniques have been developed to implement this task based on global and local measurement. A geometrical plane from each segmented leaf can be extracted and used to parameterise a 3D model of the plant image and to measure the inclination angle of each individual leaf. The stem and branch segmentation and counting method was developed based on the vesselness measure and Hough transform technique. Furthermore, a method for reconstructing the segmented parts of hibiscus plants is presented and a 2.5D model is generated for the plant. Experimental tests were conducted with two different selected plants: cotton of different sizes, and hibiscus, in an outdoor environment under varying light conditions. The proposed algorithm was evaluated using 272 cotton and hibiscus plant images. The results show an observed enhancement in leaf detection when utilising depth features, where many leaves in various positions and shapes (single, touching and overlapping) were detected successfully. Depth properties were more effective in separating between occluded and overlapping leaves with a high separation rate of 84% and these can be detected automatically without adding any artificial tags on the leaf boundaries. The results exhibit an acceptable segmentation rate of 78% for individual plant leaves thereby differentiating the leaves from their complex backgrounds and from each other. The results present almost identical performance for both species under various lighting and environmental conditions. For the stem and branch detection algorithm, experimental tests were conducted on 64 colour images of both species under different environmental conditions. The results show higher stem and branch segmentation rates for hibiscus indoor images (82%) compared to hibiscus outdoor images (49.5%) and cotton images (21%). The segmentation and counting of plant features could provide accurate estimation about plant growth parameters which can be beneficial for many agricultural tasks and applications

Prediction of Early Vigor from Overhead Images of Carinata Plants

Breeding more resilient, higher yielding crops is an essential component of ensuring ongoing food security. Early season vigor is signi cantly correlated with yields and is often used as an early indicator of tness in breeding programs. Early vigor can be a useful indicator of the health and strength of plants with bene ts such as improved light interception, reduced surface evaporation, and increased biological yield. However, vigor is challenging to measure analytically and is often rated using subjective visual scoring. This traditional method of breeder scoring becomes cumbersome as the size of breeding programs increase. In this study, we used hand-held cameras tted on gimbals to capture images which were then used as the source for automated vigor scoring. We have employed a novel image metric, the extent of plant growth from the row centerline, as an indicator of vigor. Along with this feature, additional features were used for training a random forest model and a support vector machine, both of which were able to predict expert vigor ratings with an 88:9% and 88% accuracies respectively, providing the potential for more reliable, higher throughput vigor estimates

Advances in high throughput and affordable phenotyping for adapting maize and wheat to climate change

[eng] Supplying sufficient food to an increasing population is one of the most important challenges over the next century. To meet this demand, crop productivity will need to increase while it is being threatened by climate change effects like the increase of temperatures and the intensity of drought periods. Improving crop performance is key for an efficient adaptation to these challenging growing conditions, with crop breeding being one of the pillars. In that sense selecting more productive varieties for specific environments requires a better understanding of plant acclimation to stress conditions, including efficient phenotyping approaches. Plant phenotyping research pursues the development of new methods with high-throughput capacity and affordable to characterize non-destructively plant traits of interest. The main focus of this thesis was to develop and study versatile and precise methodologies with high-throughput capacity in order to improve crop performance assessments, while saving time and costs in the phenotyping tasksof two of the most important cereal crops: maize and wheat. The use of unmanned aerial vehicles (UAV) equipped with imaging sensors (including RGB, multispectral and thermal) permits covering simultaneously hectares of experimental fields fast, precisely, and in a non-destructive way. However, ground evaluations may still be an alternative in terms of cost and spatial resolution. The performance of these methodologies to assess genotypic differences in grain yield was evaluated in maize and wheat under different agronomical and environmental growing conditions such as nutrient deficiency, conservation agriculture, drought and heat stress. On one side, maize studies were performed in trials in Zimbabwe focused on the evaluation of genotypes under either low and normal phosphorus conditions or the application of conservation agriculture together with different top-dressing nitrogen fertilization regimes, to overcome the nutrient poverty of soils. In these studies, vegetation indices, related to parameters informing on the above-ground biomass and assessed during early stages of development, performed well as grain yield indicators. Moreover, during more advanced phenological stages, indices informing on the leaf and the canopy color were the traits that reported a better association with grain yield and N content in leaves. For the case of wheat, evaluations were performed in different latitudes in Spain covering a range of environments and grown under different management conditions, and sampling was performed during the reproductive stage (heading, anthesis and grain filling). In general terms, biomass indicators, such as canopy green biomass inferred from vegetation indices, together with water status indicators, such as canopy temperature, were the most critical traits predicting GY. The delay of senescence in water-limited environments and the photosynthetic efficiency measured by multispectral indices like the photochemical reflectance index (PRI) during anthesis were also relevant traits for GY under the rainfed and late-planting trials, respectively.[cat] La producció de suficient aliment per a una població cada cop més gran és un dels reptes més importants per al pròxim segle. Per assolir la demanda, la productivitat dels cultius han d’augmentar alhora que fan front als efectes del canvi climàtic com increment de les temperatures i la intensitat dels períodes de sequera. La millora de la capacitat dels cultius és un element clau per a l’adaptació a aquestes condicions més exigents i la selecció de varietats més productives sota ambients específics requereix una millor comprensió de l’aclimatació dels cultius als estressos. La recerca en fenotipatge de cultius té com objectiu el desenvolupament de noves metodologies d’alt rendiment capaces de caracteritzar característiques d’interès de les plantes d’una manera no destructiva. Sota condicions de camp, l’aplicació de metodologies tradicionals en experiments grans laboriós i requereix molt de temps. El principal objectiu d’aquesta tesi ha estat el desenvolupament i estudi diferents metodologies de caràcter versàtil, precises i d’alta capacitat per a millorar les mesures de com es desenvolupen els cultius, alhora de que es redueixen els costos i el temps requerit per a fer els mostrejos. El treball es basa en dos dels principals cereals: el blat i el blat de moro. L’ús de vehicles aeris no tripulats (UAV, del anglès Unmanned Aerial Vehicles) equipats amb càmeres i sensors (RGB, multiespectrals i termals) permet mesurar simultàniament hectàrees de camps experimentals d’una manera ràpida, precisa i sense la destrucció de mostra. Tot i així, les mesures a nivell de terra també són una alternativa prou potent pel que fa el cost i la resolució espacial. La capacitat d’aquestes metodologies per a mesurar diferencies genotípiques en el rendiment del blat de moro i el blat ha estat analitzada sota diferents condicions de creixement com la deficiència de nutrients, pràctiques de agricultura de conservació, sequera i altes temperatures. Per una banda, els estudis de blat de moro es van desenvolupar a Zimbabwe i estaven focalitzats en l’avaluació de genotips sota condicions diferents de fòsfor o en l’aplicació de l’agricultura de conservació per combatre la pobresa mineral dels sòls. En aquests estudis, les mesures relacionades amb paràmetres de biomassa aèria durant estadis primerencs de desenvolupament va funcionar bé com a indicadors de rendiment. A més, durant estadis fenològics més avançats, mesures de color de la capçada del cultiu van estar associats tant amb el rendiment com amb el contingut de nitrogen en les fulles. En el cas del blat, les avaluacions es van dur a terme a diferents latituds d’Espanya, cobrint un ampli rang de condicions climàtiques i agronòmiques. Els mostrejos es van realitzar en diferents estadis fenològics. En termes generals, els indicadors de biomassa i d’estat hídric del cultiu han estat de les mesures més correlacionades amb el rendiment. L’endarreriment de la senescència del cultiu en els ambients on l’aigua era el factor més limitant i el potencial fotosintètic mesurat per index multiespectrals durant la floració del cultiu han estat rellevants sota condicions de sequera i sembra tardana, respectivament

Applications of remote sensing in agriculture via unmanned aerial systems and satellites

Doctor of PhilosophyDepartment of AgronomyIgnacio CiampittiThe adoption of Remote Sensing (RS) in agriculture have been mainly utilized to inference about biological processes in a scalable manner over space and time. In this context, this work first explores two non-traditional approaches for rapid derivation of plant performance under field conditions. Both approaches focus on plant metrics extraction exploiting high spatial resolution from Unmanned Aerial Systems (UAS). Second, we investigate the spatial-temporal dynamics of corn (Zea mays L.) phenology and yield in the corn belt region utilizing high temporal resolution from satellite. To evaluate the impact of the adoption of RS for deriving plant/crop performance the following objectives were established: i) investigate the implementation of digital aerial photogrammetry to derive plant metrics (plant height and biomass) in corn; ii) implement and test a methodology for detecting and counting corn plants via very high spatial resolution imagery in the context of precision agriculture; iii) derive key phenological metrics of corn via high temporal resolution satellite imagery and identify links between the derived metrics and yield trends over the last 14 years for corn within the corn belt region. For the first objective, main findings indicate that digital aerial photogrammetry can be utilized to derive plant height and assist in plant biomass estimation. Results also suggest that plant biomass predictability significantly increases when integrating the aerial plant height estimate and ground stem diameter. For the second objective, the workflow implemented demostrates adequate performance to detect and count corn plants in the image. Its robustness highly dependends on the spatial resolution of the image, limitations and future research paths are further discussed. Lastly, for the third objective, outcomes evidenced that for a long-term perspective (14 years), an extended reproductive stage significantly correlates with high yield for corn. When considering a shorter-term period (last 4 years) mainly characterized by optimal growth conditions, early season green-up rate and late season senescence rate positively describe yield trend in the region. The significance of the variables changed according to the time-span considered. It is noticed that when optimal growth conditions are met, modern-hybrids can capitalize by increasing yield, due to primarily a faster (green-up) rate before flowering and on senescence rate better describes yield under these conditions. The entire research project investigates opportunities and needs for integrating remote sensing into the agronomic-based inference process