Flower Detection Using Object Analysis: New Ways to Quantify Plant Phenology in a Warming Tundra Biome

Rising temperatures caused by global warming are affecting the distributions of many plant and animal species across the world. This can lead to structural changes in entire ecosystems, and serious, persistent environmental consequences. However, many of these changes occur in vast and poorly accessible biomes and involve myriad species. As a consequence, conventional methods of measurement and data analysis are resource-intensive, restricted in scope, and in some cases, intractable for measuring species changes in remote areas. In this article, we introduce a method for detecting flowers of tundra plant species in large data sets obtained by aerial drones, making it possible to understand ecological change at scale, in remote areas. We focus on the sedge species E. vaginatum that is dominant at the investigated tundra field site in the Canadian Arctic. Our system is a modified version of the Faster R-CNN architecture capable of real-world plant phenology analysis. Our model outperforms experienced human annotators in both detection and counting, recording much higher recall and comparable level of precision, regardless of the image quality caused by varying weather conditions during the data collection. (K. Stanski, GitHub - karoleks4/flower-detection: Flower detection using object analysis: New ways to quantify plant phenology in a warming tundra biome. GitHub. Accessed: Sep. 17, 2021. [Online]. Available: https://github.com/karoleks4/flower-detection.

Automatic Identification and Monitoring of Plant Diseases Using Unmanned Aerial Vehicles: A Review

Disease diagnosis is one of the major tasks for increasing food production in agriculture. Although precision agriculture (PA) takes less time and provides a more precise application of agricultural activities, the detection of disease using an Unmanned Aerial System (UAS) is a challenging task. Several Unmanned Aerial Vehicles (UAVs) and sensors have been used for this purpose. The UAVs’ platforms and their peripherals have their own limitations in accurately diagnosing plant diseases. Several types of image processing software are available for vignetting and orthorectification. The training and validation of datasets are important characteristics of data analysis. Currently, different algorithms and architectures of machine learning models are used to classify and detect plant diseases. These models help in image segmentation and feature extractions to interpret results. Researchers also use the values of vegetative indices, such as Normalized Difference Vegetative Index (NDVI), Crop Water Stress Index (CWSI), etc., acquired from different multispectral and hyperspectral sensors to fit into the statistical models to deliver results. There are still various drifts in the automatic detection of plant diseases as imaging sensors are limited by their own spectral bandwidth, resolution, background noise of the image, etc. The future of crop health monitoring using UAVs should include a gimble consisting of multiple sensors, large datasets for training and validation, the development of site-specific irradiance systems, and so on. This review briefly highlights the advantages of automatic detection of plant diseases to the growers

Artificial Neural Networks in Agriculture

Modern agriculture needs to have high production efficiency combined with a high quality of obtained products. This applies to both crop and livestock production. To meet these requirements, advanced methods of data analysis are more and more frequently used, including those derived from artificial intelligence methods. Artificial neural networks (ANNs) are one of the most popular tools of this kind. They are widely used in solving various classification and prediction tasks, for some time also in the broadly defined field of agriculture. They can form part of precision farming and decision support systems. Artificial neural networks can replace the classical methods of modelling many issues, and are one of the main alternatives to classical mathematical models. The spectrum of applications of artificial neural networks is very wide. For a long time now, researchers from all over the world have been using these tools to support agricultural production, making it more efficient and providing the highest-quality products possible

Forestry Applications of Unmanned Aerial Vehicles (UAVs) 2019

Unmanned aerial vehicles (UAVs) are new platforms that have been increasingly used in the last few years for forestry applications that benefit from the added value of flexibility, low cost, reliability, autonomy, and capability of timely provision of high-resolution data. The main adopted image-based technologies are RGB, multispectral, and thermal infrared. LiDAR sensors are becoming commonly used to improve the estimation of relevant plant traits. In comparison with other permanent ecosystems, forests are particularly affected by climatic changes due to the longevity of the trees, and the primary objective is the conservation and protection of forests. Nevertheless, forestry and agriculture involve the cultivation of renewable raw materials, with the difference that forestry is less tied to economic aspects and this is reflected by the delay in using new monitoring technologies. The main forestry applications are aimed toward inventory of resources, map diseases, species classification, fire monitoring, and spatial gap estimation. This Special Issue focuses on new technologies (UAV and sensors) and innovative data elaboration methodologies (object recognition and machine vision) for applications in forestry

High-throughput phenotyping of plant leaf morphological, physiological, and biochemical traits on multiple scales using optical sensing

Acquisition of plant phenotypic information facilitates plant breeding, sheds light on gene action, and can be applied to optimize the quality of agricultural and forestry products. Because leaves often show the fastest responses to external environmental stimuli, leaf phenotypic traits are indicators of plant growth, health, and stress levels. Combination of new imaging sensors, image processing, and data analytics permits measurement over the full life span of plants at high temporal resolution and at several organizational levels from organs to individual plants to field populations of plants. We review the optical sensors and associated data analytics used for measuring morphological, physiological, and biochemical traits of plant leaves on multiple scales. We summarize the characteristics, advantages and limitations of optical sensing and data-processing methods applied in various plant phenotyping scenarios. Finally, we discuss the future prospects of plant leaf phenotyping research. This review aims to help researchers choose appropriate optical sensors and data processing methods to acquire plant leaf phenotypes rapidly, accurately, and cost-effectively

Precision Weed Management Based on UAS Image Streams, Machine Learning, and PWM Sprayers

Weed populations in agricultural production fields are often scattered and unevenly distributed; however, herbicides are broadcast across fields evenly. Although effective, in the case of post-emergent herbicides, exceedingly more pesticides are used than necessary. A novel weed detection and control workflow was evaluated targeting Palmer amaranth in soybean (Glycine max) fields. High spatial resolution (0.4 cm) unmanned aircraft system (UAS) image streams were collected, annotated, and used to train 16 object detection convolutional neural networks (CNNs; RetinaNet, Faster R-CNN, Single Shot Detector, and YOLO v3) each trained on imagery with 0.4, 0.6, 0.8, and 1.2 cm spatial resolutions. Models were evaluated on imagery from four production fields containing approximately 7,800 weeds. The highest performing model was Faster R-CNN trained on 0.4 cm imagery (precision = 0.86, recall = 0.98, and F1-score = 0.91). A site-specific workflow leveraging the highest performing trained CNN models was evaluated in replicated field trials. Weed control (%) was compared between a broadcast treatment and the proposed site-specific workflow which was applied using a pulse-width modulated (PWM) sprayer. Results indicate no statistical (p \u3c .05) difference in weed control measured one (M = 96.22%, SD = 3.90 and M = 90.10%, SD = 9.96), two (M = 95.15%, SD = 5.34 and M = 89.64%, SD = 8.58), and three weeks (M = 88.55, SD = 11.07 and M = 81.78%, SD = 13.05) after application between broadcast and site-specific treatments, respectively. Furthermore, there was a significant (p \u3c 0.05) 48% mean reduction in applied area (m2) between broadcast and site-specific treatments across both years. Equivalent post application efficacy can be achieved with significant reductions in herbicides if weeds are targeted through site-specific applications. Site-specific weed maps can be generated and executed using accessible technologies like UAS, open-source CNNs, and PWM sprayers

Precision Weed Management Based on UAS Image Streams, Machine Learning, and PWM Sprayers

Weed populations in agricultural production fields are often scattered and unevenly distributed; however, herbicides are broadcast across fields evenly. Although effective, in the case of post-emergent herbicides, exceedingly more pesticides are used than necessary. A novel weed detection and control workflow was evaluated targeting Palmer amaranth in soybean (Glycine max) fields. High spatial resolution (0.4 cm) unmanned aircraft system (UAS) image streams were collected, annotated, and used to train 16 object detection convolutional neural networks (CNNs; RetinaNet, Faster R-CNN, Single Shot Detector, and YOLO v3) each trained on imagery with 0.4, 0.6, 0.8, and 1.2 cm spatial resolutions. Models were evaluated on imagery from four production fields containing approximately 7,800 weeds. The highest performing model was Faster R-CNN trained on 0.4 cm imagery (precision = 0.86, recall = 0.98, and F1-score = 0.91). A site-specific workflow leveraging the highest performing trained CNN models was evaluated in replicated field trials. Weed control (%) was compared between a broadcast treatment and the proposed site-specific workflow which was applied using a pulse-width modulated (PWM) sprayer. Results indicate no statistical (p \u3c .05) difference in weed control measured one (M = 96.22%, SD = 3.90 and M = 90.10%, SD = 9.96), two (M = 95.15%, SD = 5.34 and M = 89.64%, SD = 8.58), and three weeks (M = 88.55, SD = 11.07 and M = 81.78%, SD = 13.05) after application between broadcast and site-specific treatments, respectively. Furthermore, there was a significant (p \u3c 0.05) 48% mean reduction in applied area (m2) between broadcast and site-specific treatments across both years. Equivalent post application efficacy can be achieved with significant reductions in herbicides if weeds are targeted through site-specific applications. Site-specific weed maps can be generated and executed using accessible technologies like UAS, open-source CNNs, and PWM sprayers

Application of Artificial Intelligence algorithms to support decision-making in agriculture activities

Deep Learning has been successfully applied to image recognition, speech recognition, and natural language processing in recent years. Therefore, there has been an incentive to apply it in other fields as well. The field of agriculture is one of the most important in which the application of artificial intelligence algorithms, and particularly, of deep learning needs to be explored, as it has a direct impact on human well-being. In particular, there is a need to explore how deep learning models for decision-making can be used as a tool for optimal planting, land use, yield improvement, production/disease/pest control, and other activities. The vast amount of data received from sensors in smart farms makes it possible to use deep learning as a model for decision-making in this field. In agriculture, no two environments are exactly alike, which makes testing, validating, and successfully implementing such technologies much more complex than in most other sectors. Recent scientific developments in the field of deep learning, applied to agriculture, are reviewed and some challenges and potential solutions using deep learning algorithms in agriculture are discussed. Higher performance in terms of accuracy and lower inference time can be achieved, and the models can be made useful in real-world applications. Finally, some opportunities for future research in this area are suggested. The ability of artificial neural networks, specifically Long Short-Term Memory (LSTM) and Bidirectional LSTM (BLSTM), to model daily reference evapotranspiration and soil water content is investigated. The application of these techniques to predict these parameters was tested for three sites in Portugal. A single-layer BLSTM with 512 nodes was selected. Bayesian optimization was used to determine the hyperparameters, such as learning rate, decay, batch size, and dropout size. The model achieved mean square error (MSE) values ranging from 0.07 to 0.27 (mm d–1)² for ETo (Reference Evapotranspiration) and 0.014 to 0.056 (m³m–3)² for SWC (Soil Water Content), with R2 values ranging from 0.96 to 0.98. A Convolutional Neural Network (CNN) model was added to the LSTM to investigate potential performance improvement. Performance dropped in all datasets due to the complexity of the model. The performance of the models was also compared with CNN, traditional machine learning algorithms Support Vector Regression, and Random Forest. LSTM achieved the best performance. Finally, the impact of the loss function on the performance of the proposed models was investigated. The model with the mean square error (MSE) as loss function performed better than the model with other loss functions. Afterwards, the capabilities of these models and their extension, BLSTM and Bidirectional Gated Recurrent Units (BGRU) to predict end-of-season yields are investigated. The models use historical data, including climate data, irrigation scheduling, and soil water content, to estimate endof- season yield. The application of this technique was tested for tomato and potato yields at a site in Portugal. The BLSTM network outperformed the GRU, the LSTM, and the BGRU networks on the validation dataset. The model was able to capture the nonlinear relationship between irrigation amount, climate data, and soil water content and predict yield with an MSE of 0.017 to 0.039 kg/ha. The performance of the BLSTM in the test was compared with the most commonly used deep learning method called CNN, and machine learning methods including a Multi-Layer Perceptrons model and Random Forest regression. The BLSTM out-performed the other models with a R2-score between 0.97 and 0.99. The results show that analyzing agricultural data with the LSTM model improves the performance of the model in terms of accuracy. The CNN model achieved the second-best performance. Therefore, the deep learning model has a remarkable ability to predict the yield at the end of the season. Additionally, a Deep Q-Network was trained for irrigation scheduling. The agent was trained to schedule irrigation for a tomato field in Portugal. Two LSTM models trained previously were used as the agent environment. One predicts the total water in the soil profile on the next day. The other one was employed to estimate the yield based on the environmental condition during a season and then measure the net return. The agent uses this information to decide the following irrigation amount. LSTM and CNN networks were used to estimate the Q-table during training. Unlike the LSTM model, the ANN and the CNN could not estimate the Qtable, and the agent’s reward decreased during training. The comparison of the performance of the model was done with fixed-base irrigation and threshold-based irrigation. The trained model increased productivity by 11% and decreased water consumption by 20% to 30% compared to the fixed method. Also, an on-policy model, Advantage Actor–Critic (A2C), was implemented to compare irrigation scheduling with Deep Q-Network for the same tomato crop. The results show that the on-policy model A2C reduced water consumption by 20% compared to Deep Q-Network with a slight change in the net reward. These models can be developed to be applied to other cultures with high importance in Portugal, such as fruit, cereals, and grapevines, which also have large water requirements. The models developed along this thesis can be re-evaluated and trained with historical data from other cultures with high production in Portugal, such as fruits, cereals, and grapes, which also have high water demand, to create a decision support and recommendation system that tells farmers when and how much to irrigate. This system helps farmers avoid wasting water without reducing productivity. This thesis aims to contribute to the future steps in the development of precision agriculture and agricultural robotics. The models developed in this thesis are relevant to support decision-making in agricultural activities, aimed at optimizing resources, reducing time and costs, and maximizing production.Nos últimos anos, a técnica de aprendizagem profunda (Deep Learning) foi aplicada com sucesso ao reconhecimento de imagem, reconhecimento de fala e processamento de linguagem natural. Assim, tem havido um incen tivo para aplicá-la também em outros sectores. O sector agrícola é um dos mais importantes, em que a aplicação de algoritmos de inteligência artificial e, em particular, de deep learning, precisa ser explorada, pois tem impacto direto no bem-estar humano. Em particular, há uma necessidade de explorar como os modelos de aprendizagem profunda para a tomada de decisão podem ser usados como uma ferramenta para cultivo ou plantação ideal, uso da terra, melhoria da produtividade, controlo de produção, de doenças, de pragas e outras atividades. A grande quantidade de dados recebidos de sensores em explorações agrícolas inteligentes (smart farms) possibilita o uso de deep learning como modelo para tomada de decisão nesse campo. Na agricultura, não há dois ambientes iguais, o que torna o teste, a validação e a implementação bem-sucedida dessas tecnologias muito mais complexas do que na maioria dos outros setores. Desenvolvimentos científicos recentes no campo da aprendizagem profunda aplicada à agricultura, são revistos e alguns desafios e potenciais soluções usando algoritmos de aprendizagem profunda na agricultura são discutidos. Maior desempenho em termos de precisão e menor tempo de inferência pode ser alcançado, e os modelos podem ser úteis em aplicações do mundo real. Por fim, são sugeridas algumas oportunidades para futuras pesquisas nesta área. A capacidade de redes neuronais artificiais, especificamente Long Short-Term Memory (LSTM) e LSTM Bidirecional (BLSTM), para modelar a evapotranspiração de referência diária e o conteúdo de água do solo é investigada. A aplicação destas técnicas para prever estes parâmetros foi testada em três locais em Portugal. Um BLSTM de camada única com 512 nós foi selecionado. A otimização bayesiana foi usada para determinar os hiperparâmetros, como taxa de aprendizagem, decaimento, tamanho do lote e tamanho do ”dropout”. O modelo alcançou os valores de erro quadrático médio na faixa de 0,014 a 0,056 e R2 variando de 0,96 a 0,98. Um modelo de Rede Neural Convolucional (CNN – Convolutional Neural Network) foi adicionado ao LSTM para investigar uma potencial melhoria de desempenho. O desempenho decresceu em todos os conjuntos de dados devido à complexidade do modelo. O desempenho dos modelos também foi comparado com CNN, algoritmos tradicionais de aprendizagem máquina Support Vector Regression e Random Forest. O LSTM obteve o melhor desempenho. Por fim, investigou-se o impacto da função de perda no desempenho dos modelos propostos. O modelo com o erro quadrático médio (MSE) como função de perda teve um desempenho melhor do que o modelo com outras funções de perda. Em seguida, são investigadas as capacidades desses modelos e sua extensão, BLSTM e Bidirectional Gated Recurrent Units (BGRU) para prever os rendimentos da produção no final da campanha agrícola. Os modelos usam dados históricos, incluindo dados climáticos, calendário de rega e teor de água do solo, para estimar a produtividade no final da campanha. A aplicação desta técnica foi testada para os rendimentos de tomate e batata em um local em Portugal. A rede BLSTM superou as redes GRU, LSTM e BGRU no conjunto de dados de validação. O modelo foi capaz de captar a relação não linear entre dotação de rega, dados climáticos e teor de água do solo e prever a produtividade com um MSE variando de 0,07 a 0,27 (mm d–1)² para ETo (Evapotranspiração de Referência) e de 0,014 a 0,056 (m³m–3)² para SWC (Conteúdo de Água do Solo), com valores de R2 variando de 0,96 a 0,98. O desempenho do BLSTM no teste foi comparado com o método de aprendizagem profunda CNN, e métodos de aprendizagem máquina, incluindo um modelo Multi-Layer Perceptrons e regressão Random Forest. O BLSTM superou os outros modelos com um R2 entre 97% e 99%. Os resultados mostram que a análise de dados agrícolas com o modelo LSTM melhora o desempenho do modelo em termos de precisão. O modelo CNN obteve o segundo melhor desempenho. Portanto, o modelo de aprendizagem profunda tem uma capacidade notável de prever a produtividade no final da campanha. Além disso, uma Deep Q-Network foi treinada para programação de irrigação para a cultura do tomate. O agente foi treinado para programar a irrigação de uma plantação de tomate em Portugal. Dois modelos LSTM treinados anteriormente foram usados como ambiente de agente. Um prevê a água total no perfil do solo no dia seguinte. O outro foi empregue para estimar a produtividade com base nas condições ambientais durante uma o ciclo biológico e então medir o retorno líquido. O agente usa essas informações para decidir a quantidade de irrigação. As redes LSTM e CNN foram usadas para estimar a Q-table durante o treino. Ao contrário do modelo LSTM, a RNA e a CNN não conseguiram estimar a tabela Q, e a recompensa do agente diminuiu durante o treino. A comparação de desempenho do modelo foi realizada entre a irrigação com base fixa e a irrigação com base em um limiar. A aplicação das doses de rega preconizadas pelo modelo aumentou a produtividade em 11% e diminuiu o consumo de água em 20% a 30% em relação ao método fixo. Além disso, um modelo dentro da táctica, Advantage Actor–Critic (A2C), é foi implementado para comparar a programação de irrigação com o Deep Q-Network para a mesma cultura de tomate. Os resultados mostram que o modelo de táctica A2C reduziu o consumo de água consumo em 20% comparado ao Deep Q-Network com uma pequena mudança na recompensa líquida. Estes modelos podem ser desenvolvidos para serem aplicados a outras culturas com elevada produção em Portugal, como a fruta, cereais e vinha, que também têm grandes necessidades hídricas. Os modelos desenvolvidos ao longo desta tese podem ser reavaliados e treinados com dados históricos de outras culturas com elevada importância em Portugal, tais como frutas, cereais e uvas, que também têm elevados consumos de água. Assim, poderão ser desenvolvidos sistemas de apoio à decisão e de recomendação aos agricultores de quando e quanto irrigar. Estes sistemas poderão ajudar os agricultores a evitar o desperdício de água sem reduzir a produtividade. Esta tese visa contribuir para os passos futuros na evolução da agricultura de precisão e da robótica agrícola. Os modelos desenvolvidos ao longo desta tese são relevantes para apoiar a tomada de decisões em atividades agrícolas, direcionadas à otimização de recursos, redução de tempo e custos, e maximização da produção.Centro-01-0145-FEDER000017-EMaDeS-Energy, Materials, and Sustainable Development, co-funded by the Portugal 2020 Program (PT 2020), within the Regional Operational Program of the Center (CENTRO 2020) and the EU through the European Regional Development Fund (ERDF). Fundação para a Ciência e a Tecnologia (FCT—MCTES) also provided financial support via project UIDB/00151/2020 (C-MAST). It was also supported by the R&D Project BioDAgro – Sistema operacional inteligente de informação e suporte á decisão em AgroBiodiversidade, project PD20-00011, promoted by Fundação La Caixa and Fundação para a Ciência e a Tecnologia, taking place at the C-MAST - Centre for Mechanical and Aerospace Sciences and Technology, Department of Electromechanical Engineering of the University of Beira Interior, Covilhã, Portugal

Conteo de gránulos usando redes neuronales tipo U-Net y componentes conectados

This research develops a methodology to automate the process of counting the number of granules that remains in a toilet after being flushed (ASME A112.19.2-2018/CSA B45.1-18). This work integrates a U-Net convolutional network with a variation of the connected component algorithm. The training set consisted of 3678 images. Results show an accuracy above 98% between 0 and 180 granules. The methodology has been implemented in the production line.Este trabajo desarrolla una metodología para automatizar el conteo de gránulos remanentes en una taza sanitaria (prueba ASME A112.19.2-2018/CSA B45.1-18). Esta metodología integra una red convolucional U-Net que fue entrenada con 3678 imágenes y una variación del algoritmo de componentes conectados. Los resultados arrojan una precisión superior al 98 % para valores entre 0 y 180 gránulos. La metodología se implementó en la línea de producción

Explainable deep learning in plant phenotyping

The increasing human population and variable weather conditions, due to climate change, pose a threat to the world's food security. To improve global food security, we need to provide breeders with tools to develop crop cultivars that are more resilient to extreme weather conditions and provide growers with tools to more effectively manage biotic and abiotic stresses in their crops. Plant phenotyping, the measurement of a plant's structural and functional characteristics, has the potential to inform, improve and accelerate both breeders' selections and growers' management decisions. To improve the speed, reliability and scale of plant phenotyping procedures, many researchers have adopted deep learning methods to estimate phenotypic information from images of plants and crops. Despite the successful results of these image-based phenotyping studies, the representations learned by deep learning models remain difficult to interpret, understand, and explain. For this reason, deep learning models are still considered to be black boxes. Explainable AI (XAI) is a promising approach for opening the deep learning model's black box and providing plant scientists with image-based phenotypic information that is interpretable and trustworthy. Although various fields of study have adopted XAI to advance their understanding of deep learning models, it has yet to be well-studied in the context of plant phenotyping research. In this review article, we reviewed existing XAI studies in plant shoot phenotyping, as well as related domains, to help plant researchers understand the benefits of XAI and make it easier for them to integrate XAI into their future studies. An elucidation of the representations within a deep learning model can help researchers explain the model's decisions, relate the features detected by the model to the underlying plant physiology, and enhance the trustworthiness of image-based phenotypic information used in food production systems