Hybrid features and ensembles of convolution neural networks for weed detection

Weeds compete with plants for sunlight, nutrients and water. Conventional weed management involves spraying of herbicides to the entire crop which increases the cost of cultivation, decreasing the quality of the crop, in turn affecting human health. Precise automatic spraying of the herbicides on weeds has been in research and use. This paper discusses automatic weed detection using hybrid features which is generated by extracting the deep features from convolutional neural network (CNN) along with the texture and color features. The color and texture features are extracted by color moments, gray level co-occurrence matrix (GLCM) and Gabor wavelet transform. The proposed hybrid features are classified by Bayesian optimized support vector machine (BO-SVM) classifier. The experimental results read that the proposed hybrid features yield a maximum accuracy of 95.83%, higher precision, sensitivity and F-score. A performance analysis of the proposed hybrid features with BO-SVM classifier in terms of the evaluation parameters is made using the images from crop weed field image dataset

Local Binary Pattern based algorithms for the discrimination and detection of crops and weeds with similar morphologies

In cultivated agricultural fields, weeds are unwanted species that compete with the crop plants for nutrients, water, sunlight and soil, thus constraining their growth. Applying new real-time weed detection and spraying technologies to agriculture would enhance current farming practices, leading to higher crop yields and lower production costs. Various weed detection methods have been developed for Site-Specific Weed Management (SSWM) aimed at maximising the crop yield through efficient control of weeds. Blanket application of herbicide chemicals is currently the most popular weed eradication practice in weed management and weed invasion. However, the excessive use of herbicides has a detrimental impact on the human health, economy and environment. Before weeds are resistant to herbicides and respond better to weed control strategies, it is necessary to control them in the fallow, pre-sowing, early post-emergent and in pasture phases. Moreover, the development of herbicide resistance in weeds is the driving force for inventing precision and automation weed treatments. Various weed detection techniques have been developed to identify weed species in crop fields, aimed at improving the crop quality, reducing herbicide and water usage and minimising environmental impacts. In this thesis, Local Binary Pattern (LBP)-based algorithms are developed and tested experimentally, which are based on extracting dominant plant features from camera images to precisely detecting weeds from crops in real time. Based on the efficient computation and robustness of the first LBP method, an improved LBP-based method is developed based on using three different LBP operators for plant feature extraction in conjunction with a Support Vector Machine (SVM) method for multiclass plant classification. A 24,000-image dataset, collected using a testing facility under simulated field conditions (Testbed system), is used for algorithm training, validation and testing. The dataset, which is published online under the name “bccr-segset”, consists of four subclasses: background, Canola (Brassica napus), Corn (Zea mays), and Wild radish (Raphanus raphanistrum). In addition, the dataset comprises plant images collected at four crop growth stages, for each subclass. The computer-controlled Testbed is designed to rapidly label plant images and generate the “bccr-segset” dataset. Experimental results show that the classification accuracy of the improved LBP-based algorithm is 91.85%, for the four classes. Due to the similarity of the morphologies of the canola (crop) and wild radish (weed) leaves, the conventional LBP-based method has limited ability to discriminate broadleaf crops from weeds. To overcome this limitation and complex field conditions (illumination variation, poses, viewpoints, and occlusions), a novel LBP-based method (denoted k-FLBPCM) is developed to enhance the classification accuracy of crops and weeds with similar morphologies. Our contributions include (i) the use of opening and closing morphological operators in pre-processing of plant images, (ii) the development of the k-FLBPCM method by combining two methods, namely, the filtered local binary pattern (LBP) method and the contour-based masking method with a coefficient k, and (iii) the optimal use of SVM with the radial basis function (RBF) kernel to precisely identify broadleaf plants based on their distinctive features. The high performance of this k-FLBPCM method is demonstrated by experimentally attaining up to 98.63% classification accuracy at four different growth stages for all classes of the “bccr-segset” dataset. To evaluate performance of the k-FLBPCM algorithm in real-time, a comparison analysis between our novel method (k-FLBPCM) and deep convolutional neural networks (DCNNs) is conducted on morphologically similar crops and weeds. Various DCNN models, namely VGG-16, VGG-19, ResNet50 and InceptionV3, are optimised, by fine-tuning their hyper-parameters, and tested. Based on the experimental results on the “bccr-segset” dataset collected from the laboratory and the “fieldtrip_can_weeds” dataset collected from the field under practical environments, the classification accuracies of the DCNN models and the k-FLBPCM method are almost similar. Another experiment is conducted by training the algorithms with plant images obtained at mature stages and testing them at early stages. In this case, the new k-FLBPCM method outperformed the state-of-the-art CNN models in identifying small leaf shapes of canola-radish (crop-weed) at early growth stages, with an order of magnitude lower error rates in comparison with DCNN models. Furthermore, the execution time of the k-FLBPCM method during the training and test phases was faster than the DCNN counterparts, with an identification time difference of approximately 0.224ms per image for the laboratory dataset and 0.346ms per image for the field dataset. These results demonstrate the ability of the k-FLBPCM method to rapidly detect weeds from crops of similar appearance in real time with less data, and generalize to different size plants better than the CNN-based methods

Semantic Segmentation based deep learning approaches for weed detection

Global increase in herbicide use to control weeds has led to issues such as evolution of herbicide-resistant weeds, off-target herbicide movement, etc. Precision agriculture advocates Site Specific Weed Management (SSWM) application to achieve precise and right amount of herbicide spray and reduce off-target herbicide movement. Recent advancements in Deep Learning (DL) have opened possibilities for adaptive and accurate weed recognitions for field based SSWM applications with traditional and emerging spraying equipment; however, challenges exist in identifying the DL model structure and train the model appropriately for accurate and rapid model applications over varying crop/weed growth stages and environment. In our study, an encoder-decoder based DL architecture was proposed that performs pixel-wise Semantic Segmentation (SS) classifications of crop, soil, and weed patches in the fields. The objective of this study was to develop a robust weed detection algorithm using DL techniques that can accurately and reliably locate weed infestations in low altitude Unmanned Aerial Vehicle (UAV) imagery with acceptable application speed. Two different encoder-decoder based SS models of LinkNet and UNet were developed using transfer learning techniques. We performed various measures such as backpropagation optimization and refining of the dataset used for training to address the class-imbalance problem which is a common issue in developing weed detection models. It was found that LinkNet model with ResNet18 as the encoder section and use of ‘Focal loss’ loss function was able to achieve the highest mean and class-wise Intersection over Union scores for different class categories while performing predictions on unseen dataset. The developed state-of-art model did not require a large amount of data during training and the techniques used to develop the model in our study provides a propitious opportunity that performs better than the existing SS based weed detections models. The proposed model integrates a futuristic approach to develop a model that could be used for weed detection on aerial imagery from UAV and perform real-time SSWM applications Advisor: Yeyin Sh

Development of soft computing and applications in agricultural and biological engineering

Soft computing is a set of “inexact” computing techniques, which are able to model and analyze very complex problems. For these complex problems, more conventional methods have not been able to produce cost-effective, analytical, or complete solutions. Soft computing has been extensively studied and applied in the last three decades for scientific research and engineering computing. In agricultural and biological engineering, researchers and engineers have developed methods of fuzzy logic, artificial neural networks, genetic algorithms, decision trees, and support vector machines to study soil and water regimes related to crop growth, analyze the operation of food processing, and support decision-making in precision farming. This paper reviews the development of soft computing techniques. With the concepts and methods, applications of soft computing in the field of agricultural and biological engineering are presented, especially in the soil and water context for crop management and decision support in precision agriculture. The future of development and application of soft computing in agricultural and biological engineering is discussed

Development of Modified CNN Algorithm for Agriculture Product: A Research Review

Now a day, with the increase in world population, the demand for agricultural products is also increased. Modern days electronic technologies combined with machine vision techniques have become a good resource for precise weed and crop detection in the field. It is becoming prominent in precision agriculture and also supporting site-specific weed management. By reviewing as there are so many different kinds of weed detection algorithms that were already used in the weed removal process or in agriculture. By the comparative study of research papers on weed detection. In this paper, we have suggested advanced and improved algorithms which take care of most of the limitations of previous work. The main goal of this review is to study the different types of algorithms used to detect weeds present in crops for automated systems in agriculture. This paper used a method that is based on a convolutional neural network model, VGG16, to identify images of weeds. As the basic network, VGG16 has very good classification performance, and it is relatively easy to modify. Download the weed dataset. This image dataset has 15336 segments, being 3249 of soil, 7376 soybeans, 3520 grass, and 1191 broadleaf weeds. Our model fixes the first 16 layers of  VGG16 parameters for layer-by-layer automatic extraction of features, adding an average pooling layer, convolution layer, Dropout layer, fully connected layer, and softmax for classifiers. The results show that the final model performs well in the classification effect of 4 classes. The accuracy is 97.76 %. We will compare our result with the CNN model. It provides an accurate and reliable judgment basis for quantitative chemical pesticide spraying. The results of this study can provide an overview of the use of CNN-based techniques for weed detection

Artificial Neural Networks in Agriculture

Modern agriculture needs to have high production efficiency combined with a high quality of obtained products. This applies to both crop and livestock production. To meet these requirements, advanced methods of data analysis are more and more frequently used, including those derived from artificial intelligence methods. Artificial neural networks (ANNs) are one of the most popular tools of this kind. They are widely used in solving various classification and prediction tasks, for some time also in the broadly defined field of agriculture. They can form part of precision farming and decision support systems. Artificial neural networks can replace the classical methods of modelling many issues, and are one of the main alternatives to classical mathematical models. The spectrum of applications of artificial neural networks is very wide. For a long time now, researchers from all over the world have been using these tools to support agricultural production, making it more efficient and providing the highest-quality products possible

Performances of the LBP based algorithm over CNN models for detecting crops and weeds with similar morphologies

Weed invasions pose a threat to agricultural productivity. Weed recognition and detection play an important role in controlling weeds. The challenging problem of weed detection is how to discriminate between crops and weeds with a similar morphology under natural field conditions such as occlusion, varying lighting conditions, and different growth stages. In this paper, we evaluate a novel algorithm, filtered Local Binary Patterns with contour masks and coefficient k (k-FLBPCM), for discriminating between morphologically similar crops and weeds, which shows significant advantages, in both model size and accuracy, over state-of-the-art deep convolutional neural network (CNN) models such as VGG-16, VGG-19, ResNet-50 and InceptionV3. The experimental results on the “bccr-segset” dataset in the laboratory testbed setting show that the accuracy of CNN models with fine-tuned hyper-parameters is slightly higher than the k-FLBPCM method, while the accuracy of the k-FLBPCM algorithm is higher than the CNN models (except for VGG-16) for the more realistic “fieldtrip_can_weeds” dataset collected from real-world agricultural fields. However, the CNN models require a large amount of labelled samples for the training process. We conducted another experiment based on training with crop images at mature stages and testing at early stages. The k-FLBPCM method outperformed the state-of-the-art CNN models in recognizing small leaf shapes at early growth stages, with error rates an order of magnitude lower than CNN models for canola–radish (crop–weed) discrimination using a subset extracted from the “bccr-segset” dataset, and for the “mixed-plants” dataset. Moreover, the real-time weed–plant discrimination time attained with the k-FLBPCM algorithm is approximately 0.223 ms per image for the laboratory dataset and 0.346 ms per image for the field dataset, and this is an order of magnitude faster than that of CNN models

Weeds detection efficiency through different convolutional neural networks technology

The preservation of the environment has become a priority and a subject that is receiving more and more attention. This is particularly important in the field of precision agriculture, where pesticide and herbicide use has become more controlled. In this study, we propose to evaluate the ability of the deep learning (DL) and convolutional neural network (CNNs) technology to detect weeds in several types of crops using a perspective and proximity images to enable localized and ultra-localized herbicide spraying in the region of Beni Mellal in Morocco. We studied the detection of weeds through six recent CNN known for their speed and precision, namely, VGGNet (16 and 19), GoogLeNet (Inception V3 and V4) and MobileNet (V1 and V2). The first experiment was performed with the CNNs architectures from scratch and the second experiment with their pre-trained versions. The results showed that Inception V4 achieved the highest precision with a rate of 99.41% and 99.51% on the mixed image sets and for its version from scratch and its pre-trained version respectively, and that MobileNet V2 was the fastest and lightest with its size of 14 MB

Finding spectral features for the early identification of biotic stress in plants

Early detection of biotic stress in plants is vital for precision crop protection, but hard to achieve. Prediction of plant diseases or weeds at an early stage has significant influence on the extent and effectiveness of crop protection measures. The precise measure depends on specific weeds and plant diseases and their economic thresholds. Weeds and plant diseases at an early stage, however, are difficult to identify. Non-invasive optical sensors with high resolution are promising for early detection of biotic stress. The data of these sensors, e.g. hyperspectral or fluorescence signatures, contain relevant information about the occurrence of pathogens. Shape parameters, derived from bispectral images, have enormous potential for an early identification of weeds in crops. The analysis of this high dimensional data for an identification of weeds and pathogens as early as possible is demanding as the sensor signal is affected by many influencing factors. Nevertheless, advanced methods of machine learning facilitate the interpretation of these signals. Whereas traditional statistics estimate the posterior probability of the class by probability distribution, machine learning methods provide algorithms for optimising prediction accuracy by the discriminant function. Machine learning methods with robust training algorithms play a key role in handling non-linear classification problems. This thesis presents an approach which integrates modern sensor techniques and advanced machine learning methods for an early detection and differentiation of plant diseases and weeds. Support vector machines (SVMs) equipped with non-linear kernels prove as effective and robust classifiers. Furthermore, it is shown that even a presymptomatic identification based on the combination of spectral vegetation indices is realised. Using well-established data analysis methods of this scientific field, this has not achieved so far. Identifying disease specific features from the underlying original high dimensional sensor data selection is conducted. The high dimensionality of data affords a careful selection of relevant and non-redundant features depending on classification problem and feature properties. In the case of fluorescence signatures an extraction of new features is necessary. In this context modelling of signal noise by an analytical description of the spectral signature improves the accuracy of classification substantially. In the case of weed discrimination accuracy is improved by exploiting the hierarchy of weed species. This thesis outlines the potential of SVMs, feature construction and feature selection for precision crop protection. A problem-specific extraction and selection of relevant features, in combination with task-oriented classification methods, is essential for robust identification of pathogens and weeds as early as possible.Früherkennung von biotischem Pflanzenstress ist für den Präzisionspflanzenschutz wesentlich, aber schwierig zu erreichen. Die Vorhersage von Pflanzenkrankheiten und Unkräutern in einem frühen Entwicklungsstadium hat signifikanten Einfluss auf das Ausmaß und die Effektivität einer Pflanzenschutzmaßnahme. Aufgrund der Abhängigkeit einer Maßnahme von der Art der Pflanzenkrankheit oder des Unkrauts und ihrer ökonomischer Schadschwelle ist eine präzise Identifizierung der Schadursache essentiell, aber gerade im Frühstadium durch die Ähnlichkeit der Schadbilder problematisch. Nicht-invasive optische Sensoren mit hoher Auflösung sind vielversprechend für eine Früherkennung von biotischem Pflanzenstress. Daten dieser Sensoren, beispielsweise Hyperspektral- oder Fluoreszenzspektren, enthalten relevante Informationen über das Auftreten von Pathogenen; Formparameter, abgeleitet aus bispektralen Bildern, zeigen großes Potential für die Früherkennung von Unkräutern in Kulturpflanzen. Die Analyse dieser hochdimensionalen Sensordaten unter Berücksichtigung vielfältiger Faktoren ist eine anspruchsvolle Herausforderung. Moderne Methoden des maschinellen Lernens bieten hier zielführende Möglichkeiten. Während die traditionelle Statistik die a-posteriori Wahrscheinlichkeit der Klasse basierend auf Wahrscheinlichkeitsverteilungen schätzt, verwenden maschinelle Lernverfahren Algorithmen für eine Optimierung der Vorhersagegenauigkeit auf Basis diskriminierender Funktionen. Grundlage zur Bearbeitung dieser nicht-linearen Klassi kationsprobleme sind robuste maschinelle Lernverfahren. Die vorliegende Dissertationsschrift zeigt, dass die Integration moderner Sensortechnik mit fortgeschrittenen Methoden des maschinellen Lernens eine Erkennung und Differenzierung von Pflanzenkrankheiten und Unkräutern ermöglicht. Einen wesentlichen Beitrag für eine effektive und robuste Klassifikation leisten Support Vektor Maschinen (SVMs) mit nicht-linearen Kernels. Weiterhin wird gezeigt, dass SVMs auf Basis spektraler Vegetationsindizes die Detektion von Pflanzenkrankheiten noch vor Auftreten visuell wahrnehmbarer Symptome ermöglichen. Dies wurde mit bekannten Verfahren noch nicht erreicht. Zur Identifikation krankheitsspezifischer Merkmale aus den zugrunde liegenden originären hochdimensionalen Sensordaten wurden Merkmale konstruiert und selektiert. Die Selektion ist sowohl vom Klassifikationsproblem als auch von den Eigenschaften der Merkmale abhängig. Im Fall von Fluoreszenzspektren war eine Extraktion von neuen Merkmalen notwendig. In diesem Zusammenhang trägt die Modellierung des Signalrauschens durch eine analytische Beschreibung der spektralen Signatur zur deutlichen Verbesserung der Klassifikationsgenauigkeit bei. Im Fall der Differenzierung von unterschiedlichen Unkräutern erhöht die Ausnutzung der Hierarchie der Unkrautarten die Genauigkeit signifikant. Diese Arbeit zeigt das Potential von Support Vektor Maschinen, Merkmalskonstruktion und Selektion für den Präzisionspflanzenschutz. Eine problemspezifische Extraktion und Selektion relevanter Merkmale in Verbindung mit sachbezogenen Klassifikationsmethoden ermöglichen eine robuste Identifikation von Pathogenen und Unkräutern zu einem sehr frühen Zeitpunkt